File tree
270 files changed
+11979
-71
lines changed- 000 Zettelkasten
- 2D Convolutions
- Are less inductive biases better or worse?
- Bit Palettization
- Block Expansion
- Convergence rate and Hessian spectra
- Depthwise separable convolutions
- Do Vision Foundation models exist?
- Effect of weight symmetries on training dynamics
- Equivariance Initialization
- Grokking
- Group Axioms
- Group direct product
- Hardware-specific structured pruning
- Input-dependent convolutions
- K-Means-based Quantization
- KV Cache
- LoRa Adapter
- LoRa Adapters
- Masked Image Modelling
- Maximal pruning and functional recovery
- Multiple global minima
- Non-translationally equivariant convolutions
- Priors over Neural Network weights
- Representation (Group Theory)
- 100 Reference notes
- 101 Literature
- A Brief Review of Hypernetworks in Deep Learning
- A ConvNet for the 2020s
- A Hierarchy of Graph Neural Networks Based on Learnable Local Features
- A general theory of correct, incorrect, and extrinsic equivariance
- A survey of quantization methods for efficient neural network inference
- Adapting Vision Foundation Models for Plant Phenotyping
- An Image is Worth More Than 16x16 Patches - Exploring Transformers on Individual Pixels
- An Investigation into Neural Net Optimization via Hessian Eigenvalue Density
- An image is worth 16x16 words - Transformers for image recognition at scale
- Approximately equivariant networks for imperfectly symmetric dynamics
- Approximation-Generalization Trade-offs under (Approximate) Group Equivariance
- Autoequivariant Network Search via Group Decomposition
- Block Transformer - Global-to-Local Language Modeling for Fast Inference
- BoxeR - Box-Attention for 2D and 3D Transformers
- Building on Efficient Foundations - Effectively Training LLMs with Structured Feedforward Layers
- CKConv - Continuous Kernel Convolution For Sequential Data
- Color Equivariant Convolutional Networks
- Color Space Transformation Network
- ConViT - Improving Vision Transformers with Soft Convolutional Inductive Biases
- DETRs Beat YOLOs on Real-time Object Detection
- DINOv2 - Learning Robust Visual Features without Supervision
- Deep Learning Book
- DenseNets Reloaded - Paradigm Shift Beyond ResNets and ViTs
- Discovering Symmetry Breaking in Physical Systems with Relaxed Group Convolution
- Early Convolutions Help Transformers See Better
- Efficient Equivariant Transfer Learning from Pretrained Models
- Efficient Modulation for Vision Networks
- EfficientViT-SAM - Accelerated Segment Anything Model Without Accuracy Loss
- Emergent Equivariance in Deep Ensembles
- Emerging Properties in Self-Supervised Vision Transformers
- End-to-End Object Detection with Transformers
- Equi-Tuning - Group Equivariant Fine-Tuning of Pretrained Models
- Equivariance with Learned Canonicalization Functions
- Equivariance-aware architectural optimization of neural networks
- Exploiting Redundancy - Separable Group Convolutional Networks on Lie Groups
- Exploring Plain Vision Transformer Backbones for Object Detection
- Fast, Expressive SE(n) Equivariant Networks through Weight-Sharing in Position-Orientation Space
- FlexiViT - One Model for All Patch Sizes
- G-SGD - Optimizing ReLU Neural Networks in its Positively Scale-Invariant Space
- Grokked Transformers are Implicit Reasoners - A Mechanistic Journey to the Edge of Generalization
- Harmonics of Learning - Universal Fourier Features Emerge in Invariant Networks
- How do vision transformers work?
- Improving Convergence and Generalization Using Parameter Symmetries
- In Search of Projectively Equivariant Networks
- Knowledge Transfer from Vision Foundation Models for Efficient Training of Small Task-specific Models
- LRP-QViT - Mixed-Precision Vision Transformer Quantization via Layer-wise Relevance Propagation
- Learned Gridification for Efficient Point Cloud Processing
- Learning Partial Equivariances from Data
- Learning both Weights and Connections for Efficient Neural Networks
- Learning with Unmasked Tokens Drives Stronger Vision Learners
- LoRA - Low-Rank Adaptation of Large Language Models
- Mamba - Linear-Time Sequence Modeling with Selective State Spaces
- Mixture of LoRa Experts
- MobileCLIP - Fast Image-Text Models through Multi-Modal Reinforced Training
- MobileViT - light-weight, general-purpose, and mobile-friendly vision transformer
- Model Compression in Practice - Lessons Learned from Practitioners Creating On-device Machine Learning Experiences
- Neural Mechanics - Symmetry and Broken Conservation Laws in Deep Learning Dynamics
- On the Relationship between Self-Attention and Convolutional Layers
- On the Symmetries of Deep Learning Models and their Internal Representations
- OpenELM - An Efficient Language Model Family with Open-source Training and Inference Framework
- Optimal Brain Damage
- Optimization Dynamics of Equivariant and Augmented Neural Networks
- Parameter Efficient Fine-tuning of Self-supervised ViTs without Catastrophic Forgetting
- Parameter-Efficient Fine-Tuning for Pre-Trained Vision Models - A Survey
- Progress measures for grokking via mechanistic interpretability
- Provably Strict Generalisation Benefit for Equivariant Models
- ProxylessNAS - Direct Neural Architecture Search on Target Task and Hardware
- R-MAE - Regions Meet Masked Autoencoders
- Refusal in Language Models Is Mediated by a Single Direction
- Relaxed Octahedral Group Convolution for Learning Symmetry Breaking in 3D Physical Systems
- Relaxing Equivariance Constraints with Non-stationary Continuous Filters
- Retrospective - EIE - Efficient Inference Engine onSparse and Compressed Neural Network
- Rewrite the Stars
- SAM-CLIP - Merging Vision Foundation Models towards Semantic and Spatial Understanding
- Scaling (Down) CLIP - A Comprehensive Analysis of Data, Architecture, and Training Strategies
- Segment Anything
- Self-Supervised Detection of Perfect and Partial Input-Dependent Symmetries
- SimPLR - A Simple and Plain Transformer for Scaling-Efficient Object Detection and Segmentation
- Simultaneous linear connectivity of neural networks modulo permutation
- Stand-Alone Self-Attention in Vision Models
- Surgical Fine-Tuning Improves Adaptation to Distribution Shifts
- Surgical-DINO - Adapter Learning of Foundation Models for Depth Estimation in Endoscopic Surgery
- Symmetries in Overparametrized Neural Networks - A Mean-Field View
- Talaria - Interactively Optimizing Machine Learning Models for Efficient Inference
- The Empirical Impact of Neural Parameter Symmetries, or Lack Thereof
- The Lie derivative for measuring learned equivariance
- The Unreasonable Ineffectiveness of the Deeper Layers
- TiC-CLIP - Continual Training of CLIP models
- Understanding Deep Learning - Chapter 10
- Understanding Deep Learning - Chapter 20
- Understanding symmetries in deep networks
- Using Degeneracy in the Loss Landscape for Mechanistic Interpretability
- Vision Mamba - Efficient Visual Representation Learning with Bidirectional State Space Model
- Vision Transformers Need Registers
- What Do Self-Supervised Vision Transformers Learn?
- 102 Authors
- Alex Flinth
- Alexander Kirillov
- Alexey Dosovitskiy
- Ananya Kumar
- Andreas Loukas
- Andreas Savakis
- Annie S. Chen
- Ardavan Pedram
- Armand Joulin
- Attila Lengyel
- Boshi Wang
- Byeongho Heo
- Caglar Gulcehre
- Cees G. M. Snoek
- Chelsea Finn
- Daniel M. Roy
- David M. Knigge
- David W. Romero
- Donghyun Kim
- Dongyoon Han
- Duy-Kien Nguyen
- Edward J. Hu
- Eric Mintun
- Erik J. Bekkers
- Fahim Tajwar
- Fartash Faghri
- Francisco Massa
- Furu Wei
- Gabriel Synnaeve
- Gintare Karolina Dziugaite
- Hadi Pouransari
- Han Cai
- Hanzi Mao
- Haoxiang Wang
- Hervé Jegou
- Huaxiu Yao
- Hugo Touvron
- Huizi Mao
- Ishan Misra
- Jan E. Gerken
- Javier Maass Martinez
- Jean-Baptiste Cordonnier
- Jeff Pool
- Jing Pu
- Joaquin Fontbona
- John Denker
- John Tran
- Julien Mairal
- Kaiming He
- Lawrence Chan
- Lucius Bushnaq
- Mahmoud Assran
- Marc Finzi
- Mark A. Horowitz
- Martin Jaggi
- Martin R. Oswald
- Mathilde Caron
- Maxime Oquab
- Mehrdad Farajtabar
- Mohammad Rastegari
- Namuk Park
- Neel Nanda
- Nicolas Carion
- Nicolas Usunier
- Oncel Tuzel
- Patrick Forré
- Pavan Kumar Anasosalu Vasu
- Percy Liang
- Piotr Bojanowski
- Raviteja Vemulapalli
- Razvan Pascanu
- Robin Walters
- Rose Yu
- Ross Girshick
- Rui Wang
- Sachin Mehta
- Sanghyuk Chun
- Sara Solla
- Sergey Zagoruyko
- Shaohan Huang
- Simon J.D. Prince
- Skander Moalla
- Song Han
- Songkuk Kim
- Sourya Basu
- Stéphane d'Ascoli
- Taekyung Kim
- Tete Xiao
- Tom Lieberum
- Vaibhav Aggarwal
- William J. Dally
- Xiang Yue
- Xingyu Liu
- Xinlei Chen
- Xiuying Wei
- Xu Ma
- Xun Wu
- Yanghao Li
- Yann LeCun
- Yelong Shen
- Yoonho Lee
- Zhuoyang Zhang
- 103 Affiliations
- Anthropic
- Apollo Research
- Apple
- CLAIRE
- Carnegie Mellon University
- Chalmers University of Technology
- EPFL
- FAIR
- Google
- Google DeepMind
- IBM Research
- INRIA
- MIT
- McGill University
- Microsoft
- Mila Quebec AI Institute
- NVIDIA
- New York University
- Northeastern University
- OpenAI
- Rochester Institute of Technology
- Stanford
- TU Delft
- Tsinghua University
- UC Berkeley
- UC San Diego
- UC Santa Barbara
- University of Amsterdam
- University of Chile
- University of Illinois at Urbana-Champaign
- Vector Institute
- Vrije Universiteit Amsterdam
- Yonsei University
- 104 Other
- EPFL-CS439 - Optimization for Machine Learning
- Introducing Apple’s On-Device and Server Foundation Models
- MIT-65940 - TinyML and Efficient Deep Learning Computing
- TinyML and Efficient Deep Learning Computing
- TinyML and Efficient Deep Learning Computing - Lecture 12
- TinyML and Efficient Deep Learning Computing - Lecture 3
- TinyML and Efficient Deep Learning Computing - Lecture 5
- Tweet - Stable Diffusion XL on iPhone with Core ML!
- search
Some content is hidden
Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.
270 files changed
+11979
-71
lines changedOriginal file line number | Diff line number | Diff line change | |
---|---|---|---|
| |||
2813 | 2813 |
| |
2814 | 2814 |
| |
2815 | 2815 |
| |
| 2816 | + | |
| 2817 | + | |
| 2818 | + | |
| 2819 | + | |
| 2820 | + | |
| 2821 | + | |
| 2822 | + | |
| 2823 | + | |
| 2824 | + | |
| 2825 | + | |
| 2826 | + | |
| 2827 | + | |
| 2828 | + | |
| 2829 | + | |
| 2830 | + | |
| 2831 | + | |
| 2832 | + | |
| 2833 | + | |
| 2834 | + | |
| 2835 | + | |
| 2836 | + | |
2816 | 2837 |
| |
2817 | 2838 |
| |
2818 | 2839 |
| |
|
Original file line number | Diff line number | Diff line change | |
---|---|---|---|
| |||
2855 | 2855 |
| |
2856 | 2856 |
| |
2857 | 2857 |
| |
| 2858 | + | |
| 2859 | + | |
| 2860 | + | |
| 2861 | + | |
| 2862 | + | |
| 2863 | + | |
| 2864 | + | |
| 2865 | + | |
| 2866 | + | |
| 2867 | + | |
| 2868 | + | |
| 2869 | + | |
| 2870 | + | |
| 2871 | + | |
| 2872 | + | |
| 2873 | + | |
| 2874 | + | |
| 2875 | + | |
| 2876 | + | |
| 2877 | + | |
| 2878 | + | |
2858 | 2879 |
| |
2859 | 2880 |
| |
2860 | 2881 |
| |
|
Original file line number | Diff line number | Diff line change | |
---|---|---|---|
| |||
2813 | 2813 |
| |
2814 | 2814 |
| |
2815 | 2815 |
| |
| 2816 | + | |
| 2817 | + | |
| 2818 | + | |
| 2819 | + | |
| 2820 | + | |
| 2821 | + | |
| 2822 | + | |
| 2823 | + | |
| 2824 | + | |
| 2825 | + | |
| 2826 | + | |
| 2827 | + | |
| 2828 | + | |
| 2829 | + | |
| 2830 | + | |
| 2831 | + | |
| 2832 | + | |
| 2833 | + | |
| 2834 | + | |
| 2835 | + | |
| 2836 | + | |
2816 | 2837 |
| |
2817 | 2838 |
| |
2818 | 2839 |
| |
|
Original file line number | Diff line number | Diff line change | |
---|---|---|---|
| |||
2813 | 2813 |
| |
2814 | 2814 |
| |
2815 | 2815 |
| |
| 2816 | + | |
| 2817 | + | |
| 2818 | + | |
| 2819 | + | |
| 2820 | + | |
| 2821 | + | |
| 2822 | + | |
| 2823 | + | |
| 2824 | + | |
| 2825 | + | |
| 2826 | + | |
| 2827 | + | |
| 2828 | + | |
| 2829 | + | |
| 2830 | + | |
| 2831 | + | |
| 2832 | + | |
| 2833 | + | |
| 2834 | + | |
| 2835 | + | |
| 2836 | + | |
2816 | 2837 |
| |
2817 | 2838 |
| |
2818 | 2839 |
| |
|
Original file line number | Diff line number | Diff line change | |
---|---|---|---|
| |||
2813 | 2813 |
| |
2814 | 2814 |
| |
2815 | 2815 |
| |
| 2816 | + | |
| 2817 | + | |
| 2818 | + | |
| 2819 | + | |
| 2820 | + | |
| 2821 | + | |
| 2822 | + | |
| 2823 | + | |
| 2824 | + | |
| 2825 | + | |
| 2826 | + | |
| 2827 | + | |
| 2828 | + | |
| 2829 | + | |
| 2830 | + | |
| 2831 | + | |
| 2832 | + | |
| 2833 | + | |
| 2834 | + | |
| 2835 | + | |
| 2836 | + | |
2816 | 2837 |
| |
2817 | 2838 |
| |
2818 | 2839 |
| |
|
Original file line number | Diff line number | Diff line change | |
---|---|---|---|
| |||
2813 | 2813 |
| |
2814 | 2814 |
| |
2815 | 2815 |
| |
| 2816 | + | |
| 2817 | + | |
| 2818 | + | |
| 2819 | + | |
| 2820 | + | |
| 2821 | + | |
| 2822 | + | |
| 2823 | + | |
| 2824 | + | |
| 2825 | + | |
| 2826 | + | |
| 2827 | + | |
| 2828 | + | |
| 2829 | + | |
| 2830 | + | |
| 2831 | + | |
| 2832 | + | |
| 2833 | + | |
| 2834 | + | |
| 2835 | + | |
| 2836 | + | |
2816 | 2837 |
| |
2817 | 2838 |
| |
2818 | 2839 |
| |
|
Original file line number | Diff line number | Diff line change | |
---|---|---|---|
| |||
2855 | 2855 |
| |
2856 | 2856 |
| |
2857 | 2857 |
| |
| 2858 | + | |
| 2859 | + | |
| 2860 | + | |
| 2861 | + | |
| 2862 | + | |
| 2863 | + | |
| 2864 | + | |
| 2865 | + | |
| 2866 | + | |
| 2867 | + | |
| 2868 | + | |
| 2869 | + | |
| 2870 | + | |
| 2871 | + | |
| 2872 | + | |
| 2873 | + | |
| 2874 | + | |
| 2875 | + | |
| 2876 | + | |
| 2877 | + | |
| 2878 | + | |
2858 | 2879 |
| |
2859 | 2880 |
| |
2860 | 2881 |
| |
|
Original file line number | Diff line number | Diff line change | |
---|---|---|---|
| |||
2813 | 2813 |
| |
2814 | 2814 |
| |
2815 | 2815 |
| |
| 2816 | + | |
| 2817 | + | |
| 2818 | + | |
| 2819 | + | |
| 2820 | + | |
| 2821 | + | |
| 2822 | + | |
| 2823 | + | |
| 2824 | + | |
| 2825 | + | |
| 2826 | + | |
| 2827 | + | |
| 2828 | + | |
| 2829 | + | |
| 2830 | + | |
| 2831 | + | |
| 2832 | + | |
| 2833 | + | |
| 2834 | + | |
| 2835 | + | |
| 2836 | + | |
2816 | 2837 |
| |
2817 | 2838 |
| |
2818 | 2839 |
| |
|
Original file line number | Diff line number | Diff line change | |
---|---|---|---|
| |||
2813 | 2813 |
| |
2814 | 2814 |
| |
2815 | 2815 |
| |
| 2816 | + | |
| 2817 | + | |
| 2818 | + | |
| 2819 | + | |
| 2820 | + | |
| 2821 | + | |
| 2822 | + | |
| 2823 | + | |
| 2824 | + | |
| 2825 | + | |
| 2826 | + | |
| 2827 | + | |
| 2828 | + | |
| 2829 | + | |
| 2830 | + | |
| 2831 | + | |
| 2832 | + | |
| 2833 | + | |
| 2834 | + | |
| 2835 | + | |
| 2836 | + | |
2816 | 2837 |
| |
2817 | 2838 |
| |
2818 | 2839 |
| |
|
Original file line number | Diff line number | Diff line change | |
---|---|---|---|
| |||
2813 | 2813 |
| |
2814 | 2814 |
| |
2815 | 2815 |
| |
| 2816 | + | |
| 2817 | + | |
| 2818 | + | |
| 2819 | + | |
| 2820 | + | |
| 2821 | + | |
| 2822 | + | |
| 2823 | + | |
| 2824 | + | |
| 2825 | + | |
| 2826 | + | |
| 2827 | + | |
| 2828 | + | |
| 2829 | + | |
| 2830 | + | |
| 2831 | + | |
| 2832 | + | |
| 2833 | + | |
| 2834 | + | |
| 2835 | + | |
| 2836 | + | |
2816 | 2837 |
| |
2817 | 2838 |
| |
2818 | 2839 |
| |
|
Original file line number | Diff line number | Diff line change | |
---|---|---|---|
| |||
2813 | 2813 |
| |
2814 | 2814 |
| |
2815 | 2815 |
| |
| 2816 | + | |
| 2817 | + | |
| 2818 | + | |
| 2819 | + | |
| 2820 | + | |
| 2821 | + | |
| 2822 | + | |
| 2823 | + | |
| 2824 | + | |
| 2825 | + | |
| 2826 | + | |
| 2827 | + | |
| 2828 | + | |
| 2829 | + | |
| 2830 | + | |
| 2831 | + | |
| 2832 | + | |
| 2833 | + | |
| 2834 | + | |
| 2835 | + | |
| 2836 | + | |
2816 | 2837 |
| |
2817 | 2838 |
| |
2818 | 2839 |
| |
|
Original file line number | Diff line number | Diff line change | |
---|---|---|---|
| |||
2813 | 2813 |
| |
2814 | 2814 |
| |
2815 | 2815 |
| |
| 2816 | + | |
| 2817 | + | |
| 2818 | + | |
| 2819 | + | |
| 2820 | + | |
| 2821 | + | |
| 2822 | + | |
| 2823 | + | |
| 2824 | + | |
| 2825 | + | |
| 2826 | + | |
| 2827 | + | |
| 2828 | + | |
| 2829 | + | |
| 2830 | + | |
| 2831 | + | |
| 2832 | + | |
| 2833 | + | |
| 2834 | + | |
| 2835 | + | |
| 2836 | + | |
2816 | 2837 |
| |
2817 | 2838 |
| |
2818 | 2839 |
| |
|
Original file line number | Diff line number | Diff line change | |
---|---|---|---|
| |||
2813 | 2813 |
| |
2814 | 2814 |
| |
2815 | 2815 |
| |
| 2816 | + | |
| 2817 | + | |
| 2818 | + | |
| 2819 | + | |
| 2820 | + | |
| 2821 | + | |
| 2822 | + | |
| 2823 | + | |
| 2824 | + | |
| 2825 | + | |
| 2826 | + | |
| 2827 | + | |
| 2828 | + | |
| 2829 | + | |
| 2830 | + | |
| 2831 | + | |
| 2832 | + | |
| 2833 | + | |
| 2834 | + | |
| 2835 | + | |
| 2836 | + | |
2816 | 2837 |
| |
2817 | 2838 |
| |
2818 | 2839 |
| |
|
Original file line number | Diff line number | Diff line change | |
---|---|---|---|
| |||
2813 | 2813 |
| |
2814 | 2814 |
| |
2815 | 2815 |
| |
| 2816 | + | |
| 2817 | + | |
| 2818 | + | |
| 2819 | + | |
| 2820 | + | |
| 2821 | + | |
| 2822 | + | |
| 2823 | + | |
| 2824 | + | |
| 2825 | + | |
| 2826 | + | |
| 2827 | + | |
| 2828 | + | |
| 2829 | + | |
| 2830 | + | |
| 2831 | + | |
| 2832 | + | |
| 2833 | + | |
| 2834 | + | |
| 2835 | + | |
| 2836 | + | |
2816 | 2837 |
| |
2817 | 2838 |
| |
2818 | 2839 |
| |
|
0 commit comments