Browse State-of-the-Art
Datasets
Methods
More
Newsletter
RC2022
About
Trends
Portals
Libraries
Sign In
Subscribe to the PwC Newsletter
×
Stay informed on the latest trending ML papers with code, research developments, libraries, methods, and datasets.
Read previous issues
Join the community
×
You need to
log in
to edit.
You can
create a new account
if you don't have one.
Methods
> General
General
1129 methods • 601773 papers with code
Attention
Scaled Dot-Product Attention
22346 papers with code
Multi-Head Attention
22274 papers with code
Strided Attention
1789 papers with code
Fixed Factorized Attention
1789 papers with code
VAE
1195 papers with code
See all 131 methods
Optimization
Adam
21937 papers with code
SGD
1905 papers with code
Adafactor
683 papers with code
RMSProp
493 papers with code
ADMM
377 papers with code
See all 111 methods
Attention Mechanisms
Scaled Dot-Product Attention
22346 papers with code
Strided Attention
1789 papers with code
Fixed Factorized Attention
1789 papers with code
VAE
1195 papers with code
RAN
310 papers with code
See all 86 methods
Activation Functions
ReLU
10201 papers with code
GELU
10127 papers with code
Sigmoid Activation
6808 papers with code
Tanh Activation
6084 papers with code
Leaky ReLU
1287 papers with code
See all 82 methods
Stochastic Optimization
Adam
21937 papers with code
SGD
1905 papers with code
Adafactor
683 papers with code
RMSProp
493 papers with code
ADOPT
223 papers with code
See all 72 methods
Regularization
Dropout
24614 papers with code
Label Smoothing
12751 papers with code
Attention Dropout
9706 papers with code
Weight Decay
9658 papers with code
Entropy Regularization
984 papers with code
See all 62 methods
Skip Connection Blocks
Residual Block
2767 papers with code
Bottleneck Residual Block
2048 papers with code
Inverted Residual Block
771 papers with code
Dense Block
476 papers with code
Non-Local Block
167 papers with code
See all 49 methods
Self-Supervised Learning
Inpainting
869 papers with code
MAE
589 papers with code
SimCLR
229 papers with code
Colorization
197 papers with code
DINO
153 papers with code
See all 47 methods
Attention Modules
Multi-Head Attention
22274 papers with code
Spatial Attention Module
205 papers with code
SAGAN Self-Attention Module
138 papers with code
Cross-Attention Module
113 papers with code
Channel Attention Module
93 papers with code
See all 43 methods
Distributed Methods
Local SGD
65 papers with code
Parallax
58 papers with code
Gradient Sparsification
34 papers with code
IMPALA
15 papers with code
Chimera
14 papers with code
See all 43 methods
Loss Functions
Focal Loss
445 papers with code
Cycle Consistency Loss
425 papers with code
Triplet Loss
410 papers with code
GAN Least Squares Loss
400 papers with code
InfoNCE
389 papers with code
See all 42 methods
Normalization
Layer Normalization
22177 papers with code
Batch Normalization
6069 papers with code
Instance Normalization
541 papers with code
Adaptive Instance Normalization
419 papers with code
Local Response Normalization
414 papers with code
See all 38 methods
Deep Tabular Learning
BiLSTM
623 papers with code
NON
347 papers with code
NODE
35 papers with code
TabNet
23 papers with code
TABPFN
18 papers with code
See all 29 methods
Semi-Supervised Learning Methods
MoCo
142 papers with code
Contrastive Predictive Coding
108 papers with code
FixMatch
80 papers with code
SPS
41 papers with code
SPL
36 papers with code
See all 27 methods
Clustering
k-Means Clustering
719 papers with code
Spectral Clustering
492 papers with code
SOM
98 papers with code
Coresets
96 papers with code
Ensemble Clustering
18 papers with code
See all 21 methods
Neural Architecture Search
Capsule Network
179 papers with code
DARTS
139 papers with code
Differentiable NAS
37 papers with code
Neural Architecture Search
23 papers with code
DNAS
19 papers with code
See all 20 methods
Domain Adaptation
CORAL
106 papers with code
DANCE
10 papers with code
Source Hypothesis Transfer
3 papers with code
PGA
3 papers with code
MSGAN
2 papers with code
See all 19 methods
Adversarial Training
GAIL
40 papers with code
AdvProp
6 papers with code
Singular Value Clipping
3 papers with code
DiffAugment
3 papers with code
SimAug
3 papers with code
See all 17 methods
Adaptive Activation Functions
Swish
390 papers with code
LEAF
124 papers with code
KAN
115 papers with code
NFN
5 papers with code
modReLU
5 papers with code
See all 17 methods
Feedforward Networks
Dense Connections
26245 papers with code
Linear Layer
22762 papers with code
Position-Wise Feed-Forward Layer
12517 papers with code
Feedforward Network
1275 papers with code
Adapter
898 papers with code
See all 16 methods
Data Parallel Methods
Local SGD
65 papers with code
Gradient Sparsification
34 papers with code
ZeRO
8 papers with code
Accordion
3 papers with code
PyTorch DDP
3 papers with code
See all 16 methods
Interpretability
SHAP
448 papers with code
LIME
332 papers with code
CAM
243 papers with code
Monte Carlo Dropout
179 papers with code
Visual Analytics
65 papers with code
See all 16 methods
Miscellaneous Components
Auxiliary Classifier
382 papers with code
Highway Layer
107 papers with code
Channel Shuffle
77 papers with code
PixelShuffle
44 papers with code
Scatter Connection
10 papers with code
See all 14 methods
Learning Rate Schedules
Linear Warmup With Linear Decay
6258 papers with code
Cosine Annealing
3579 papers with code
Linear Warmup With Cosine Annealing
3431 papers with code
Inverse Square Root Schedule
655 papers with code
Exponential Decay
97 papers with code
See all 13 methods
Large Batch Optimization
Adam
21937 papers with code
Adafactor
683 papers with code
LAMB
190 papers with code
AdaGrad
183 papers with code
LARS
71 papers with code
See all 13 methods
Knowledge Distillation
Knowledge Distillation
2750 papers with code
Ontology
563 papers with code
SFT
231 papers with code
Hydra
28 papers with code
STD
25 papers with code
See all 11 methods
AutoML
Feature Selection
1447 papers with code
HPO
136 papers with code
MDL
91 papers with code
AutoParsimony
13 papers with code
VEGA
5 papers with code
See all 10 methods
Output Functions
Softmax
27837 papers with code
Heatmap
393 papers with code
Mixture of Logistic Distributions
166 papers with code
Adaptive Softmax
69 papers with code
EVM
27 papers with code
See all 10 methods
Fine-Tuning
Discriminative Fine-Tuning
1733 papers with code
Virtual Data Augmentation
3 papers with code
Leverage Learning
2 papers with code
ERNIE-GEN
1 papers with code
Child-Tuning
1 papers with code
See all 10 methods
Hybrid Parallel Methods
Parallax
58 papers with code
dhf
5 papers with code
Herring
5 papers with code
FastMoE
2 papers with code
BytePS
1 papers with code
See all 10 methods
Non-Parametric Regression
Gaussian Process
2311 papers with code
SVM
1756 papers with code
k-NN
186 papers with code
GAM
51 papers with code
MFEC
2 papers with code
See all 9 methods
Model Parallel Methods
Chimera
14 papers with code
Tofu
13 papers with code
GPipe
7 papers with code
GShard
6 papers with code
PipeDream
5 papers with code
See all 9 methods
Dimensionality Reduction
PCA
1225 papers with code
Variational Inference
769 papers with code
LDA
439 papers with code
Latent Diffusion Model
293 papers with code
AE
264 papers with code
See all 8 methods
Initialization
Kaiming Initialization
2814 papers with code
Xavier Initialization
115 papers with code
SkipInit
2 papers with code
Fixup Initialization
2 papers with code
T-Fixup
2 papers with code
See all 7 methods
Discriminators
PatchGAN
491 papers with code
Projection Discriminator
135 papers with code
Window-based Discriminator
14 papers with code
NIMA
8 papers with code
Multiple Random Window Discriminator
2 papers with code
See all 7 methods
Prioritized Sampling
ATSS
16 papers with code
PISA
15 papers with code
OHEM
7 papers with code
IoU-Balanced Sampling
6 papers with code
See all 7 methods
Meta-Learning Algorithms
MAML
225 papers with code
MeRL
9 papers with code
MEND
9 papers with code
OCD
8 papers with code
Meta-augmentation
4 papers with code
See all 7 methods
Robust Training
DEQ
39 papers with code
ELR
6 papers with code
Self-adaptive Training
4 papers with code
CW-ERM
1 papers with code
See all 7 methods
Adversarial Attacks
Fast Minimum-Norm Attack
2 papers with code
G-NIA
1 papers with code
Morphence
1 papers with code
Adversarial Solarization
1 papers with code
RandSol
1 papers with code
See all 7 methods
2D Parallel Distributed Methods
dhf
5 papers with code
FastMoE
2 papers with code
PipeTransformer
1 papers with code
E2EAdaptiveDistTraining
1 papers with code
See all 6 methods
Representation Learning
MIM
132 papers with code
ViLBERT
30 papers with code
VideoBERT
3 papers with code
Playstyle Distance
3 papers with code
CV-MIM
2 papers with code
See all 6 methods
Non-Parametric Classification
Gaussian Process
2311 papers with code
SVM
1756 papers with code
k-NN
186 papers with code
MFF
14 papers with code
ZLaP
1 papers with code
See all 6 methods
Active Learning
BASE
4696 papers with code
EWC
66 papers with code
Snapshot Ensembles
8 papers with code
ALDEN
1 papers with code
Poker Software Developer
1 papers with code
See all 6 methods
Position Embeddings
Absolute Position Encodings
12355 papers with code
Relative Position Encodings
43 papers with code
ALiBi
15 papers with code
Rotary Embeddings
8 papers with code
Conditional Positional Encoding
6 papers with code
See all 6 methods
Working Memory Models
Memory Network
420 papers with code
Neural Turing Machine
21 papers with code
Dynamic Memory Network
19 papers with code
End-To-End Memory Network
2 papers with code
Recurrent Entity Network
2 papers with code
See all 5 methods
Approximate Inference
ABC
114 papers with code
Metropolis Hastings
10 papers with code
reSGLD
3 papers with code
CSGLD
2 papers with code
See all 5 methods
Model Compression
Pruning
3389 papers with code
DST
198 papers with code
Soups
18 papers with code
NNCF
2 papers with code
CORAD
1 papers with code
See all 5 methods
Pruning
Movement Pruning
5 papers with code
RigL
5 papers with code
HRank
1 papers with code
Spectral-Normalized Identity Priors
1 papers with code
LLM-Rank
1 papers with code
See all 5 methods
Robustness Methods
Randomized Smoothing
140 papers with code
Denoised Smoothing
7 papers with code
Fishr
2 papers with code
Randomized Deletion
2 papers with code
See all 5 methods
Recommendation Systems
LightGCN
42 papers with code
CPE
22 papers with code
ComiRec
3 papers with code
SmeLU
2 papers with code
FM with splines
1 papers with code
See all 5 methods
Affinity Functions
Embedded Gaussian Affinity
8 papers with code
Embedded Dot Product Affinity
1 papers with code
Concatenation Affinity
1 papers with code
Gaussian Affinity
See all 5 methods
Fourier-related Transforms
Discrete Cosine Transform
195 papers with code
Characteristic Functions
9 papers with code
MCKERNEL
2 papers with code
timecausgabor
1 papers with code
discgabor
1 papers with code
See all 5 methods
Hyperparameter Search
Random Search
242 papers with code
DAC
69 papers with code
Population Based Training
18 papers with code
Differentiable Hyperparameter Search
1 papers with code
See all 4 methods
Generalized Linear Models
Logistic Regression
1743 papers with code
Linear Regression
1539 papers with code
Procrustes
72 papers with code
See all 4 methods
Asynchronous Pipeline Parallel
PipeDream
5 papers with code
PipeDream-2BW
3 papers with code
PipeMare
1 papers with code
Pipelined Backpropagation
1 papers with code
See all 4 methods
Replicated Data Parallel
PyTorch DDP
3 papers with code
BAGUA
1 papers with code
ByteScheduler
1 papers with code
DABMD
1 papers with code
See all 4 methods
Information Retrieval Methods
SERAC
4 papers with code
TILDEv2
3 papers with code
ReInfoSelect
1 papers with code
See all 4 methods
SLAM Methods
DROID-SLAM
6 papers with code
NICE-SLAM
6 papers with code
NeuralRecon
2 papers with code
VDO-SLAM
1 papers with code
See all 4 methods
Quantum Methods
gCANS
2 papers with code
VQSVD
1 papers with code
VTDE
1 papers with code
Variational Entanglement Detection
1 papers with code
See all 4 methods
Parameter Server Methods
Parallax
58 papers with code
Herring
5 papers with code
BytePS
1 papers with code
HetPipe
See all 4 methods
Skip Connections
Residual Connection
25805 papers with code
Concatenated Skip Connection
3059 papers with code
Zero-padded Shortcut Connection
8 papers with code
Deactivable Skip Connection
1 papers with code
See all 4 methods
Rule-based systems
WFST
6 papers with code
Symbolic rule learning
4 papers with code
SAFRAN
2 papers with code
See all 3 methods
Lifelong Learning
Self-Learning
208 papers with code
CLASSP
2 papers with code
LIMix
1 papers with code
See all 3 methods
Structured Prediction
CRF
456 papers with code
LTLS
2 papers with code
DMVFN
1 papers with code
See all 3 methods
Markov Chain Monte Carlo
Metropolis Hastings
10 papers with code
reSGLD
3 papers with code
CSGLD
2 papers with code
See all 3 methods
Auto Parallel Methods
AutoSync
1 papers with code
KungFu
FlexFlow
See all 3 methods
Ensembling
MoE
207 papers with code
EMEA
2 papers with code
See all 3 methods
Prompt Engineering
CoT Prompting
61 papers with code
CoOp
27 papers with code
KnowPrompt
6 papers with code
See all 3 methods
Variational Optimization
GECO
14 papers with code
Residual Normal Distribution
5 papers with code
ASVI
2 papers with code
See all 3 methods
Parameter Norm Penalties
Weight Decay
9658 papers with code
L1 Regularization
78 papers with code
ROME
20 papers with code
See all 3 methods
Gated Linear Networks
GLN
8 papers with code
G-GLN
1 papers with code
G-GLN Neuron
1 papers with code
See all 3 methods
Adaptive Computation
PonderNet
3 papers with code
AdaptiveBins
2 papers with code
PoAPL
1 papers with code
See all 3 methods
Intra-Layer Parallel
Tofu
13 papers with code
GShard
6 papers with code
Mesh-TensorFlow
2 papers with code
See all 3 methods
Sharded Data Parallel Methods
ZeRO
8 papers with code
ZeRO-Offload
2 papers with code
ZeRO-Infinity
1 papers with code
See all 3 methods
Feature Matching
Reliability Balancing
2 papers with code
Good Feature Matching
1 papers with code
MaskFlownet
1 papers with code
See all 3 methods
Information Bottleneck
Aggregated Learning
1 papers with code
ReInfoSelect
1 papers with code
See all 3 methods
Inference Attack
Canvas Method
1 papers with code
Spectral DeTuning
1 papers with code
See all 3 methods
Asynchronous Data Parallel
Crossbow
2 papers with code
SlowMo
1 papers with code
Wavelet Distributed Training
See all 3 methods
Robotic Manipulation Models
VSF
4 papers with code
myGym
2 papers with code
See all 3 methods
Action Recognition Blocks
TPN
6 papers with code
G3D
1 papers with code
See all 3 methods
Latent Variable Sampling
Truncation Trick
136 papers with code
Latent Optimisation
13 papers with code
Polya-Gamma Augmentation
6 papers with code
See all 3 methods
Tabular Data Generation
CTAB-GAN
2 papers with code
Outlier Generation
1 papers with code
See all 2 methods
Augmented Reality Methods
ACGPN
3 papers with code
ARShoe
1 papers with code
See all 2 methods
Negative Sampling
Self-Adversarial Negative Sampling
28 papers with code
MBS
9 papers with code
See all 2 methods
Generalized Additive Models
NAM
15 papers with code
Base Boosting
2 papers with code
See all 2 methods
Hybrid Optimization
MEUZZ
1 papers with code
HFPSO
See all 2 methods
Distributions
Gumbel Softmax
50 papers with code
See all 2 methods
Synchronous Pipeline Parallel
Chimera
14 papers with code
GPipe
7 papers with code
See all 2 methods
Inference Engines
Mobile Neural Network
4 papers with code
See all 2 methods
Kernel Methods
NTK
220 papers with code
StreaMRAK
1 papers with code
See all 2 methods
Control and Decision Systems
PPMC
1 papers with code
GA-PID/NN-PID
See all 2 methods
Manifold Disentangling
GEOMANCER
1 papers with code
See all 2 methods
Sparsity
SET
9899 papers with code
Neural Tangent Transfer
1 papers with code
See all 2 methods
Leadership Inference
FLICA
3 papers with code
See all 2 methods
Distribution Approximation
Normalizing Flows
715 papers with code
QuantTree
3 papers with code
See all 2 methods
Self-Training Methods
STraTA
1 papers with code
self-mem + new data
1 papers with code
See all 2 methods
Network Shrinking
TinyNet
4 papers with code
NetAdapt
3 papers with code
See all 2 methods
Factorization Machines
FEFM
2 papers with code
FM with splines
1 papers with code
See all 2 methods
Out-of-Distribution Example Detection
DIME
4 papers with code
See all 1 methods
Global Context Modules
Hamburger
1 papers with code
See all 1 methods
Parameter Sharing
Weight Tying
62 papers with code
See all 1 methods
Genetic Algorithm
See all 1 methods
Theorem Proving Models
NeuroTactic
1 papers with code
See all 1 methods
Playstyle
Playstyle Distance
3 papers with code
See all 1 methods
Diffusion Models
LSDM
1 papers with code
See all 1 methods
Spreadsheet Formula Prediction Models
SpreadsheetCoder
1 papers with code
See all 1 methods
Momentum Rules
Demon
15 papers with code
See all 1 methods
Distributed Communication
Blink Communication
1 papers with code
See all 1 methods
Bijective Transformation
Affine Coupling
224 papers with code
See all 1 methods
Distillation
SFT
231 papers with code
See all 1 methods
Probability distribution representation
kdm
3 papers with code
See all 1 methods
Geometric Matching
CHM
5 papers with code
See all 1 methods
Sample Re-Weighting
Fast Sample Re-Weighting
1 papers with code
See all 1 methods
Cache Replacement Models
Parrot
13 papers with code
See all 1 methods
Generalization
Distributional Generalization
4 papers with code
See all 1 methods
Confidence Calibration
CCAC
2 papers with code
See all 1 methods
Confidence Estimators
MACEst
1 papers with code
See all 1 methods
Long-Range Interaction Layers
Lambda Layer
2 papers with code
See all 1 methods
Hybrid Fuzzing
MEUZZ
1 papers with code
See all 1 methods
Binary Neural Networks
BiDet
1 papers with code
See all 1 methods
Bot Detection
BIMAN
1 papers with code
See all 1 methods
Rule Learners
GPFL
3 papers with code
See all 1 methods
Learning to Rank Models
DCN-V2
3 papers with code
See all 1 methods
Position Recovery Models
PRNet+
1 papers with code
See all 1 methods
Table Parsing Models
Cycle-CenterNet
1 papers with code
See all 1 methods
Mixture-of-Experts
DSelect-k
2 papers with code
See all 1 methods
Mesh-Based Simulation Models
MeshGraphNet
5 papers with code
See all 1 methods
Statistical Inference
Estimation Statistics
1 papers with code
See all 1 methods
Air Quality Forecasting
GAGNN
1 papers with code
See all 1 methods
Twin Networks
Siamese Network
333 papers with code
See all 1 methods
Incident Aggregation Models
GRLIA
1 papers with code
See all 1 methods
Label Correction
Label Quality Model
1 papers with code
See all 1 methods
Ternarization
Ternary Weight Splitting
2 papers with code
See all 1 methods
Fuzzy Logic
NFN
5 papers with code
See all 1 methods
