Inductive Bias
561 papers with code • 0 benchmarks • 0 datasets
Benchmarks
These leaderboards are used to track progress in Inductive Bias
Libraries
Use these libraries to find Inductive Bias models and implementationsMost implemented papers
Global Context Vision Transformers
nvlabs/gcvit
•
•
Pre-trained GC ViT backbones in downstream tasks of object detection, instance segmentation, and semantic segmentation using MS COCO and ADE20K datasets outperform prior work consistently.
Ordered Neurons: Integrating Tree Structures into Recurrent Neural Networks
yikangshen/Ordered-Neurons
•
•
ICLR 2019
When a larger constituent ends, all of the smaller constituents that are nested within it must also be closed.
Train Short, Test Long: Attention with Linear Biases Enables Input Length Extrapolation
ofirpress/attention_with_linear_biases
•
•
ICLR 2022
Since the introduction of the transformer model by Vaswani et al. (2017), a fundamental question has yet to be answered: how does a model achieve extrapolation at inference time for sequences that are longer than it saw during training?
Multi-Task Learning as Multi-Objective Optimization
IntelVCL/MultiObjectiveOptimization
•
•
NeurIPS 2018
These algorithms are not directly applicable to large-scale learning problems since they scale poorly with the dimensionality of the gradients and the number of tasks.
Knowledge-aware Graph Neural Networks with Label Smoothness Regularization for Recommender Systems
hwwang55/KGNN-LS
•
•
Here we propose Knowledge-aware Graph Neural Networks with Label Smoothness regularization (KGNN-LS) to provide better recommendations.
Vision Transformer for Small-Size Datasets
However, the high performance of the ViT results from pre-training using a large-size dataset such as JFT-300M, and its dependence on a large dataset is interpreted as due to low locality inductive bias.
Mega: Moving Average Equipped Gated Attention
The design choices in the Transformer attention mechanism, including weak inductive bias and quadratic computational complexity, have limited its application for modeling long sequences.
3D Packing for Self-Supervised Monocular Depth Estimation
TRI-ML/packnet-sfm
•
•
CVPR 2020
Although cameras are ubiquitous, robotic platforms typically rely on active sensors like LiDAR for direct 3D perception.
A Hierarchical Probabilistic U-Net for Modeling Multi-Scale Ambiguities
SimonKohl/probabilistic_unet
•
•
Medical imaging only indirectly measures the molecular identity of the tissue within each voxel, which often produces only ambiguous image evidence for target measures of interest, like semantic segmentation.
On the Transfer of Inductive Bias from Simulation to the Real World: a New Disentanglement Dataset
Learning meaningful and compact representations with disentangled semantic aspects is considered to be of key importance in representation learning.
