Search Results for author:
Found 23 papers, 20 papers with code
The Llama 3 Herd of Models
This paper presents a new set of foundation models, called Llama 3.
Ranked #3 on
Question Answering
on PeerQA
Automatic Data Curation for Self-Supervised Learning: A Clustering-Based Approach
This manual process has some limitations similar to those encountered in supervised learning, e. g., the crowd-sourced selection of data is costly and time-consuming, preventing scaling the dataset size.
Code Llama: Open Foundation Models for Code
We release Code Llama, a family of large language models for code based on Llama 2 providing state-of-the-art performance among open models, infilling capabilities, support for large input contexts, and zero-shot instruction following ability for programming tasks.
Ranked #37 on
Code Generation
on MBPP
(using extra training data)
Llama 2: Open Foundation and Fine-Tuned Chat Models
In this work, we develop and release Llama 2, a collection of pretrained and fine-tuned large language models (LLMs) ranging in scale from 7 billion to 70 billion parameters.
Ranked #2 on
Question Answering
on PubChemQA
LLaMA: Open and Efficient Foundation Language Models
We introduce LLaMA, a collection of foundation language models ranging from 7B to 65B parameters.
Ranked #3 on
Few-Shot Learning
on MedConceptsQA
Co-Training 2L Submodels for Visual Recognition
Given a neural network to be trained, for each sample we implicitly instantiate two altered networks, "submodels", with stochastic depth: i. e. activating only a subset of the layers and skipping others.
Co-training $2^L$ Submodels for Visual Recognition
We introduce submodel co-training, a regularization method related to co-training, self-distillation and stochastic depth.
Ranked #63 on
Image Classification
on ImageNet
DeiT III: Revenge of the ViT
Our evaluations on Image classification (ImageNet-1k with and without pre-training on ImageNet-21k), transfer learning and semantic segmentation show that our procedure outperforms by a large margin previous fully supervised training recipes for ViT.
Ranked #1 on
Image Classification
on ImageNet ReaL
(Number of params metric)
Three things everyone should know about Vision Transformers
(2) Fine-tuning the weights of the attention layers is sufficient to adapt vision transformers to a higher resolution and to other classification tasks.
Ranked #10 on
Image Classification
on CIFAR-10
(using extra training data)
Augmenting Convolutional networks with attention-based aggregation
We show how to augment any convolutional network with an attention-based global map to achieve non-local reasoning.
Ranked #41 on
Semantic Segmentation
on ADE20K val
Are Large-scale Datasets Necessary for Self-Supervised Pre-training?
Our study shows that denoising autoencoders, such as BEiT or a variant that we introduce in this paper, are more robust to the type and size of the pre-training data than popular self-supervised methods trained by comparing image embeddings. We obtain competitive performance compared to ImageNet pre-training on a variety of classification datasets, from different domains.
ResNet strikes back: An improved training procedure in timm
We share competitive training settings and pre-trained models in the timm open-source library, with the hope that they will serve as better baselines for future work.
Ranked #2 on
Medical Image Classification
on NCT-CRC-HE-100K
XCiT: Cross-Covariance Image Transformers
We propose a "transposed" version of self-attention that operates across feature channels rather than tokens, where the interactions are based on the cross-covariance matrix between keys and queries.
Ranked #58 on
Instance Segmentation
on COCO minival
ResMLP: Feedforward networks for image classification with data-efficient training
We present ResMLP, an architecture built entirely upon multi-layer perceptrons for image classification.
Ranked #1 on
Image Classification
on Certificate Verification
Emerging Properties in Self-Supervised Vision Transformers
In this paper, we question if self-supervised learning provides new properties to Vision Transformer (ViT) that stand out compared to convolutional networks (convnets).
Ranked #2 on
Copy Detection
on Copydays strong subset
LeViT: a Vision Transformer in ConvNet's Clothing for Faster Inference
We design a family of image classification architectures that optimize the trade-off between accuracy and efficiency in a high-speed regime.
Ranked #15 on
Image Classification
on Stanford Cars
Going deeper with Image Transformers
In particular, we investigate the interplay of architecture and optimization of such dedicated transformers.
Ranked #5 on
Image Classification
on Stanford Cars
ConViT: Improving Vision Transformers with Soft Convolutional Inductive Biases
We initialise the GPSA layers to mimic the locality of convolutional layers, then give each attention head the freedom to escape locality by adjusting a gating parameter regulating the attention paid to position versus content information.
Ranked #529 on
Image Classification
on ImageNet
Training data-efficient image transformers & distillation through attention
In this work, we produce a competitive convolution-free transformer by training on Imagenet only.
Ranked #4 on
Efficient ViTs
on ImageNet-1K (with DeiT-S)
Grafit: Learning fine-grained image representations with coarse labels
By jointly leveraging the coarse labels and the underlying fine-grained latent space, it significantly improves the accuracy of category-level retrieval methods.
Ranked #2 on
Learning with coarse labels
on cifar100
Fine-Grained Image Classification
Learning with coarse labels
+3
Powers of layers for image-to-image translation
We propose a simple architecture to address unpaired image-to-image translation tasks: style or class transfer, denoising, deblurring, deblocking, etc.
Ranked #1 on
Image-to-Image Translation
on vangogh2photo
(Frechet Inception Distance metric)
Fixing the train-test resolution discrepancy: FixEfficientNet
An EfficientNet-L2 pre-trained with weak supervision on 300M unlabeled images and further optimized with FixRes achieves 88. 5% top-1 accuracy (top-5: 98. 7%), which establishes the new state of the art for ImageNet with a single crop.
Ranked #9 on
Image Classification
on ImageNet ReaL
(using extra training data)
Fixing the train-test resolution discrepancy
Conversely, when training a ResNeXt-101 32x48d pre-trained in weakly-supervised fashion on 940 million public images at resolution 224x224 and further optimizing for test resolution 320x320, we obtain a test top-1 accuracy of 86. 4% (top-5: 98. 0%) (single-crop).
Ranked #2 on
Fine-Grained Image Classification
on Birdsnap
(using extra training data)
