Search Results for author:
Found 60 papers, 34 papers with code
Learning Visual Composition through Improved Semantic Guidance
One can observe this limitation in representations learned through captions or contrastive learning -- where the learned model treats an image essentially as a bag of words.
Towards flexible perception with visual memory
Training a neural network is a monolithic endeavor, akin to carving knowledge into stone: once the process is completed, editing the knowledge in a network is nearly impossible, since all information is distributed across the network's weights.
Capabilities of Gemini Models in Medicine
We evaluate Med-Gemini on 14 medical benchmarks, establishing new state-of-the-art (SoTA) performance on 10 of them, and surpass the GPT-4 model family on every benchmark where a direct comparison is viable, often by a wide margin.
Ranked #1 on
Question Answering
on MedQA
(using extra training data)
MOFI: Learning Image Representations from Noisy Entity Annotated Images
Our approach involves employing a named entity recognition model to extract entities from the alt-text, and then using a CLIP model to select the correct entities as labels of the paired image.
On Robustness in Multimodal Learning
Multimodal learning is defined as learning over multiple heterogeneous input modalities such as video, audio, and text.
STAIR: Learning Sparse Text and Image Representation in Grounded Tokens
We extend the CLIP model and build a sparse text and image representation (STAIR), where the image and text are mapped to a sparse token space.
PseudoAugment: Learning to Use Unlabeled Data for Data Augmentation in Point Clouds
To alleviate the cost of hyperparameter tuning and iterative pseudo labeling, we develop a population-based data augmentation framework for 3D detection, named AutoPseudoAugment.
Perceptual Grouping in Contrastive Vision-Language Models
In this work we examine how well vision-language models are able to understand where objects reside within an image and group together visually related parts of the imagery.
Scene Transformer: A unified architecture for predicting future trajectories of multiple agents
In this work, we formulate a model for predicting the behavior of all agents jointly, producing consistent futures that account for interactions between agents.
Soft Calibration Objectives for Neural Networks
When incorporated into training, these soft calibration losses achieve state-of-the-art single-model ECE across multiple datasets with less than 1% decrease in accuracy.
Scene Transformer: A unified architecture for predicting multiple agent trajectories
In this work, we formulate a model for predicting the behavior of all agents jointly, producing consistent futures that account for interactions between agents.
Large Scale Interactive Motion Forecasting for Autonomous Driving : The Waymo Open Motion Dataset
Furthermore, we introduce a new set of metrics that provides a comprehensive evaluation of both single agent and joint agent interaction motion forecasting models.
Scaling Local Self-Attention for Parameter Efficient Visual Backbones
Self-attention models have recently been shown to have encouraging improvements on accuracy-parameter trade-offs compared to baseline convolutional models such as ResNet-50.
Ranked #226 on
Image Classification
on ImageNet
Revisiting ResNets: Improved Training and Scaling Strategies
Using improved training and scaling strategies, we design a family of ResNet architectures, ResNet-RS, which are 1. 7x - 2. 7x faster than EfficientNets on TPUs, while achieving similar accuracies on ImageNet.
Pseudo-labeling for Scalable 3D Object Detection
To safely deploy autonomous vehicles, onboard perception systems must work reliably at high accuracy across a diverse set of environments and geographies.
Scalable Scene Flow from Point Clouds in the Real World
In this work, we introduce a new large-scale dataset for scene flow estimation derived from corresponding tracked 3D objects, which is $\sim$1, 000$\times$ larger than previous real-world datasets in terms of the number of annotated frames.
Ranked #6 on
Scene Flow Estimation
on Argoverse 2
Bottleneck Transformers for Visual Recognition
Finally, we present a simple adaptation of the BoTNet design for image classification, resulting in models that achieve a strong performance of 84. 7% top-1 accuracy on the ImageNet benchmark while being up to 1. 64x faster in compute time than the popular EfficientNet models on TPU-v3 hardware.
Ranked #54 on
Instance Segmentation
on COCO minival
Large Scale Interactive Motion Forecasting for Autonomous Driving: The Waymo Open Motion Dataset
Furthermore, we introduce a new set of metrics that provides a comprehensive evaluation of both single agent and joint agent interaction motion forecasting models.
Mitigating Bias in Calibration Error Estimation
We find that binning-based estimators with bins of equal mass (number of instances) have lower bias than estimators with bins of equal width.
Naive-Student: Leveraging Semi-Supervised Learning in Video Sequences for Urban Scene Segmentation
We view this work as a notable step towards building a simple procedure to harness unlabeled video sequences and extra images to surpass state-of-the-art performance on core computer vision tasks.
Streaming Object Detection for 3-D Point Clouds
This built-in data capture latency is artificial, and based on treating the point cloud as a camera image in order to leverage camera-inspired architectures.
Improving 3D Object Detection through Progressive Population Based Augmentation
Data augmentation has been widely adopted for object detection in 3D point clouds.
Revisiting Spatial Invariance with Low-Rank Local Connectivity
Convolutional neural networks are among the most successful architectures in deep learning with this success at least partially attributable to the efficacy of spatial invariance as an inductive bias.
Scalability in Perception for Autonomous Driving: Waymo Open Dataset
In an effort to help align the research community's contributions with real-world self-driving problems, we introduce a new large scale, high quality, diverse dataset.
RandAugment: Practical automated data augmentation with a reduced search space
Additionally, due to the separate search phase, these approaches are unable to adjust the regularization strength based on model or dataset size.
Ranked #12 on
Data Augmentation
on ImageNet
StarNet: Targeted Computation for Object Detection in Point Clouds
We show how our redesign---namely using only local information and using sampling instead of learned proposals---leads to a significantly more flexible and adaptable system: we demonstrate how we can vary the computational cost of a single trained StarNet without retraining, and how we can target proposals towards areas of interest with priors and heuristics.
Learning Data Augmentation Strategies for Object Detection
Importantly, the best policy found on COCO may be transferred unchanged to other detection datasets and models to improve predictive accuracy.
Ranked #6 on
Robust Object Detection
on Cityscapes
A Fourier Perspective on Model Robustness in Computer Vision
Achieving robustness to distributional shift is a longstanding and challenging goal of computer vision.
Stand-Alone Self-Attention in Vision Models
The natural question that arises is whether attention can be a stand-alone primitive for vision models instead of serving as just an augmentation on top of convolutions.
Visual Wake Words Dataset
To facilitate the development of microcontroller friendly models, we present a new dataset, Visual Wake Words, that represents a common microcontroller vision use-case of identifying whether a person is present in the image or not, and provides a realistic benchmark for tiny vision models.
Using learned optimizers to make models robust to input noise
State-of-the art vision models can achieve superhuman performance on image classification tasks when testing and training data come from the same distribution.
Using Videos to Evaluate Image Model Robustness
Compared to previous studies on adversarial examples and synthetic distortions, natural robustness captures a more diverse set of common image transformations that occur in the natural environment.
Attention Augmented Convolutional Networks
Convolutional networks have been the paradigm of choice in many computer vision applications.
Ranked #115 on
Image Classification
on CIFAR-100
(using extra training data)
A Learned Representation for Scalable Vector Graphics
Dramatic advances in generative models have resulted in near photographic quality for artificially rendered faces, animals and other objects in the natural world.
Accelerating Training of Deep Neural Networks with a Standardization Loss
We find that a standardization loss accelerates training on both small- and large-scale image classification experiments, works with a variety of architectures, and is largely robust to training across different batch sizes.
Searching for Efficient Multi-Scale Architectures for Dense Image Prediction
Recent progress has demonstrated that such meta-learning methods may exceed scalable human-invented architectures on image classification tasks.
Ranked #1 on
Human Part Segmentation
on PASCAL-Person-Part
Do Better ImageNet Models Transfer Better?
Transfer learning is a cornerstone of computer vision, yet little work has been done to evaluate the relationship between architecture and transfer.
A Dataset and Architecture for Visual Reasoning with a Working Memory
COG is much simpler than the general problem of video analysis, yet it addresses many of the problems relating to visual and logical reasoning and memory -- problems that remain challenging for modern deep learning architectures.
Learning a neural response metric for retinal prosthesis
Retinal prostheses for treating incurable blindness are designed to electrically stimulate surviving retinal neurons, causing them to send artificial visual signals to the brain.
Progressive Neural Architecture Search
We propose a new method for learning the structure of convolutional neural networks (CNNs) that is more efficient than recent state-of-the-art methods based on reinforcement learning and evolutionary algorithms.
Ranked #17 on
Neural Architecture Search
on NAS-Bench-201, ImageNet-16-120
(Accuracy (Val) metric)
Recurrent Segmentation for Variable Computational Budgets
Importantly, the RNN may be deployed across a range of computational budgets by merely running the model for a variable number of iterations.
Learning Transferable Architectures for Scalable Image Recognition
In our experiments, we search for the best convolutional layer (or "cell") on the CIFAR-10 dataset and then apply this cell to the ImageNet dataset by stacking together more copies of this cell, each with their own parameters to design a convolutional architecture, named "NASNet architecture".
Ranked #6 on
Classification
on InDL
PixColor: Pixel Recursive Colorization
Then, given the generated low-resolution color image and the original grayscale image as inputs, we train a second CNN to generate a high-resolution colorization of an image.
Ranked #3 on
Colorization
on ImageNet val
Exploring the structure of a real-time, arbitrary neural artistic stylization network
In this paper, we present a method which combines the flexibility of the neural algorithm of artistic style with the speed of fast style transfer networks to allow real-time stylization using any content/style image pair.
Pixel Recursive Super Resolution
A low resolution image may correspond to multiple plausible high resolution images, thus modeling the super resolution process with a pixel independent conditional model often results in averaging different details--hence blurry edges.
YouTube-BoundingBoxes: A Large High-Precision Human-Annotated Data Set for Object Detection in Video
We introduce a new large-scale data set of video URLs with densely-sampled object bounding box annotations called YouTube-BoundingBoxes (YT-BB).
Conditional Image Synthesis With Auxiliary Classifier GANs
We expand on previous work for image quality assessment to provide two new analyses for assessing the discriminability and diversity of samples from class-conditional image synthesis models.
Ranked #14 on
Conditional Image Generation
on CIFAR-10
(Inception score metric)
A Learned Representation For Artistic Style
In this work we investigate the construction of a single, scalable deep network that can parsimoniously capture the artistic style of a diversity of paintings.
TensorFlow: Large-Scale Machine Learning on Heterogeneous Distributed Systems
TensorFlow is an interface for expressing machine learning algorithms, and an implementation for executing such algorithms.
Rethinking the Inception Architecture for Computer Vision
Convolutional networks are at the core of most state-of-the-art computer vision solutions for a wide variety of tasks.
Ranked #8 on
Retinal OCT Disease Classification
on OCT2017
Adversarial Autoencoders
In this paper, we propose the "adversarial autoencoder" (AAE), which is a probabilistic autoencoder that uses the recently proposed generative adversarial networks (GAN) to perform variational inference by matching the aggregated posterior of the hidden code vector of the autoencoder with an arbitrary prior distribution.
Ranked #7 on
Unsupervised MNIST
on MNIST
Net2Net: Accelerating Learning via Knowledge Transfer
Our Net2Net technique accelerates the experimentation process by instantaneously transferring the knowledge from a previous network to each new deeper or wider network.
Deep Networks With Large Output Spaces
Deep neural networks have been extremely successful at various image, speech, video recognition tasks because of their ability to model deep structures within the data.
Explaining and Harnessing Adversarial Examples
Several machine learning models, including neural networks, consistently misclassify adversarial examples---inputs formed by applying small but intentionally worst-case perturbations to examples from the dataset, such that the perturbed input results in the model outputting an incorrect answer with high confidence.
Ranked #44 on
Image Classification
on MNIST
A Tutorial on Independent Component Analysis
This tutorial provides an introduction to ICA based on linear algebra formulating an intuition for ICA from first principles.
Notes on Generalized Linear Models of Neurons
Experimental neuroscience increasingly requires tractable models for analyzing and predicting the behavior of neurons and networks.
A Tutorial on Principal Component Analysis
Principal component analysis (PCA) is a mainstay of modern data analysis - a black box that is widely used but (sometimes) poorly understood.
Zero-Shot Learning by Convex Combination of Semantic Embeddings
In other cases the semantic embedding space is established by an independent natural language processing task, and then the image transformation into that space is learned in a second stage.
Ranked #8 on
Multi-label zero-shot learning
on Open Images V4
Using Web Co-occurrence Statistics for Improving Image Categorization
Albeit the simplicity of the resulting optimization problem, it is effective in improving both recognition and localization accuracy.
Fast, Accurate Detection of 100,000 Object Classes on a Single Machine
Many object detection systems are constrained by the time required to convolve a target image with a bank of filters that code for different aspects of an object's appearance, such as the presence of component parts.
