Search Results for author:
Found 46 papers, 20 papers with code
BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and Generation
Furthermore, performance improvement has been largely achieved by scaling up the dataset with noisy image-text pairs collected from the web, which is a suboptimal source of supervision.
Ranked #3 on
Open Vocabulary Attribute Detection
on OVAD-Box benchmark
(using extra training data)
BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Models
The cost of vision-and-language pre-training has become increasingly prohibitive due to end-to-end training of large-scale models.
Ranked #1 on
Image Retrieval
on Flickr30k
RegNet: Self-Regulated Network for Image Classification
The ResNet and its variants have achieved remarkable successes in various computer vision tasks.
Ranked #3 on
Medical Image Classification
on NCT-CRC-HE-100K
Align before Fuse: Vision and Language Representation Learning with Momentum Distillation
Most existing methods employ a transformer-based multimodal encoder to jointly model visual tokens (region-based image features) and word tokens.
Ranked #5 on
Open Vocabulary Attribute Detection
on OVAD-Box benchmark
(using extra training data)
InstructBLIP: Towards General-purpose Vision-Language Models with Instruction Tuning
Large-scale pre-training and instruction tuning have been successful at creating general-purpose language models with broad competence.
Ranked #5 on
visual instruction following
on LLaVA-Bench
Merlion: A Machine Learning Library for Time Series
We introduce Merlion, an open-source machine learning library for time series.
ETSformer: Exponential Smoothing Transformers for Time-series Forecasting
Transformers have been actively studied for time-series forecasting in recent years.
LogAI: A Library for Log Analytics and Intelligence
In order to enable users to perform multiple types of AI-based log analysis tasks in a uniform manner, we introduce LogAI (https://github. com/salesforce/logai), a one-stop open source library for log analytics and intelligence.
Learning Deep Time-index Models for Time Series Forecasting
Deep learning has been actively applied to time series forecasting, leading to a deluge of new methods, belonging to the class of historical-value models.
TOD-BERT: Pre-trained Natural Language Understanding for Task-Oriented Dialogue
The underlying difference of linguistic patterns between general text and task-oriented dialogue makes existing pre-trained language models less useful in practice.
CoST: Contrastive Learning of Disentangled Seasonal-Trend Representations for Time Series Forecasting
Motivated by the recent success of representation learning in computer vision and natural language processing, we argue that a more promising paradigm for time series forecasting, is to first learn disentangled feature representations, followed by a simple regression fine-tuning step -- we justify such a paradigm from a causal perspective.
CoMatch: Semi-supervised Learning with Contrastive Graph Regularization
CoMatch jointly learns two representations of the training data, their class probabilities and low-dimensional embeddings.
BotSIM: An End-to-End Bot Simulation Framework for Commercial Task-Oriented Dialog Systems
We have open-sourced the toolkit at https://github. com/salesforce/botsim
BotSIM: An End-to-End Bot Simulation Toolkit for Commercial Task-Oriented Dialog Systems
BotSIM adopts a layered design comprising the infrastructure layer, the adaptor layer and the application layer.
Classification Calibration for Long-tail Instance Segmentation
In this report, we investigate the performance drop phenomenon of state-of-the-art two-stage instance segmentation models when processing extreme long-tail training data based on the LVIS [5] dataset, and find a major cause is the inaccurate classification of object proposals.
The Devil is in Classification: A Simple Framework for Long-tail Object Detection and Instance Segmentation
Specifically, we systematically investigate performance drop of the state-of-the-art two-stage instance segmentation model Mask R-CNN on the recent long-tail LVIS dataset, and unveil that a major cause is the inaccurate classification of object proposals.
DualNet: Continual Learning, Fast and Slow
According to Complementary Learning Systems (CLS) theory~\citep{mcclelland1995there} in neuroscience, humans do effective \emph{continual learning} through two complementary systems: a fast learning system centered on the hippocampus for rapid learning of the specifics and individual experiences, and a slow learning system located in the neocortex for the gradual acquisition of structured knowledge about the environment.
HyperRouter: Towards Efficient Training and Inference of Sparse Mixture of Experts
By routing input tokens to only a few split experts, Sparse Mixture-of-Experts has enabled efficient training of large language models.
Continual Normalization: Rethinking Batch Normalization for Online Continual Learning
Existing continual learning methods use Batch Normalization (BN) to facilitate training and improve generalization across tasks.
Personalised Distillation: Empowering Open-Sourced LLMs with Adaptive Learning for Code Generation
With the rise of powerful closed-sourced LLMs (ChatGPT, GPT-4), there are increasing interests in distilling the capabilies of close-sourced LLMs to smaller open-sourced LLMs.
Ranked #48 on
Code Generation
on HumanEval
An Incremental Path-Following Splitting Method for Linearly Constrained Nonconvex Nonsmooth Programs
The stationary point of Problem 2 is NOT the stationary point of Problem 1.
Active Learning with Expert Advice
In this paper, we address a new problem of active learning with expert advice, where the outcome of an instance is disclosed only when it is requested by the online learner.
Question-Guided Hybrid Convolution for Visual Question Answering
Most state-of-the-art VQA methods fuse the high-level textual and visual features from the neural network and abandon the visual spatial information when learning multi-modal features. To address these problems, question-guided kernels generated from the input question are designed to convolute with visual features for capturing the textual and visual relationship in the early stage.
Ranked #14 on
Visual Question Answering (VQA)
on CLEVR
Dynamic Fusion with Intra- and Inter- Modality Attention Flow for Visual Question Answering
It can robustly capture the high-level interactions between language and vision domains, thus significantly improves the performance of visual question answering.
DART: Domain-Adversarial Residual-Transfer Networks for Unsupervised Cross-Domain Image Classification
In this paper, we propose a new unsupervised domain adaptation method named Domain-Adversarial Residual-Transfer (DART) learning of Deep Neural Networks to tackle cross-domain image classification tasks.
Distilled Siamese Networks for Visual Tracking
This model is intuitively inspired by the one teacher vs. multiple students learning method typically employed in schools.
Towards Noise-resistant Object Detection with Noisy Annotations
We address the challenging problem of training object detectors with noisy annotations, where the noise contains a mixture of label noise and bounding box noise.
Extreme Low-Light Imaging with Multi-granulation Cooperative Networks
Low-light imaging is challenging since images may appear to be dark and noised due to low signal-to-noise ratio, complex image content, and the variety in shooting scenes in extreme low-light condition.
Partially Observable Online Change Detection via Smooth-Sparse Decomposition
We consider online change detection of high dimensional data streams with sparse changes, where only a subset of data streams can be observed at each sensing time point due to limited sensing capacities.
Localized Meta-Learning: A PAC-Bayes Analysis for Meta-Learning Beyond Global Prior
Meta-learning methods learn the meta-knowledge among various training tasks and aim to promote the learning of new tasks under the task similarity assumption.
Online Continual Learning Under Domain Shift
CIER employs an adversarial training to correct the shift in $P(X, Y)$ by matching $P(X|Y)$, which results in an invariant representation that can generalize to unseen domains during inference.
Contextual Transformation Networks for Online Continual Learning
Continual learning methods with fixed architectures rely on a single network to learn models that can perform well on all tasks.
VilNMN: A Neural Module Network approach to Video-Grounded Language Tasks
Neural module networks (NMN) have achieved success in image-grounded tasks such as question answering (QA) on synthetic images.
PolarNet: Learning to Optimize Polar Keypoints for Keypoint Based Object Detection
Despite achieving promising performance at par with anchor-based detectors, the existing anchor-free detectors such as FCOS or CenterNet predict objects based on standard Cartesian coordinates, which often yield poor quality keypoints.
Noise-Robust Contrastive Learning
In contrast to most existing methods, we combat noise by learning robust representation.
Towards Theoretically Understanding Why SGD Generalizes Better Than ADAM in Deep Learning
The result shows that (1) the escaping time of both SGD and ADAM~depends on the Radon measure of the basin positively and the heaviness of gradient noise negatively; (2) for the same basin, SGD enjoys smaller escaping time than ADAM, mainly because (a) the geometry adaptation in ADAM~via adaptively scaling each gradient coordinate well diminishes the anisotropic structure in gradient noise and results in larger Radon measure of a basin; (b) the exponential gradient average in ADAM~smooths its gradient and leads to lighter gradient noise tails than SGD.
Improving Limited Labeled Dialogue State Tracking with Self-Supervision
We present and investigate two self-supervised objectives: preserving latent consistency and modeling conversational behavior.
Adapt-and-Adjust: Overcoming the Long-Tail Problem of Multilingual Speech Recognition
One crucial challenge of real-world multilingual speech recognition is the long-tailed distribution problem, where some resource-rich languages like English have abundant training data, but a long tail of low-resource languages have varying amounts of limited training data.
Detection and Rectification of Arbitrary Shaped Scene Texts by using Text Keypoints and Links
Detection and recognition of scene texts of arbitrary shapes remain a grand challenge due to the super-rich text shape variation in text line orientations, lengths, curvatures, etc.
A Theory-Driven Self-Labeling Refinement Method for Contrastive Representation Learning
Although intuitive, such a native label assignment strategy cannot reveal the underlying semantic similarity between a query and its positives and negatives, and impairs performance, since some negatives are semantically similar to the query or even share the same semantic class as the query.
Attention-based Feature Aggregation
To implement this module, we define two variants of attention: self-attention on the summed-up feature map, and cross-attention between two feature maps before summed up.
VGNMN: Video-grounded Neural Module Networks for Video-Grounded Dialogue Systems
Neural module networks (NMN) have achieved success in image-grounded tasks such as Visual Question Answering (VQA) on synthetic images.
Detect-Localize-Repair: A Unified Framework for Learning to Debug with CodeT5
Specifically, we propose three objectives to adapt the generic CodeT5 for debugging: a bug detection objective to determine whether a given code snippet is buggy or not, a bug localization objective to identify the buggy lines, and a program repair objective to translate the buggy code to its fixed version.
From Images to Textual Prompts: Zero-Shot Visual Question Answering With Frozen Large Language Models
To address this issue, we propose Img2Prompt, a plug-and-play module that provides the prompts that can bridge the aforementioned modality and task disconnections, so that LLMs can perform zero-shot VQA tasks without end-to-end training.
CompeteSMoE - Effective Training of Sparse Mixture of Experts via Competition
Sparse mixture of experts (SMoE) offers an appealing solution to scale up the model complexity beyond the mean of increasing the network's depth or width.
