Search Results for author:
Found 99 papers, 56 papers with code
Activity Regularization for Continual Learning
While deep neural networks have achieved remarkable successes, they suffer the well-known catastrophic forgetting issue when switching from existing tasks to tackle a new one.
What Are We Measuring When We Evaluate Large Vision-Language Models? An Analysis of Latent Factors and Biases
Vision-language (VL) models, pretrained on colossal image-text datasets, have attained broad VL competence that is difficult to evaluate.
A Survey of Few-Shot Learning on Graphs: from Meta-Learning to Pre-Training and Prompt Learning
The techniques can be broadly categorized into meta-learning, pre-training, and hybrid approaches, with a finer-grained classification in each category to aid readers in their method selection process.
RAP-Gen: Retrieval-Augmented Patch Generation with CodeT5 for Automatic Program Repair
Automatic program repair (APR) is crucial to reduce manual debugging efforts for developers and improve software reliability.
PyRCA: A Library for Metric-based Root Cause Analysis
We introduce PyRCA, an open-source Python machine learning library of Root Cause Analysis (RCA) for Artificial Intelligence for IT Operations (AIOps).
OTW: Optimal Transport Warping for Time Series
Dynamic Time Warping (DTW) has become the pragmatic choice for measuring distance between time series.
CodeTF: One-stop Transformer Library for State-of-the-art Code LLM
In this paper, we present CodeTF, an open-source Transformer-based library for state-of-the-art Code LLMs and code intelligence.
BLIP-Diffusion: Pre-trained Subject Representation for Controllable Text-to-Image Generation and Editing
Then we design a subject representation learning task which enables a diffusion model to leverage such visual representation and generates new subject renditions.
Ranked #6 on
Personalized Image Generation
on DreamBooth
CodeT5+: Open Code Large Language Models for Code Understanding and Generation
To address these limitations, we propose ``CodeT5+'', a family of encoder-decoder LLMs for code in which component modules can be flexibly combined to suit a wide range of downstream code tasks.
Ranked #1 on
Code Search
on CodeXGLUE - AdvTest
AI for IT Operations (AIOps) on Cloud Platforms: Reviews, Opportunities and Challenges
There are a wide variety of problems to address, and multiple use-cases, where AI capabilities can be leveraged to enhance operational efficiency.
From Images to Textual Prompts: Zero-shot VQA with Frozen Large Language Models
To address this issue, we propose \emph{Img2Prompt}, a plug-and-play module that provides the prompts that can bridge the aforementioned modality and task disconnections, so that LLMs can perform zero-shot VQA tasks without end-to-end training.
Learning Label Modular Prompts for Text Classification in the Wild
Machine learning models usually assume i. i. d data during training and testing, but data and tasks in real world often change over time.
Plug-and-Play VQA: Zero-shot VQA by Conjoining Large Pretrained Models with Zero Training
Visual question answering (VQA) is a hallmark of vision and language reasoning and a challenging task under the zero-shot setting.
Ranked #2 on
Visual Question Answering (VQA)
on VQA v2 val
LAVIS: A Library for Language-Vision Intelligence
We introduce LAVIS, an open-source deep learning library for LAnguage-VISion research and applications.
Continual Learning, Fast and Slow
Motivated by this theory, we propose \emph{DualNets} (for Dual Networks), a general continual learning framework comprising a fast learning system for supervised learning of pattern-separated representation from specific tasks and a slow learning system for representation learning of task-agnostic general representation via Self-Supervised Learning (SSL).
Paired Cross-Modal Data Augmentation for Fine-Grained Image-to-Text Retrieval
When we do online paired data augmentation, we first generate augmented text through random token replacement, then pass the augmented text into the latent space alignment module to output the latent codes, which are finally fed to StyleGAN2 to generate the augmented images.
3D Cartoon Face Generation with Controllable Expressions from a Single GAN Image
To this end, we discover the semantic meanings of StyleGAN latent space, such that we are able to produce face images of various expressions, poses, and lighting by controlling the latent codes.
CodeRL: Mastering Code Generation through Pretrained Models and Deep Reinforcement Learning
To address the limitations, we propose "CodeRL", a new framework for program synthesis tasks through pretrained LMs and deep reinforcement learning (RL).
Ranked #7 on
Code Generation
on APPS
A Causal Approach to Detecting Multivariate Time-series Anomalies and Root Causes
In light of the modularity property of causal systems (the causal processes to generate different variables are irrelevant modules), the original problem is divided into a series of separate, simpler, and low-dimensional anomaly detection problems so that where an anomaly happens (root causes) can be directly identified.
Multimodal Dialogue State Tracking
Specifically, we introduce a novel dialogue state tracking task to track the information of visual objects that are mentioned in video-grounded dialogues.
Masked Unsupervised Self-training for Label-free Image Classification
We demonstrate the efficacy of MUST on a variety of downstream tasks, where it improves upon CLIP by a large margin.
OmniXAI: A Library for Explainable AI
We introduce OmniXAI (short for Omni eXplainable AI), an open-source Python library of eXplainable AI (XAI), which offers omni-way explainable AI capabilities and various interpretable machine learning techniques to address the pain points of understanding and interpreting the decisions made by machine learning (ML) in practice.
MACE: An Efficient Model-Agnostic Framework for Counterfactual Explanation
Counterfactual explanation is an important Explainable AI technique to explain machine learning predictions.
Vector-Quantized Input-Contextualized Soft Prompts for Natural Language Understanding
VIP particularly focuses on two aspects -- contextual prompts that learns input-specific contextualization of the soft prompt tokens through a small-scale sentence encoder and quantized prompts that maps the contextualized prompts to a set of learnable codebook vectors through a Vector quantization network.
Mining Root Cause Knowledge from Cloud Service Incident Investigations for AIOps
ICA forms the backbone of a simple-yet-effective Retrieval based RCA for new incidents, through an Information Retrieval system to search and rank past incidents and detect likely root causes from them, given the incident symptom.
Learning Fast and Slow for Online Time Series Forecasting
The fast adaptation capability of deep neural networks in non-stationary environments is critical for online time series forecasting.
Align and Prompt: Video-and-Language Pre-training with Entity Prompts
To achieve this, we first introduce an entity prompter module, which is trained with VTC to produce the similarity between a video crop and text prompts instantiated with entity names.
Ranked #20 on
Zero-Shot Video Retrieval
on DiDeMo
Node-wise Localization of Graph Neural Networks
Ideally, how a node receives its neighborhood information should be a function of its local context, to diverge from the global GNN model shared by all nodes.
Improving Tail-Class Representation with Centroid Contrastive Learning
ICCL interpolates two images from a class-agnostic sampler and a class-aware sampler, and trains the model such that the representation of the interpolative image can be used to retrieve the centroids for both source classes.
Ranked #22 on
Long-tail Learning
on CIFAR-10-LT (ρ=10)
Cascaded Fast and Slow Models for Efficient Semantic Code Search
The goal of natural language semantic code search is to retrieve a semantically relevant code snippet from a fixed set of candidates using a natural language query.
Learning Structural Representations for Recipe Generation and Food Retrieval
Our approach brings together several novel ideas in a systematic framework: (1) exploiting an unsupervised learning approach to obtain the sentence-level tree structure labels before training; (2) generating trees of target recipes from images with the supervision of tree structure labels learned from (1); and (3) integrating the learned tree structures into the recipe generation and food cross-modal retrieval procedure.
CodeT5: Identifier-aware Unified Pre-trained Encoder-Decoder Models for Code Understanding and Generation
We present CodeT5, a unified pre-trained encoder-decoder Transformer model that better leverages the code semantics conveyed from the developer-assigned identifiers.
Ranked #1 on
Text-to-Code Generation
on CodeXGLUE - CONCODE
Cross-Modal Graph with Meta Concepts for Video Captioning
Video captioning targets interpreting the complex visual contents as text descriptions, which requires the model to fully understand video scenes including objects and their interactions.
Cycle-Consistent Inverse GAN for Text-to-Image Synthesis
In this paper, we propose a novel unified framework of Cycle-consistent Inverse GAN (CI-GAN) for both text-to-image generation and text-guided image manipulation tasks.
$C^3$: Compositional Counterfactual Contrastive Learning for Video-grounded Dialogues
Video-grounded dialogue systems aim to integrate video understanding and dialogue understanding to generate responses that are relevant to both the dialogue and video context.
A Large-Scale Benchmark for Food Image Segmentation
Existing food image segmentation models are underperforming due to two reasons: (1) there is a lack of high quality food image datasets with fine-grained ingredient labels and pixel-wise location masks -- the existing datasets either carry coarse ingredient labels or are small in size; and (2) the complex appearance of food makes it difficult to localize and recognize ingredients in food images, e. g., the ingredients may overlap one another in the same image, and the identical ingredient may appear distinctly in different food images.
Ranked #3 on
Semantic Segmentation
on FoodSeg103
(using extra training data)
VGNMN: Video-grounded Neural Module Network to Video-Grounded Language Tasks
Neural module networks (NMN) have achieved success in image-grounded tasks such as Visual Question Answering (VQA) on synthetic images.
Learning Reasoning Paths over Semantic Graphs for Video-grounded Dialogues
PDC model then learns to predict reasoning paths over this semantic graph.
Weakly Supervised Neuro-Symbolic Module Networks for Numerical Reasoning
Neural Module Networks (NMNs) have been quite successful in incorporating explicit reasoning as learnable modules in various question answering tasks, including the most generic form of numerical reasoning over text in Machine Reading Comprehension (MRC).
Weakly-Supervised Multi-Face 3D Reconstruction
3D face reconstruction plays a very important role in many real-world multimedia applications, including digital entertainment, social media, affection analysis, and person identification.
Learning Transferrable Parameters for Long-tailed Sequential User Behavior Modeling
Sequential user behavior modeling plays a crucial role in online user-oriented services, such as product purchasing, news feed consumption, and online advertising.
BiST: Bi-directional Spatio-Temporal Reasoning for Video-Grounded Dialogues
Video-grounded dialogues are very challenging due to (i) the complexity of videos which contain both spatial and temporal variations, and (ii) the complexity of user utterances which query different segments and/or different objects in videos over multiple dialogue turns.
Response Selection for Multi-Party Conversations withDynamic Topic Tracking
While participants in a multi-party multi-turn conversation simultaneously engage in multiple conversation topics, existing response selection methods are developed mainly focusing on a two-party single-conversation scenario.
Discern: Discourse-Aware Entailment Reasoning Network for Conversational Machine Reading
Based on the learned EDU and entailment representations, we either reply to the user our final decision "yes/no/irrelevant" of the initial question, or generate a follow-up question to inquiry more information.
MoPro: Webly Supervised Learning with Momentum Prototypes
We propose momentum prototypes (MoPro), a simple contrastive learning method that achieves online label noise correction, out-of-distribution sample removal, and representation learning.
Ranked #12 on
Image Classification
on OmniBenchmark
(using extra training data)
Structure-Aware Generation Network for Recipe Generation from Images
We investigate an open research task of generating cooking instructions based on only food images and ingredients, which is similar to the image captioning task.
Bilevel Continual Learning
Continual learning aims to learn continuously from a stream of tasks and data in an online-learning fashion, being capable of exploiting what was learned previously to improve current and future tasks while still being able to perform well on the previous tasks.
Photon: A Robust Cross-Domain Text-to-SQL System
Natural language interfaces to databases (NLIDB) democratize end user access to relational data.
Decomposing Generation Networks with Structure Prediction for Recipe Generation
Recipe generation from food images and ingredients is a challenging task, which requires the interpretation of the information from another modality.
Multi-label Thoracic Disease Image Classification with Cross-Attention Networks
Automated disease classification of radiology images has been emerging as a promising technique to support clinical diagnosis and treatment planning.
Adaptive Task Sampling for Meta-Learning
Meta-learning methods have been extensively studied and applied in computer vision, especially for few-shot classification tasks.
Explicit Memory Tracker with Coarse-to-Fine Reasoning for Conversational Machine Reading
The goal of conversational machine reading is to answer user questions given a knowledge base text which may require asking clarification questions.
Theory-Inspired Path-Regularized Differential Network Architecture Search
Then we propose a theory-inspired path-regularized DARTS that consists of two key modules: (i) a differential group-structured sparse binary gate introduced for each operation to avoid unfair competition among operations, and (ii) a path-depth-wise regularization used to incite search exploration for deep architectures that often converge slower than shallow ones as shown in our theory and are not well explored during the search.
Video-Grounded Dialogues with Pretrained Generation Language Models
Pre-trained language models have shown remarkable success in improving various downstream NLP tasks due to their ability to capture dependencies in textual data and generate natural responses.
EMT: Explicit Memory Tracker with Coarse-to-Fine Reasoning for Conversational Machine Reading
The goal of conversational machine reading is to answer user questions given a knowledge base text which may require asking clarification questions.
Prototypical Contrastive Learning of Unsupervised Representations
This paper presents Prototypical Contrastive Learning (PCL), an unsupervised representation learning method that addresses the fundamental limitations of instance-wise contrastive learning.
Ranked #5 on
Contrastive Learning
on imagenet-1k
UniConv: A Unified Conversational Neural Architecture for Multi-domain Task-oriented Dialogues
Building an end-to-end conversational agent for multi-domain task-oriented dialogues has been an open challenge for two main reasons.
VD-BERT: A Unified Vision and Dialog Transformer with BERT
By contrast, in this work, we propose VD-BERT, a simple yet effective framework of unified vision-dialog Transformer that leverages the pretrained BERT language models for Visual Dialog tasks.
Learning Video Object Segmentation from Unlabeled Videos
We propose a new method for video object segmentation (VOS) that addresses object pattern learning from unlabeled videos, unlike most existing methods which rely heavily on extensive annotated data.
Cross-Modal Food Retrieval: Learning a Joint Embedding of Food Images and Recipes with Semantic Consistency and Attention Mechanism
Food retrieval is an important task to perform analysis of food-related information, where we are interested in retrieving relevant information about the queried food item such as ingredients, cooking instructions, etc.
Ranked #7 on
Cross-Modal Retrieval
on Recipe1M
Tree-structured Attention with Hierarchical Accumulation
Incorporating hierarchical structures like constituency trees has been shown to be effective for various natural language processing (NLP) tasks.
Non-Autoregressive Dialog State Tracking
Recent efforts in Dialogue State Tracking (DST) for task-oriented dialogues have progressed toward open-vocabulary or generation-based approaches where the models can generate slot value candidates from the dialogue history itself.
Ranked #13 on
Multi-domain Dialogue State Tracking
on MULTIWOZ 2.0
DivideMix: Learning with Noisy Labels as Semi-supervised Learning
Two prominent directions include learning with noisy labels and semi-supervised learning by exploiting unlabeled data.
Ranked #4 on
Learning with noisy labels
on CIFAR-100N
Deep Learning for Person Re-identification: A Survey and Outlook
The widely studied closed-world setting is usually applied under various research-oriented assumptions, and has achieved inspiring success using deep learning techniques on a number of datasets.
Ranked #1 on
Cross-Modal Person Re-Identification
on RegDB-C
Detecting Cyberattacks in Industrial Control Systems Using Online Learning Algorithms
Similar to other information systems, a significant threat to industrial control systems is the attack from cyberspace---the offensive maneuvers launched by "anonymous" in the digital world that target computer-based assets with the goal of compromising a system's functions or probing for information.
Attribute-aware Pedestrian Detection in a Crowd
To tackle this critical problem, we propose an attribute-aware pedestrian detector to explicitly model people's semantic attributes in a high-level feature detection fashion.
FoodAI: Food Image Recognition via Deep Learning for Smart Food Logging
FoodAI has made food logging convenient, aiding smart consumption and a healthy lifestyle.
Meta-RCNN: Meta Learning for Few-Shot Object Detection
Specifically, Meta-RCNN learns an object detector in an episodic learning paradigm on the (meta) training data.
Ranked #7 on
Cross-Domain Few-Shot Object Detection
on DIOR
Recent Advances in Deep Learning for Object Detection
Object detection is a fundamental visual recognition problem in computer vision and has been widely studied in the past decades.
Multimodal Transformer Networks for End-to-End Video-Grounded Dialogue Systems
Developing Video-Grounded Dialogue Systems (VGDS), where a dialogue is conducted based on visual and audio aspects of a given video, is significantly more challenging than traditional image or text-grounded dialogue systems because (1) feature space of videos span across multiple picture frames, making it difficult to obtain semantic information; and (2) a dialogue agent must perceive and process information from different modalities (audio, video, caption, etc.)
Ranked #4 on
Response Generation
on SIMMC2.0
Compositional Coding for Collaborative Filtering
However, CF with binary codes naturally suffers from low accuracy due to limited representation capability in each bit, which impedes it from modeling complex structure of the data.
Learning Cross-Modal Embeddings with Adversarial Networks for Cooking Recipes and Food Images
Food computing is playing an increasingly important role in human daily life, and has found tremendous applications in guiding human behavior towards smart food consumption and healthy lifestyle.
Ranked #8 on
Cross-Modal Retrieval
on Recipe1M
Meta-Learning with Domain Adaptation for Few-Shot Learning under Domain Shift
Most existing work assumes that both training and test tasks are drawn from the same distribution, and a large amount of labeled data is available in the training tasks.
Deep Learning for Image Super-resolution: A Survey
Image Super-Resolution (SR) is an important class of image processing techniques to enhance the resolution of images and videos in computer vision.
Semi-supervised deep embedded clustering
Deep embedded clustering (DEC) is one of the state-of-the-art deep clustering methods.
Robust Graph Learning from Noisy Data
The proposed model is able to boost the performance of data clustering, semisupervised classification, and data recovery significantly, primarily due to two key factors: 1) enhanced low-rank recovery by exploiting the graph smoothness assumption, 2) improved graph construction by exploiting clean data recovered by robust PCA.
Adaptive Cost-sensitive Online Classification
Cost-Sensitive Online Classification has drawn extensive attention in recent years, where the main approach is to directly online optimize two well-known cost-sensitive metrics: (i) weighted sum of sensitivity and specificity; (ii) weighted misclassification cost.
Single-Shot Bidirectional Pyramid Networks for High-Quality Object Detection
Recent years have witnessed many exciting achievements for object detection using deep learning techniques.
URLNet: Learning a URL Representation with Deep Learning for Malicious URL Detection
This approach allows the model to capture several types of semantic information, which was not possible by the existing models.
Online Learning: A Comprehensive Survey
Online learning represents an important family of machine learning algorithms, in which a learner attempts to resolve an online prediction (or any type of decision-making) task by learning a model/hypothesis from a sequence of data instances one at a time.
Robust Estimation of Similarity Transformation for Visual Object Tracking
In order to efficiently search in such a large 4-DoF space in real-time, we formulate the problem into two 2-DoF sub-problems and apply an efficient Block Coordinates Descent solver to optimize the estimation result.
Ranked #37 on
Video Object Tracking
on NT-VOT211
Feature Agglomeration Networks for Single Stage Face Detection
In this paper, we propose a novel simple yet effective framework of "Feature Agglomeration Networks" (FANet) to build a new single stage face detector, which not only achieves state-of-the-art performance but also runs efficiently.
Online Deep Learning: Learning Deep Neural Networks on the Fly
Deep Neural Networks (DNNs) are typically trained by backpropagation in a batch learning setting, which requires the entire training data to be made available prior to the learning task.
Projection-free Distributed Online Learning in Networks
The conditional gradient algorithm has regained a surge of research interest in recent years due to its high efficiency in handling large-scale machine learning problems.
Face Detection using Deep Learning: An Improved Faster RCNN Approach
In this report, we present a new face detection scheme using deep learning and achieve the state-of-the-art detection performance on the well-known FDDB face detetion benchmark evaluation.
Malicious URL Detection using Machine Learning: A Survey
This article aims to provide a comprehensive survey and a structural understanding of Malicious URL Detection techniques using machine learning.
SOL: A Library for Scalable Online Learning Algorithms
SOL is an open-source library for scalable online learning algorithms, and is particularly suitable for learning with high-dimensional data.
Online Bayesian Collaborative Topic Regression
In this paper, we propose a novel scheme of Online Bayesian Collaborative Topic Regression (OBCTR) which is efficient and scalable for learning from data streams.
Scalable Image Retrieval by Sparse Product Quantization
Unlike the existing approaches, in this paper, we propose a novel approach called Sparse Product Quantization (SPQ) to encoding the high-dimensional feature vectors into sparse representation.
Adaptive Subgradient Methods for Online AUC Maximization
Despite their encouraging results reported, the existing online AUC maximization algorithms often adopt simple online gradient descent approaches that fail to exploit the geometrical knowledge of the data observed during the online learning process, and thus could suffer from relatively larger regret.
Budget Online Multiple Kernel Learning
To overcome this drawback, we present a novel framework of Budget Online Multiple Kernel Learning (BOMKL) and propose a new Sparse Passive Aggressive learning to perform effective budget online learning.
LOGO-Net: Large-scale Deep Logo Detection and Brand Recognition with Deep Region-based Convolutional Networks
In this paper, we introduce "LOGO-Net", a large-scale logo image database for logo detection and brand recognition from real-world product images.
A Framework of Sparse Online Learning and Its Applications
Unlike some existing online data stream classification techniques that are often based on first-order online learning, we propose a framework of Sparse Online Classification (SOC) for data stream classification, which includes some state-of-the-art first-order sparse online learning algorithms as special cases and allows us to derive a new effective second-order online learning algorithm for data stream classification.
Reliable Patch Trackers: Robust Visual Tracking by Exploiting Reliable Patches
Most modern trackers typically employ a bounding box given in the first frame to track visual objects, where their tracking results are often sensitive to the initialization.
Large-scale Online Feature Selection for Ultra-high Dimensional Sparse Data
However, unlike many second-order learning methods that often suffer from extra high computational cost, we devise a novel smart algorithm for second-order online feature selection using a MaxHeap-based approach, which is not only more effective than the existing first-order approaches, but also significantly more efficient and scalable for large-scale feature selection with ultra-high dimensional sparse data, as validated from our extensive experiments.
Online Portfolio Selection: A Survey
This article aims to provide a timely and comprehensive survey for both machine learning and data mining researchers in academia and quantitative portfolio managers in the financial industry to help them understand the state-of-the-art and facilitate their research and practical applications.
On-Line Portfolio Selection with Moving Average Reversion
Online portfolio selection has attracted increasing interests in machine learning and AI communities recently.
PAMR: Passive aggressive mean reversion strategy for portfolio selection
This article proposes a novel online portfolio selection strategy named “Passive Aggressive Mean Reversion” (PAMR).
CORN: Correlation-driven Non parametric Learning Approach for Portfolio Selection
Machine learning techniques have been adopted to select portfolios from financial markets in some emerging intelligent business applications.
