Search Results for author:
Found 45 papers, 24 papers with code
Learning Equi-angular Representations for Online Continual Learning
Online continual learning suffers from an underfitted solution due to insufficient training for prompt model update (e. g., single-epoch training).
Just Say the Name: Online Continual Learning with Category Names Only via Data Generation
In real-world scenarios, extensive manual annotation for continual learning is impractical due to prohibitive costs.
Online Continual Learning For Interactive Instruction Following Agents
To take a step towards a more realistic embodied agent learning scenario, we propose two continual learning setups for embodied agents; learning new behaviors (Behavior Incremental Learning, Behavior-IL) and new environments (Environment Incremental Learning, Environment-IL) For the tasks, previous 'data prior' based continual learning methods maintain logits for the past tasks.
PAC-FNO: Parallel-Structured All-Component Fourier Neural Operators for Recognizing Low-Quality Images
Extensively evaluating methods with seven image recognition benchmarks, we show that the proposed PAC-FNO improves the performance of existing baseline models on images with various resolutions by up to 77. 1% and various types of natural variations in the images at inference.
Tuning Large Multimodal Models for Videos using Reinforcement Learning from AI Feedback
Recent advancements in large language models have influenced the development of video large multimodal models (VLMMs).
Operator-learning-inspired Modeling of Neural Ordinary Differential Equations
Neural ordinary differential equations (NODEs), one of the most influential works of the differential equation-based deep learning, are to continuously generalize residual networks and opened a new field.
Online Continual Learning on Hierarchical Label Expansion
To address this challenge, we propose a novel multi-level hierarchical class incremental task configuration with an online learning constraint, called hierarchical label expansion (HLE).
Multi-Level Compositional Reasoning for Interactive Instruction Following
At the middle level, we discriminatively control the agent's navigation by a master policy by alternating between a navigation policy and various independent interaction policies.
Story Visualization by Online Text Augmentation with Context Memory
Story visualization (SV) is a challenging text-to-image generation task for the difficulty of not only rendering visual details from the text descriptions but also encoding a long-term context across multiple sentences.
Context-Aware Planning and Environment-Aware Memory for Instruction Following Embodied Agents
Accomplishing household tasks requires to plan step-by-step actions considering the consequences of previous actions.
Cost-effective On-device Continual Learning over Memory Hierarchy with Miro
Continual learning (CL) trains NN models incrementally from a continuous stream of tasks.
EaSyGuide : ESG Issue Identification Framework leveraging Abilities of Generative Large Language Models
These outcomes underscore the effectiveness of our methodology in identifying ESG issues in news articles across different languages.
MEnsA: Mix-up Ensemble Average for Unsupervised Multi Target Domain Adaptation on 3D Point Clouds
Unsupervised domain adaptation (UDA) addresses the problem of distribution shift between the unlabelled target domain and labelled source domain.
Ask4Help: Learning to Leverage an Expert for Embodied Tasks
Embodied AI agents continue to become more capable every year with the advent of new models, environments, and benchmarks, but are still far away from being performant and reliable enough to be deployed in real, user-facing, applications.
Attentive Fine-Grained Structured Sparsity for Image Restoration
Image restoration tasks have witnessed great performance improvement in recent years by developing large deep models.
Online Continual Learning on a Contaminated Data Stream with Blurry Task Boundaries
A large body of continual learning (CL) methods, however, assumes data streams with clean labels, and online learning scenarios under noisy data streams are yet underexplored.
Stereo Depth From Events Cameras: Concentrate and Focus on the Future
To alleviate the event missing or overriding issue, we propose to learn to concentrate on the dense events to produce a compact event representation with high details for depth estimation.
Iconary: A Pictionary-Based Game for Testing Multimodal Communication with Drawings and Text
We investigate these challenges in the context of Iconary, a collaborative game of drawing and guessing based on Pictionary, that poses a novel challenge for the research community.
Online Continual Learning on Class Incremental Blurry Task Configuration with Anytime Inference
For better practicality, we first propose a novel continual learning setup that is online, task-free, class-incremental, of blurry task boundaries and subject to inference queries at any moment.
Unsupervised Representation Learning for Binary Networks by Joint Classifier Learning
To accelerate deployment of models with the benefit of unsupervised representation learning to such resource limited devices for various downstream tasks, we propose a self-supervised learning method for binary networks that uses a moving target network.
BNAS v2: Learning Architectures for Binary Networks with Empirical Improvements
Questioning that the architectures designed for FP networks might not be the best for binary networks, we propose to search architectures for binary networks (BNAS) by defining a new search space for binary architectures and a novel search objective.
Carousel Memory: Rethinking the Design of Episodic Memory for Continual Learning
In particular, in mobile and IoT devices, real-time data can be stored not just in high-speed RAMs but in internal storage devices as well, which offer significantly larger capacity than the RAMs.
Hierarchical Modular Framework for Long Horizon Instruction Following
To address such composite tasks, we propose a hierarchical modular approach to learn agents that navigate and manipulate objects in a divide-and-conquer manner for the diverse nature of the entailing tasks.
Chameleon Sampling: Diverse and Pure Example Selection for Online Continual Learning with Noisy Labels
Specifically, we argue the importance of both diversity and purity of examples in the episodic memory of continual learning models.
Zero-shot Natural Language Video Localization
Understanding videos to localize moments with natural language often requires large expensive annotated video regions paired with language queries.
Rethinking Deep Image Prior for Denoising
Deep image prior (DIP) serves as a good inductive bias for diverse inverse problems.
Rainbow Memory: Continual Learning with a Memory of Diverse Samples
Prevalent scenario of continual learning, however, assumes disjoint sets of classes as tasks and is less realistic rather artificial.
Event-Intensity Stereo: Estimating Depth by the Best of Both Worlds
Event cameras can report scene movements as an asynchronous stream of data called the events.
Learning to Solve Nonlinear Partial Differential Equation Systems To Accelerate MOSFET Simulation
In order to accelerate the semiconductor device simulation, we propose to use a neural network to learn an approximate solution for desired boundary conditions.
Learning the Connections in Direct Feedback Alignment
We propose to learn the backward weight matrices in DFA, adopting the methodology of Kolen-Pollack learning, to improve training and inference accuracy in deep convolutional neural networks by updating the direct feedback connections such that they come to estimate the forward path.
Ranked #177 on
Image Classification
on CIFAR-100
Factorizing Perception and Policy for Interactive Instruction Following
Performing simple household tasks based on language directives is very natural to humans, yet it remains an open challenge for AI agents.
Learning Visual Representations for Transfer Learning by Suppressing Texture
Recent literature has shown that features obtained from supervised training of CNNs may over-emphasize texture rather than encoding high-level information.
Ranked #19 on
Object Detection
on PASCAL VOC 2007
Learning Architectures for Binary Networks
Specifically, based on the cell based search method, we define the new search space of binary layer types, design a new cell template, and rediscover the utility of and propose to use the Zeroise layer instead of using it as a placeholder.
Learning to Super Resolve Intensity Images from Events
An event camera detects per-pixel intensity difference and produces asynchronous event stream with low latency, high dynamic range, and low power consumption.
Incremental Learning with Maximum Entropy Regularization: Rethinking Forgetting and Intransigence
Incremental learning suffers from two challenging problems; forgetting of old knowledge and intransigence on learning new knowledge.
Training with the Invisibles: Obfuscating Images to Share Safely for Learning Visual Recognition Models
High-performance visual recognition systems generally require a large collection of labeled images to train.
ScreenerNet: Learning Self-Paced Curriculum for Deep Neural Networks
We propose to learn a curriculum or a syllabus for supervised learning and deep reinforcement learning with deep neural networks by an attachable deep neural network, called ScreenerNet.
Structured Set Matching Networks for One-Shot Part Labeling
Diagrams often depict complex phenomena and serve as a good test bed for visual and textual reasoning.
Are You Smarter Than a Sixth Grader? Textbook Question Answering for Multimodal Machine Comprehension
Our analysis shows that a significant portion of questions require complex parsing of the text and the diagrams and reasoning, indicating that our dataset is more complex compared to previous machine comprehension and visual question answering datasets.
ActionFlowNet: Learning Motion Representation for Action Recognition
Even with the recent advances in convolutional neural networks (CNN) in various visual recognition tasks, the state-of-the-art action recognition system still relies on hand crafted motion feature such as optical flow to achieve the best performance.
Ranked #69 on
Action Recognition
on HMDB-51
Mining Discriminative Triplets of Patches for Fine-Grained Classification
Fine-grained classification involves distinguishing between similar sub-categories based on subtle differences in highly localized regions; therefore, accurate localization of discriminative regions remains a major challenge.
Learning Temporal Regularity in Video Sequences
Perceiving meaningful activities in a long video sequence is a challenging problem due to ambiguous definition of 'meaningfulness' as well as clutters in the scene.
Ranked #2 on
Traffic Accident Detection
on A3D
Comparing apples to apples in the evaluation of binary coding methods
We discuss methodological issues related to the evaluation of unsupervised binary code construction methods for nearest neighbor search.
Multi-Directional Multi-Level Dual-Cross Patterns for Robust Face Recognition
To perform unconstrained face recognition robust to variations in illumination, pose and expression, this paper presents a new scheme to extract "Multi-Directional Multi-Level Dual-Cross Patterns" (MDML-DCPs) from face images.
Adding Unlabeled Samples to Categories by Learned Attributes
We propose a method to expand the visual coverage of training sets that consist of a small number of labeled examples using learned attributes.
