Search Results for author:
Found 268 papers, 121 papers with code
Distractor-free Generalizable 3D Gaussian Splatting
To achieve these objectives, DGGS introduces a scene-agnostic reference-based mask prediction and refinement methodology during training phase, coupled with a training view selection strategy, effectively improving distractor prediction accuracy and training stability.
Reviving Dormant Memories: Investigating Catastrophic Forgetting in Language Models through Rationale-Guidance Difficulty
In this paper, we discover that when a forgetting model passively receives an externally provided partial appropriate rationale, its performance on the forgotten task can be restored.
PSPO*: An Effective Process-supervised Policy Optimization for Reasoning Alignment
Process supervision enhances the performance of large language models in reasoning tasks by providing feedback at each step of chain-of-thought reasoning.
METEOR: Evolutionary Journey of Large Language Models from Guidance to Self-Growth
Model evolution enables learning from feedback to refine experiences and update skills, transforming models from having no domain knowledge to becoming domain experts.
SRA-MCTS: Self-driven Reasoning Augmentation with Monte Carlo Tree Search for Code Generation
Large language models demonstrate exceptional performance in simple code generation tasks but still face challenges in tackling complex problems.
UniHOI: Learning Fast, Dense and Generalizable 4D Reconstruction for Egocentric Hand Object Interaction Videos
To address this, we introduce UniHOI, a model that unifies the estimation of all variables necessary for dense 4D reconstruction, including camera intrinsic, camera poses, and video depth, for egocentric HOI scene in a fast feed-forward manner.
Multi-Transmotion: Pre-trained Model for Human Motion Prediction
However, the complexity of human motion have prevented the development of a standardized dataset for human motion prediction, thereby hindering the establishment of pre-trained models.
START: A Generalized State Space Model with Saliency-Driven Token-Aware Transformation
Existing DG methods primarily rely on convolutional neural networks (CNNs), which inherently learn texture biases due to their limited receptive fields, making them prone to overfitting source domains.
RRM: Robust Reward Model Training Mitigates Reward Hacking
Our RRM improves the performance of a pairwise reward model trained on Gemma-2-9b-it, on RewardBench, increasing accuracy from 80. 61% to 84. 15%.
3D Gaussian Splatting: Survey, Technologies, Challenges, and Opportunities
3D Gaussian Splatting (3DGS) has emerged as a prominent technique with the potential to become a mainstream method for 3D representations.
Learn to Preserve and Diversify: Parameter-Efficient Group with Orthogonal Regularization for Domain Generalization
To address the above issue, we propose Parameter-Efficient Group with Orthogonal regularization (PEGO) for vision transformers, which effectively preserves the generalization ability of the pre-trained network and learns more diverse knowledge compared with conventional PEFT.
The Devil is in the Statistics: Mitigating and Exploiting Statistics Difference for Generalizable Semi-supervised Medical Image Segmentation
Despite the recent success of domain generalization in medical image segmentation, voxel-wise annotation for all source domains remains a huge burden.
EMPL: A novel Efficient Meta Prompt Learning Framework for Few-shot Unsupervised Domain Adaptation
However, current FS-UDA methods are still suffer from two issues: 1) the data from different domains can not be effectively aligned by few-shot labeled data due to the large domain gaps, 2) it is unstable and time-consuming to generalize to new FS-UDA tasks. To address this issue, we put forward a novel Efficient Meta Prompt Learning Framework for FS-UDA.
InternLM-XComposer-2.5: A Versatile Large Vision Language Model Supporting Long-Contextual Input and Output
This long-context capability allows IXC-2. 5 to excel in tasks requiring extensive input and output contexts.
Ranked #4 on
Video Question Answering
on TVBench
Domain Generalizable Knowledge Tracing via Concept Aggregation and Relation-Based Attention
To address this issue, we propose a domain generalization approach for knowledge tracing, where existing education systems are considered source domains, and new education systems with limited data are considered target domains.
Scalable and Domain-General Abstractive Proposition Segmentation
We model proposition segmentation as a supervised task by training LLMs on existing annotated datasets and show that training yields significantly improved results.
Word Matters: What Influences Domain Adaptation in Summarization?
Our experiments conclude that, when considering dataset learning difficulty, the cross-domain overlap and the performance gain in summarization tasks exhibit an approximate linear relationship, which is not directly related to the number of words.
IR2QSM: Quantitative Susceptibility Mapping via Deep Neural Networks with Iterative Reverse Concatenations and Recurrent Modules
Quantitative susceptibility mapping (QSM) is an MRI phase-based post-processing technique to extract the distribution of tissue susceptibilities, demonstrating significant potential in studying neurological diseases.
How Far Can In-Context Alignment Go? Exploring the State of In-Context Alignment
Recent studies have demonstrated that In-Context Learning (ICL), through the use of specific demonstrations, can align Large Language Models (LLMs) with human preferences known as In-Context Alignment (ICA), indicating that models can comprehend human instructions without requiring parameter adjustments.
CItruS: Chunked Instruction-aware State Eviction for Long Sequence Modeling
Recent research has identified that a large portion of hidden states within the key-value caches of Transformer models can be discarded (also termed evicted) without affecting the perplexity performance in generating long sequences.
Leveraging Locality to Boost Sample Efficiency in Robotic Manipulation
SGRv2 excels in RLBench tasks with keyframe control using merely 5 demonstrations and surpasses the RVT baseline in 23 of 26 tasks.
Zero-Shot Video Editing through Adaptive Sliding Score Distillation
To address these challenges, this study proposes a novel paradigm of video-based score distillation, facilitating direct manipulation of original video content.
Combinatorial Optimization with Automated Graph Neural Networks
The idea of AutoGNP is to use graph neural architecture search algorithms to automatically find the best GNNs for a given NP-hard combinatorial optimization problem.
Analogist: Out-of-the-box Visual In-Context Learning with Image Diffusion Model
For visual prompting, we propose a self-attention cloning (SAC) method to guide the fine-grained structural-level analogy between image examples.
Impact of Preference Noise on the Alignment Performance of Generative Language Models
A key requirement in developing Generative Language Models (GLMs) is to have their values aligned with human values.
Constructing and Exploring Intermediate Domains in Mixed Domain Semi-supervised Medical Image Segmentation
To fully utilize the information within the intermediate domain, we propose a symmetric Guidance training strategy (SymGD), which additionally offers direct guidance to unlabeled data by merging pseudo labels from intermediate samples.
ONNXPruner: ONNX-Based General Model Pruning Adapter
However, the practical application of these algorithms across various models and platforms remains a significant challenge.
InternLM-XComposer2-4KHD: A Pioneering Large Vision-Language Model Handling Resolutions from 336 Pixels to 4K HD
The Large Vision-Language Model (LVLM) field has seen significant advancements, yet its progression has been hindered by challenges in comprehending fine-grained visual content due to limited resolution.
Ranked #47 on
Visual Question Answering
on MM-Vet
Best-of-Venom: Attacking RLHF by Injecting Poisoned Preference Data
Reinforcement Learning from Human Feedback (RLHF) is a popular method for aligning Language Models (LM) with human values and preferences.
Exploiting Inter-sample and Inter-feature Relations in Dataset Distillation
However, these methods face two primary limitations: the dispersed feature distribution within the same class in synthetic datasets, reducing class discrimination, and an exclusive focus on mean feature consistency, lacking precision and comprehensiveness.
InternLM2 Technical Report
The evolution of Large Language Models (LLMs) like ChatGPT and GPT-4 has sparked discussions on the advent of Artificial General Intelligence (AGI).
Ranked #5 on
Long-Context Understanding
on Ada-LEval (BestAnswer)
QSMDiff: Unsupervised 3D Diffusion Models for Quantitative Susceptibility Mapping
In this work, we developed a 3D image patch-based diffusion model, namely QSMDiff, for robust QSM reconstruction across different scan parameters, alongside simultaneous super-resolution and image-denoising tasks.
SETA: Semantic-Aware Token Augmentation for Domain Generalization
In this paper, we study the impact of prior CNN-based augmentation methods on token-based models, revealing their performance is suboptimal due to the lack of incentivizing the model to learn holistic shape information.
Concatenate, Fine-tuning, Re-training: A SAM-enabled Framework for Semi-supervised 3D Medical Image Segmentation
Segment Anything Model (SAM) fine-tuning has shown remarkable performance in medical image segmentation in a fully supervised manner, but requires precise annotations.
Gemini 1.5: Unlocking multimodal understanding across millions of tokens of context
In this report, we introduce the Gemini 1. 5 family of models, representing the next generation of highly compute-efficient multimodal models capable of recalling and reasoning over fine-grained information from millions of tokens of context, including multiple long documents and hours of video and audio.
EfficientZero V2: Mastering Discrete and Continuous Control with Limited Data
We have expanded the performance of EfficientZero to multiple domains, encompassing both continuous and discrete actions, as well as visual and low-dimensional inputs.
Revisiting Disentanglement in Downstream Tasks: A Study on Its Necessity for Abstract Visual Reasoning
This paper further investigates the necessity of disentangled representation in downstream applications.
Can Transformers Capture Spatial Relations between Objects?
Spatial relationships between objects represent key scene information for humans to understand and interact with the world.
Data-freeWeight Compress and Denoise for Large Language Models
Significantly, our method is characterized by without necessitating additional involvement of any corpus, while simultaneously preserving orthogonality in conjunction with pruning and quantization methods.
Distributionally Robust Graph-based Recommendation System
DR-GNN addresses two core challenges: 1) To enable DRO to cater to graph data intertwined with GNN, we reinterpret GNN as a graph smoothing regularizer, thereby facilitating the nuanced application of DRO; 2) Given the typically sparse nature of recommendation data, which might impede robust optimization, we introduce slight perturbations in the training distribution to expand its support.
Angle Robustness Unmanned Aerial Vehicle Navigation in GNSS-Denied Scenarios
Due to the inability to receive signals from the Global Navigation Satellite System (GNSS) in extreme conditions, achieving accurate and robust navigation for Unmanned Aerial Vehicles (UAVs) is a challenging task.
InternLM-XComposer2: Mastering Free-form Text-Image Composition and Comprehension in Vision-Language Large Model
We introduce InternLM-XComposer2, a cutting-edge vision-language model excelling in free-form text-image composition and comprehension.
Ranked #15 on
Visual Question Answering
on MM-Vet v2
General Flow as Foundation Affordance for Scalable Robot Learning
Therefore, we propose to utilize 3D flow, which represents the future trajectories of 3D points on objects of interest, as an ideal prediction target.
Identifying and Analyzing Task-Encoding Tokens in Large Language Models
In-context learning (ICL) has become an effective solution for few-shot learning in natural language processing.
Learning Generalizable Models via Disentangling Spurious and Enhancing Potential Correlations
2) From the feature perspective, the simple Tail Interaction module implicitly enhances potential correlations among all samples from all source domains, facilitating the acquisition of domain-invariant representations across multiple domains for the model.
HOIAnimator: Generating Text-prompt Human-object Animations using Novel Perceptive Diffusion Models
To date the quest to rapidly and effectively produce human-object interaction (HOI) animations directly from textual descriptions stands at the forefront of computer vision research.
PG-LBO: Enhancing High-Dimensional Bayesian Optimization with Pseudo-Label and Gaussian Process Guidance
Based on this technique, we assign appropriate training weights to unlabeled data to enhance the construction of a discriminative latent space.
Any-point Trajectory Modeling for Policy Learning
Learning from demonstration is a powerful method for teaching robots new skills, and having more demonstration data often improves policy learning.
Social-Transmotion: Promptable Human Trajectory Prediction
We translate the idea of a prompt from Natural Language Processing (NLP) to the task of human trajectory prediction, where a prompt can be a sequence of x-y coordinates on the ground, bounding boxes in the image plane, or body pose keypoints in either 2D or 3D.
Heterogeneous Graph Neural Architecture Search with GPT-4
The basic idea of GHGNAS is to design a set of prompts that can guide GPT-4 toward the task of generating new heterogeneous graph neural architectures.
Graph vs. Sequence: An Empirical Study on Knowledge Forms for Knowledge-Grounded Dialogue
Knowledge-grounded dialogue is a task of generating an informative response based on both the dialogue history and external knowledge source.
Digital Life Project: Autonomous 3D Characters with Social Intelligence
In this work, we present Digital Life Project, a framework utilizing language as the universal medium to build autonomous 3D characters, who are capable of engaging in social interactions and expressing with articulated body motions, thereby simulating life in a digital environment.
Ranked #3 on
Motion Synthesis
on InterHuman
Look Before You Leap: Unveiling the Power of GPT-4V in Robotic Vision-Language Planning
In this study, we are interested in imbuing robots with the capability of physically-grounded task planning.
Plug-and-Play Latent Feature Editing for Orientation-Adaptive Quantitative Susceptibility Mapping Neural Networks
The proposed OA-LFE-empowered iQSM, which we refer to as iQSM+, is trained in a self-supervised manner on a specially-designed simulation brain dataset.
PSST: A Benchmark for Evaluation-driven Text Public-Speaking Style Transfer
Language style is necessary for AI systems to understand and generate diverse human language accurately.
Uni-O4: Unifying Online and Offline Deep Reinforcement Learning with Multi-Step On-Policy Optimization
Owning to the alignment of objectives in two phases, the RL agent can transfer between offline and online learning seamlessly.
JRDB-Traj: A Dataset and Benchmark for Trajectory Forecasting in Crowds
To address this, we introduce a novel dataset for end-to-end trajectory forecasting, facilitating the evaluation of models in scenarios involving less-than-ideal preceding modules such as tracking.
The Eval4NLP 2023 Shared Task on Prompting Large Language Models as Explainable Metrics
Specifically, we propose a novel competition setting in which we select a list of allowed LLMs and disallow fine-tuning to ensure a focus on prompting.
MindLLM: Pre-training Lightweight Large Language Model from Scratch, Evaluations and Domain Applications
Large Language Models (LLMs) have demonstrated remarkable performance across various natural language tasks, marking significant strides towards general artificial intelligence.
DexCatch: Learning to Catch Arbitrary Objects with Dexterous Hands
Achieving human-like dexterous manipulation remains a crucial area of research in robotics.
Imitation Learning from Observation with Automatic Discount Scheduling
Nonetheless, we identify that tasks characterized by a progress dependency property pose significant challenges for such approaches; in these tasks, the agent needs to initially learn the expert's preceding behaviors before mastering the subsequent ones.
A New Baseline Assumption of Integated Gradients Based on Shaply value
The selection of appropriate baselines in IG is crucial for crafting meaningful and unbiased explanations of model predictions in diverse settings.
Reinforcement Learning with Foundation Priors: Let the Embodied Agent Efficiently Learn on Its Own
We propose Reinforcement Learning with Foundation Priors (RLFP) to utilize guidance and feedback from policy, value, and success-reward foundation models.
Graph Neural Architecture Search with GPT-4
In this paper, we integrate GPT-4 into GNAS and propose a new GPT-4 based Graph Neural Architecture Search method (GPT4GNAS for short).
CasIL: Cognizing and Imitating Skills via a Dual Cognition-Action Architecture
Heuristically, we extend the usual notion of action to a dual Cognition (high-level)-Action (low-level) architecture by introducing intuitive human cognitive priors, and propose a novel skill IL framework through human-robot interaction, called Cognition-Action-based Skill Imitation Learning (CasIL), for the robotic agent to effectively cognize and imitate the critical skills from raw visual demonstrations.
OpenMSD: Towards Multilingual Scientific Documents Similarity Measurement
We propose the first multilingual scientific documents dataset, Open-access Multilingual Scientific Documents (OpenMSD), which has 74M papers in 103 languages and 778M citation pairs.
Exploring Flat Minima for Domain Generalization with Large Learning Rates
Instead, we observe that leveraging a large learning rate can simultaneously promote weight diversity and facilitate the identification of flat regions in the loss landscape.
A Theoretical Explanation of Activation Sparsity through Flat Minima and Adversarial Robustness
Although having attributed it to training dynamics, existing theoretical explanations of activation sparsity are restricted to shallow networks, small training steps and special training, despite its emergence in deep models standardly trained for a large number of steps.
InsertNeRF: Instilling Generalizability into NeRF with HyperNet Modules
By utilizing multiple plug-and-play HyperNet modules, InsertNeRF dynamically tailors NeRF's weights to specific reference scenes, transforming multi-scale sampling-aware features into scene-specific representations.
IOMatch: Simplifying Open-Set Semi-Supervised Learning with Joint Inliers and Outliers Utilization
Semi-supervised learning (SSL) aims to leverage massive unlabeled data when labels are expensive to obtain.
Efficient Last-iterate Convergence Algorithms in Solving Games
Recent works propose a Reward Transformation (RT) framework for MWU, which removes the uniqueness condition and achieves competitive performance with OMWU.
DomainAdaptor: A Novel Approach to Test-time Adaptation
To deal with the domain shift between training and test samples, current methods have primarily focused on learning generalizable features during training and ignore the specificity of unseen samples that are also critical during the test.
Quantitative Susceptibility Mapping through Model-based Deep Image Prior (MoDIP)
The data-driven approach of supervised learning methods has limited applicability in solving dipole inversion in Quantitative Susceptibility Mapping (QSM) with varying scan parameters across different objects.
Where and How: Mitigating Confusion in Neural Radiance Fields from Sparse Inputs
Neural Radiance Fields from Sparse input} (NeRF-S) have shown great potential in synthesizing novel views with a limited number of observed viewpoints.
3D Medical Image Segmentation with Sparse Annotation via Cross-Teaching between 3D and 2D Networks
Our experimental results on the MMWHS dataset demonstrate that our method outperforms the state-of-the-art (SOTA) semi-supervised segmentation methods.
DNA-Rendering: A Diverse Neural Actor Repository for High-Fidelity Human-centric Rendering
Realistic human-centric rendering plays a key role in both computer vision and computer graphics.
Policy Contrastive Imitation Learning
However, the performance of AIL is still unsatisfactory on the more challenging tasks.
Towards Explainable Evaluation Metrics for Machine Translation
In this context, we also discuss the latest state-of-the-art approaches to explainable metrics based on generative models such as ChatGPT and GPT4.
A Universal Semantic-Geometric Representation for Robotic Manipulation
To this end, we present $\textbf{Semantic-Geometric Representation} (\textbf{SGR})$, a universal perception module for robotics that leverages the rich semantic information of large-scale pre-trained 2D models and inherits the merits of 3D spatial reasoning.
Programmatically Grounded, Compositionally Generalizable Robotic Manipulation
Robots operating in the real world require both rich manipulation skills as well as the ability to semantically reason about when to apply those skills.
Shades of meaning: Uncovering the geometry of ambiguous word representations through contextualised language models
Lexical ambiguity presents a profound and enduring challenge to the language sciences.
For Pre-Trained Vision Models in Motor Control, Not All Policy Learning Methods are Created Equal
Our study yields a series of intriguing results, including the discovery that the effectiveness of pre-training is highly dependent on the choice of the downstream policy learning algorithm.
Seer: Language Instructed Video Prediction with Latent Diffusion Models
Imagining the future trajectory is the key for robots to make sound planning and successfully reach their goals.
Orthogonal Annotation Benefits Barely-supervised Medical Image Segmentation
Subsequently, by introducing unlabeled volumes, we propose a dual-network paradigm named Dense-Sparse Co-training (DeSCO) that exploits dense pseudo labels in early stage and sparse labels in later stage and meanwhile forces consistent output of two networks.
Real-time scheduling of renewable power systems through planning-based reinforcement learning
The well-trained scheduling agent significantly reduces renewable curtailment and load shedding, which are issues arising from traditional scheduling's reliance on inaccurate day-ahead forecasts.
Decision Transformer under Random Frame Dropping
Typical RL methods usually require considerable online interaction data that are costly and unsafe to collect in the real world.
Efficient Exploration Using Extra Safety Budget in Constrained Policy Optimization
In the early stage, our method loosens the practical constraints of unsafe transitions (adding extra safety budget) with the aid of a new metric we propose.
Entity-Agnostic Representation Learning for Parameter-Efficient Knowledge Graph Embedding
In our proposed model, Entity-Agnostic Representation Learning (EARL), we only learn the embeddings for a small set of entities and refer to them as reserved entities.
Few-shot Semantic Segmentation with Support-induced Graph Convolutional Network
However, the appearance variations between objects from the same category could be extremely large, leading to unreliable feature matching and query mask prediction.
Ranked #41 on
Few-Shot Semantic Segmentation
on PASCAL-5i (1-Shot)
A Policy Optimization Method Towards Optimal-time Stability
In current model-free reinforcement learning (RL) algorithms, stability criteria based on sampling methods are commonly utilized to guide policy optimization.
PanoViT: Vision Transformer for Room Layout Estimation from a Single Panoramic Image
In this paper, we propose PanoViT, a panorama vision transformer to estimate the room layout from a single panoramic image.
Pre-Trained Image Encoder for Generalizable Visual Reinforcement Learning
Hence, we propose Pre-trained Image Encoder for Generalizable visual reinforcement learning (PIE-G), a simple yet effective framework that can generalize to the unseen visual scenarios in a zero-shot manner.
A Unified Framework for Contrastive Learning from a Perspective of Affinity Matrix
In this study, we present a new unified contrastive learning representation framework (named UniCLR) suitable for all the above four kinds of methods from a novel perspective of basic affinity matrix.
Affine Transformation Edited and Refined Deep Neural Network for Quantitative Susceptibility Mapping
We propose an end-to-end AFfine Transformation Edited and Refined (AFTER) deep neural network for QSM, which is robust against arbitrary acquisition orientation and spatial resolution up to 0. 6 mm isotropic at the finest.
Spending Thinking Time Wisely: Accelerating MCTS with Virtual Expansions
This paper proposes the Virtual MCTS (V-MCTS), a variant of MCTS that spends more search time on harder states and less search time on simpler states adaptively.
Planning for Sample Efficient Imitation Learning
Inspired by the recent success of EfficientZero in RL, we propose EfficientImitate (EI), a planning-based imitation learning method that can achieve high in-environment sample efficiency and performance simultaneously.
Learning Explicit Credit Assignment for Cooperative Multi-Agent Reinforcement Learning via Polarization Policy Gradient
Empirically, we evaluate MAPPG on the well-known matrix game and differential game, and verify that MAPPG can converge to the global optimum for both discrete and continuous action spaces.
Multi-agent Reinforcement Learning
reinforcement-learning
+3
Modeling Inter-Class and Intra-Class Constraints in Novel Class Discovery
Specifically, we propose an inter-class sKLD constraint to effectively exploit the disjoint relationship between labelled and unlabelled classes, enforcing the separability for different classes in the embedding space.
Predictive Inference with Feature Conformal Prediction
Conformal prediction is a distribution-free technique for establishing valid prediction intervals.
PLN: Parasitic-Like Network for Barely Supervised Medical Image Segmentation
In this paper, by introducing an extremely sparse annotation way of labeling only one slice per 3D image, we investigate a novel barely-supervised segmentation setting with only a few sparsely-labeled images along with a large amount of unlabeled images.
USEEK: Unsupervised SE(3)-Equivariant 3D Keypoints for Generalizable Manipulation
Can a robot manipulate intra-category unseen objects in arbitrary poses with the help of a mere demonstration of grasping pose on a single object instance?
MIXRTs: Toward Interpretable Multi-Agent Reinforcement Learning via Mixing Recurrent Soft Decision Trees
While achieving tremendous success in various fields, existing multi-agent reinforcement learning (MARL) with a black-box neural network architecture makes decisions in an opaque manner that hinders humans from understanding the learned knowledge and how input observations influence decisions.
Multi-agent Reinforcement Learning
reinforcement-learning
+3
Semantic-Aware Fine-Grained Correspondence
Establishing visual correspondence across images is a challenging and essential task.
Resolving Copycat Problems in Visual Imitation Learning via Residual Action Prediction
This is especially the case with image observations, where a single image only includes one view of the scene, and it suffers from a lack of motion information and object occlusions.
EleGANt: Exquisite and Locally Editable GAN for Makeup Transfer
Most existing methods view makeup transfer as transferring color distributions of different facial regions and ignore details such as eye shadows and blushes.
Fighting Fire with Fire: Avoiding DNN Shortcuts through Priming
Across applications spanning supervised classification and sequential control, deep learning has been reported to find "shortcut" solutions that fail catastrophically under minor changes in the data distribution.
Auto-Encoding Adversarial Imitation Learning
In this work, we propose Auto-Encoding Adversarial Imitation Learning (AEAIL), a robust and scalable AIL framework.
An Empirical Study on Disentanglement of Negative-free Contrastive Learning
In this paper, we examine negative-free contrastive learning methods to study the disentanglement property empirically.