Search Results for author:
Found 66 papers, 30 papers with code
Are Pre-trained Transformers Robust in Intent Classification? A Missing Ingredient in Evaluation of Out-of-Scope Intent Detection
Pre-trained Transformer-based models were reported to be robust in intent classification.
h-calibration: Rethinking Classifier Recalibration with Probabilistic Error-Bounded Objective
In this study, we summarize and categorize previous works into three general strategies: intuitively designed methods, binning-based methods, and methods based on formulations of ideal calibration.
LAM SIMULATOR: Advancing Data Generation for Large Action Model Training via Online Exploration and Trajectory Feedback
Large Action Models (LAMs) for AI Agents offer incredible potential but face challenges due to the need for high-quality training data, especially for multi-steps tasks that involve planning, executing tool calls, and responding to feedback.
WorldGenBench: A World-Knowledge-Integrated Benchmark for Reasoning-Driven Text-to-Image Generation
Recent advances in text-to-image (T2I) generation have achieved impressive results, yet existing models still struggle with prompts that require rich world knowledge and implicit reasoning: both of which are critical for producing semantically accurate, coherent, and contextually appropriate images in real-world scenarios.
APIGen-MT: Agentic Pipeline for Multi-Turn Data Generation via Simulated Agent-Human Interplay
We open-source 5K synthetic data trajectories and the trained xLAM-2-fc-r models to advance research in AI agents.
Mitigating Knowledge Discrepancies among Multiple Datasets for Task-agnostic Unified Face Alignment
To explicitly align these mean shapes on an interpretable plane based on their semantics, each shape is then incorporated with a group of semantic alignment embeddings.
ActionStudio: A Lightweight Framework for Data and Training of Large Action Models
However, training large action models remains challenging due to the diversity of agent environments and the complexity of agentic data.
Unsupervised Patch-GAN with Targeted Patch Ranking for Fine-Grained Novelty Detection in Medical Imaging
Detecting novel anomalies in medical imaging is challenging due to the limited availability of labeled data for rare abnormalities, which often display high variability and subtlety.
ReferDINO: Referring Video Object Segmentation with Visual Grounding Foundations
Referring video object segmentation (RVOS) aims to segment target objects throughout a video based on a text description.
Ranked #2 on
Referring Video Object Segmentation
on Long-RVOS
Hengqin-RA-v1: Advanced Large Language Model for Diagnosis and Treatment of Rheumatoid Arthritis with Dataset based Traditional Chinese Medicine
Large language models (LLMs) primarily trained on English texts, often face biases and inaccuracies in Chinese contexts.
Bridging the Data Provenance Gap Across Text, Speech and Video
In this work we conduct the largest and first-of-its-kind longitudinal audit across modalities--popular text, speech, and video datasets--from their detailed sourcing trends and use restrictions to their geographical and linguistic representation.
Continual Learning for Segment Anything Model Adaptation
Driven by the practical need, in this paper, we first propose a novel Continual SAM adaptation (CoSAM) benchmark with 8 different task domains and carefully analyze the limitations of the existing SAM one-step adaptation methods in the continual segmentation scenario.
TACO: Learning Multi-modal Action Models with Synthetic Chains-of-Thought-and-Action
While open-source multi-modal language models perform well on simple question answering tasks, they often fail on complex questions that require multiple capabilities, such as fine-grained recognition, visual grounding, and reasoning, and that demand multi-step solutions.
Ranked #61 on
Visual Question Answering
on MM-Vet
ToolScan: A Benchmark for Characterizing Errors in Tool-Use LLMs
To solve this problem, we introduce TOOLSCAN, a new benchmark to identify error patterns in LLM output on tool-use tasks.
Spatial-Temporal Search for Spiking Neural Networks
Drawing inspiration from the heterogeneity of biological neural networks, we propose a differentiable approach to optimize SNN on both spatial and temporal dimensions.
PRACT: Optimizing Principled Reasoning and Acting of LLM Agent
We introduce the Principled Reasoning and Acting (PRAct) framework, a novel method for learning and enforcing action principles from trajectory data.
SPikE-SSM: A Sparse, Precise, and Efficient Spiking State Space Model for Long Sequences Learning
However, applying the highly capable SSMs to SNNs for long sequences learning poses three major challenges: (1) The membrane potential is determined by the past spiking history of the neuron, leading to reduced efficiency for sequence modeling in parallel computing scenarios.
An Enhanced Federated Prototype Learning Method under Domain Shift
Federated Learning (FL) allows collaborative machine learning training without sharing private data.
Learning Brain Tumor Representation in 3D High-Resolution MR Images via Interpretable State Space Models
Learning meaningful and interpretable representations from high-dimensional volumetric magnetic resonance (MR) images is essential for advancing personalized medicine.
xLAM: A Family of Large Action Models to Empower AI Agent Systems
By releasing the xLAM series, we aim to advance the performance of open-source LLMs for autonomous AI agents, potentially accelerating progress and democratizing access to high-performance models for agent tasks.
SpikingSSMs: Learning Long Sequences with Sparse and Parallel Spiking State Space Models
Known as low energy consumption networks, spiking neural networks (SNNs) have gained a lot of attention within the past decades.
Unsupervised Part Discovery via Dual Representation Alignment
Object parts serve as crucial intermediate representations in various downstream tasks, but part-level representation learning still has not received as much attention as other vision tasks.
Personalized Multi-task Training for Recommender System
In the vast landscape of internet information, recommender systems (RecSys) have become essential for guiding users through a sea of choices aligned with their preferences.
Consent in Crisis: The Rapid Decline of the AI Data Commons
To our knowledge, we conduct the first, large-scale, longitudinal audit of the consent protocols for the web domains underlying AI training corpora.
APIGen: Automated Pipeline for Generating Verifiable and Diverse Function-Calling Datasets
The advancement of function-calling agent models requires diverse, reliable, and high-quality datasets.
Evolutionary Spiking Neural Networks: A Survey
Spiking neural networks (SNNs) are gaining increasing attention as potential computationally efficient alternatives to traditional artificial neural networks(ANNs).
MobileAIBench: Benchmarking LLMs and LMMs for On-Device Use Cases
The deployment of Large Language Models (LLMs) and Large Multimodal Models (LMMs) on mobile devices has gained significant attention due to the benefits of enhanced privacy, stability, and personalization.
Gradient-Guided Modality Decoupling for Missing-Modality Robustness
In aid of this indicator, we present a novel Gradient-guided Modality Decoupling (GMD) method to decouple the dependency on dominating modalities.
AgentLite: A Lightweight Library for Building and Advancing Task-Oriented LLM Agent System
Thus, we open-source a new AI agent library, AgentLite, which simplifies this process by offering a lightweight, user-friendly platform for innovating LLM agent reasoning, architectures, and applications with ease.
AgentOhana: Design Unified Data and Training Pipeline for Effective Agent Learning
It meticulously standardizes and unifies these trajectories into a consistent format, streamlining the creation of a generic data loader optimized for agent training.
Using Left and Right Brains Together: Towards Vision and Language Planning
Large Language Models (LLMs) and Large Multi-modality Models (LMMs) have demonstrated remarkable decision masking capabilities on a variety of tasks.
Deep Learning for Code Intelligence: Survey, Benchmark and Toolkit
We also benchmark several state-of-the-art neural models for code intelligence, and provide an open-source toolkit tailored for the rapid prototyping of deep-learning-based code intelligence models.
Video Understanding with Large Language Models: A Survey
With the burgeoning growth of online video platforms and the escalating volume of video content, the demand for proficient video understanding tools has intensified markedly.
DRDT: Dynamic Reflection with Divergent Thinking for LLM-based Sequential Recommendation
With our principle, we managed to outperform GPT-Turbo-3. 5 on three datasets using 7b models e. g., Vicuna-7b and Openchat-7b on NDCG@10.
Semi-supervised Semantic Segmentation via Boosting Uncertainty on Unlabeled Data
We first figure out that the distribution gap between labeled and unlabeled datasets cannot be ignored, even though the two datasets are sampled from the same distribution.
Enhancing Performance on Seen and Unseen Dialogue Scenarios using Retrieval-Augmented End-to-End Task-Oriented System
End-to-end task-oriented dialogue (TOD) systems have achieved promising performance by leveraging sophisticated natural language understanding and natural language generation capabilities of pre-trained models.
BOLAA: Benchmarking and Orchestrating LLM-augmented Autonomous Agents
The massive successes of large language models (LLMs) encourage the emerging exploration of LLM-augmented Autonomous Agents (LAAs).
Retroformer: Retrospective Large Language Agents with Policy Gradient Optimization
This demonstrates that using policy gradient optimization to improve language agents, for which we believe our work is one of the first, seems promising and can be applied to optimize other models in the agent architecture to enhance agent performances over time.
Cross Contrasting Feature Perturbation for Domain Generalization
Domain generalization (DG) aims to learn a robust model from source domains that generalize well on unseen target domains.
Strip-MLP: Efficient Token Interaction for Vision MLP
Finally, based on the Strip MLP layer, we propose a novel \textbf{L}ocal \textbf{S}trip \textbf{M}ixing \textbf{M}odule (LSMM) to boost the token interaction power in the local region.
DialogStudio: Towards Richest and Most Diverse Unified Dataset Collection for Conversational AI
Despite advancements in conversational AI, language models encounter challenges to handle diverse conversational tasks, and existing dialogue dataset collections often lack diversity and comprehensiveness.
Class Attention to Regions of Lesion for Imbalanced Medical Image Recognition
The CARE framework needs bounding boxes to represent the lesion regions of rare diseases.
DNAGPT: A Generalized Pre-trained Tool for Versatile DNA Sequence Analysis Tasks
Pre-trained large language models demonstrate potential in extracting information from DNA sequences, yet adapting to a variety of tasks and data modalities remains a challenge.
Inter-Rater Uncertainty Quantification in Medical Image Segmentation via Rater-Specific Bayesian Neural Networks
We introduce a novel Bayesian neural network-based architecture to estimate inter-rater uncertainty in medical image segmentation.
When SAM Meets Sonar Images
Notably, there is a lack of research on the application of SAM to sonar imaging.
Efficient Deep Spiking Multi-Layer Perceptrons with Multiplication-Free Inference
As a result, we establish an efficient multi-stage spiking MLP network that blends effectively global receptive fields with local feature extraction for comprehensive spike-based computation.
Zero-shot Item-based Recommendation via Multi-task Product Knowledge Graph Pre-Training
This paper presents a novel paradigm for the Zero-Shot Item-based Recommendation (ZSIR) task, which pre-trains a model on product knowledge graph (PKG) to refine the item features from PLMs.
Feature Alignment and Uniformity for Test Time Adaptation
Test time adaptation (TTA) aims to adapt deep neural networks when receiving out of distribution test domain samples.
Bootstrap The Original Latent: Learning a Private Model from a Black-box Model
In this paper, considering the balance of data/model privacy of model owners and user needs, we propose a new setting called Back-Propagated Black-Box Adaptation (BPBA) for users to better train their private models via the guidance of the back-propagated results of a Black-box foundation/source model.
Fantastic Rewards and How to Tame Them: A Case Study on Reward Learning for Task-oriented Dialogue Systems
Prior works mainly focus on adopting advanced RL techniques to train the ToD agents, while the design of the reward function is not well studied.
Aggregation of Disentanglement: Reconsidering Domain Variations in Domain Generalization
Different from learning domain invariant features from source domains, we decouple the input images into Domain Expert Features and noise.
AugTriever: Unsupervised Dense Retrieval and Domain Adaptation by Scalable Data Augmentation
In this study, we aim to develop unsupervised methods for improving dense retrieval models.
A Domain-specific Perceptual Metric via Contrastive Self-supervised Representation: Applications on Natural and Medical Images
We find that in the natural image domain, CSR behaves on par with the supervised one on several perceptual tests as a metric, and in the medical domain, CSR better quantifies perceptual similarity concerning the experts' ratings.
Rethinking Alignment and Uniformity in Unsupervised Semantic Segmentation
Experimental results on multiple semantic segmentation benchmarks show that our unsupervised segmentation framework specializes in catching semantic representations, which outperforms all the unpretrained and even several pretrained methods.
Ranked #1 on
Unsupervised Semantic Segmentation
on COCO-Stuff-3
Partial Least Square Regression via Three-factor SVD-type Manifold Optimization for EEG Decoding
To this end, we propose a new method to solve the partial least square regression, named PLSR via optimization on bi-Grassmann manifold (PLSRbiGr).
Domain-Adaptive 3D Medical Image Synthesis: An Efficient Unsupervised Approach
This work focuses on exploring domain adaptation (DA) of 3D image-to-image synthesis models.
Sparse Local Patch Transformer for Robust Face Alignment and Landmarks Inherent Relation Learning
The SLPT generates the representation of each single landmark from a local patch and aggregates them by an adaptive inherent relation based on the attention mechanism.
Ranked #2 on
Face Alignment
on COFW-68
Discrete Time Convolution for Fast Event-Based Stereo
Inspired by continuous dynamics of biological neuron models, we propose a novel encoding method for sparse events - continuous time convolution (CTC) - which learns to model the spatial feature of the data with intrinsic dynamics.
Detect Faces Efficiently: A Survey and Evaluations
However, with the tremendous increase in images and videos with variations in face scale, appearance, expression, occlusion and pose, traditional face detectors are challenged to detect various "in the wild" faces.
Few-Shot Intent Detection via Contrastive Pre-Training and Fine-Tuning
In this work, we focus on a more challenging few-shot intent detection scenario where many intents are fine-grained and semantically similar.
Are Pretrained Transformers Robust in Intent Classification? A Missing Ingredient in Evaluation of Out-of-Scope Intent Detection
Pre-trained Transformer-based models were reported to be robust in intent classification.
Enriching Non-Autoregressive Transformer with Syntactic and Semantic Structures for Neural Machine Translation
In this paper, we claim that the syntactic and semantic structures among natural language are critical for non-autoregressive machine translation and can further improve the performance.
Imbalance-Aware Self-Supervised Learning for 3D Radiomic Representations
Radiomic representations can quantify properties of regions of interest in medical image data.
Deep Class-Specific Affinity-Guided Convolutional Network for Multimodal Unpaired Image Segmentation
Our affinity matrix does not depend on spatial alignments of the visual features and thus allows us to train with unpaired, multimodal inputs.
Sign-Agnostic Implicit Learning of Surface Self-Similarities for Shape Modeling and Reconstruction from Raw Point Clouds
Motivated from a universal phenomenon that self-similar shape patterns of local surface patches repeat across the entire surface of an object, we aim to push forward the data-driven strategies and propose to learn a local implicit surface network for a shared, adaptive modeling of the entire surface for a direct surface reconstruction from raw point cloud; we also enhance the leveraging of surface self-similarities by improving correlations among the optimized latent codes of individual surface patches.
Deep CNNs for HEp-2 Cells Classification : A Cross-specimen Analysis
Automatic classification of Human Epithelial Type-2 (HEp-2) cells staining patterns is an important and yet a challenging problem.
