Search Results for author:
Found 68 papers, 26 papers with code
UniGeo: Taming Video Diffusion for Unified Consistent Geometry Estimation
Recently, methods leveraging diffusion model priors to assist monocular geometric estimation (e. g., depth and normal) have gained significant attention due to their strong generalization ability.
QwenLong-L1: Towards Long-Context Large Reasoning Models with Reinforcement Learning
To bridge this gap, we first formalize the paradigm of long-context reasoning RL, and identify key challenges in suboptimal training efficiency and unstable optimization process.
ThinkSwitcher: When to Think Hard, When to Think Fast
To leverage this capability, we propose ThinkSwitcher, a framework that enables a single LRM to dynamically switch between short and long CoT modes based on task complexity.
Mutual-Taught for Co-adapting Policy and Reward Models
During the preference optimization of large language models (LLMs), distribution shifts may arise between newly generated model samples and the data used to train the reward model (RM).
FuseRL: Dense Preference Optimization for Heterogeneous Model Fusion
Heterogeneous model fusion enhances the performance of LLMs by integrating the knowledge and capabilities of multiple structurally diverse models.
Scaling Laws of Synthetic Data for Language Models
Key findings from our extensive mathematical experiments on SynthLLM include: (1) SynthLLM generates synthetic data that reliably adheres to the rectified scaling law across various model sizes; (2) Performance improvements plateau near 300B tokens; and (3) Larger models approach optimal performance with fewer training tokens.
FuseChat-3.0: Preference Optimization Meets Heterogeneous Model Fusion
The FuseChat-3. 0 training pipeline consists of two key stages: (1) supervised fine-tuning (SFT) to align the target and source model distributions, and (2) Direct Preference Optimization (DPO) to apply preferences from multiple source LLMs to fine-tune the target model.
Segmenting Text and Learning Their Rewards for Improved RLHF in Language Model
Reinforcement learning from human feedback (RLHF) has been widely adopted to align language models (LMs) with human preference.
Frequency-aware Event Cloud Network
Event cameras are biologically inspired sensors that emit events asynchronously with remarkable temporal resolution, garnering significant attention from both industry and academia.
Deformable Radial Kernel Splatting
Recently, Gaussian splatting has emerged as a robust technique for representing 3D scenes, enabling real-time rasterization and high-fidelity rendering.
Weighted-Reward Preference Optimization for Implicit Model Fusion
To address distributional deviations between the source and target LLMs, WRPO introduces a progressive adaptation strategy that gradually shifts reliance on preferred examples from the target LLM to the source LLMs.
A Comprehensive Evaluation of Large Language Models on Aspect-Based Sentiment Analysis
For the fine-tuning-dependent paradigm, we efficiently fine-tune LLMs using instruction-based multi-task learning.
OASIS: Open Agent Social Interaction Simulations with One Million Agents
There has been a growing interest in enhancing rule-based agent-based models (ABMs) for social media platforms (i. e., X, Reddit) with more realistic large language model (LLM) agents, thereby allowing for a more nuanced study of complex systems.
Towards Realistic Example-based Modeling via 3D Gaussian Stitching
To this end, we present an example-based modeling method that combines multiple Gaussian fields in a point-based representation using sample-guided synthesis.
See What LLMs Cannot Answer: A Self-Challenge Framework for Uncovering LLM Weaknesses
We then build a benchmark, SC-G4, consisting of 1, 835 instances generated by GPT-4 using these patterns, with human-annotated gold responses.
FuseChat: Knowledge Fusion of Chat Models
In this work, we propose a new framework for the knowledge fusion of chat LLMs through two main stages, resulting in FuseChat.
Phi-3 Safety Post-Training: Aligning Language Models with a "Break-Fix" Cycle
Recent innovations in language model training have demonstrated that it is possible to create highly performant models that are small enough to run on a smartphone.
Cost-Effective Proxy Reward Model Construction with On-Policy and Active Learning
Reinforcement learning with human feedback (RLHF), as a widely adopted approach in current large language model pipelines, is \textit{bottlenecked by the size of human preference data}.
Self-Exploring Language Models: Active Preference Elicitation for Online Alignment
Preference optimization, particularly through Reinforcement Learning from Human Feedback (RLHF), has achieved significant success in aligning Large Language Models (LLMs) to adhere to human intentions.
Phi-3 Technical Report: A Highly Capable Language Model Locally on Your Phone
We introduce phi-3-mini, a 3. 8 billion parameter language model trained on 3. 3 trillion tokens, whose overall performance, as measured by both academic benchmarks and internal testing, rivals that of models such as Mixtral 8x7B and GPT-3. 5 (e. g., phi-3-mini achieves 69% on MMLU and 8. 38 on MT-bench), despite being small enough to be deployed on a phone.
Ranked #5 on
MMR total
on MRR-Benchmark
(using extra training data)
3DGSR: Implicit Surface Reconstruction with 3D Gaussian Splatting
In this paper, we present an implicit surface reconstruction method with 3D Gaussian Splatting (3DGS), namely 3DGSR, that allows for accurate 3D reconstruction with intricate details while inheriting the high efficiency and rendering quality of 3DGS.
Towards a clinically accessible radiology foundation model: open-access and lightweight, with automated evaluation
Frontier general-domain models such as GPT-4V still have significant performance gaps in multimodal biomedical applications.
Knowledge Fusion of Chat LLMs: A Preliminary Technical Report
Recently, FuseLLM introduced the concept of knowledge fusion to transfer the collective knowledge of multiple structurally varied LLMs into a target LLM through lightweight continual training.
Spec-Gaussian: Anisotropic View-Dependent Appearance for 3D Gaussian Splatting
Thanks to ASG, we have significantly improved the ability of 3D-GS to model scenes with specular and anisotropic components without increasing the number of 3D Gaussians.
CoDi-2: In-Context Interleaved and Interactive Any-to-Any Generation
We present CoDi-2 a Multimodal Large Language Model (MLLM) for learning in-context interleaved multimodal representations.
SC-GS: Sparse-Controlled Gaussian Splatting for Editable Dynamic Scenes
During learning, the location and number of control points are adaptively adjusted to accommodate varying motion complexities in different regions, and an ARAP loss following the principle of as rigid as possible is developed to enforce spatial continuity and local rigidity of learned motions.
CoDi-2: In-Context, Interleaved, and Interactive Any-to-Any Generation
We present CoDi-2, a versatile and interactive Multimodal Large Language Model (MLLM) that can follow complex multimodal interleaved instructions, conduct in-context learning (ICL), reason, chat, edit, etc., in an any-to-any input-output modality paradigm.
SIRe-IR: Inverse Rendering for BRDF Reconstruction with Shadow and Illumination Removal in High-Illuminance Scenes
Implicit neural representation has opened up new possibilities for inverse rendering.
Soft Convex Quantization: Revisiting Vector Quantization with Convex Optimization
SCQ works like a differentiable convex optimization (DCO) layer: in the forward pass, we solve for the optimal convex combination of codebook vectors that quantize the inputs.
Deformable 3D Gaussians for High-Fidelity Monocular Dynamic Scene Reconstruction
Implicit neural representation has paved the way for new approaches to dynamic scene reconstruction and rendering.
Plug in the Safety Chip: Enforcing Constraints for LLM-driven Robot Agents
Recent advancements in large language models (LLMs) have enabled a new research domain, LLM agents, for solving robotics and planning tasks by leveraging the world knowledge and general reasoning abilities of LLMs obtained during pretraining.
A General Implicit Framework for Fast NeRF Composition and Rendering
Mainly, our work introduces a new surface representation known as Neural Depth Fields (NeDF) that quickly determines the spatial relationship between objects by allowing direct intersection computation between rays and implicit surfaces.
Multi-task Bioassay Pre-training for Protein-ligand Binding Affinity Prediction
By introducing multi-task pre-training to treat the prediction of different affinity labels as different tasks and classifying relative rankings between samples from the same bioassay, MBP learns robust and transferrable structural knowledge from our new ChEMBL-Dock dataset with varied and noisy labels.
i-Code Studio: A Configurable and Composable Framework for Integrative AI
Artificial General Intelligence (AGI) requires comprehensive understanding and generation capabilities for a variety of tasks spanning different modalities and functionalities.
i-Code V2: An Autoregressive Generation Framework over Vision, Language, and Speech Data
The convergence of text, visual, and audio data is a key step towards human-like artificial intelligence, however the current Vision-Language-Speech landscape is dominated by encoder-only models which lack generative abilities.
Any-to-Any Generation via Composable Diffusion
We present Composable Diffusion (CoDi), a novel generative model capable of generating any combination of output modalities, such as language, image, video, or audio, from any combination of input modalities.
Ranked #8 on
Audio Generation
on AudioCaps
(FAD metric)
Real-Time Audio-Visual End-to-End Speech Enhancement
In this paper, we propose a low-latency real-time audio-visual end-to-end enhancement (AV-E3Net) model based on the recently proposed end-to-end enhancement network (E3Net).
Grounding Complex Natural Language Commands for Temporal Tasks in Unseen Environments
We propose Lang2LTL, a modular system and a software package that leverages large language models (LLMs) to ground temporal navigational commands to LTL specifications in environments without prior language data.
APOLLO: A Simple Approach for Adaptive Pretraining of Language Models for Logical Reasoning
Logical reasoning of text is an important ability that requires understanding the information present in the text, their interconnections, and then reasoning through them to infer new conclusions.
Unifying Vision, Text, and Layout for Universal Document Processing
UDOP leverages the spatial correlation between textual content and document image to model image, text, and layout modalities with one uniform representation.
Ranked #5 on
Visual Question Answering (VQA)
on InfographicVQA
(using extra training data)
UniSumm and SummZoo: Unified Model and Diverse Benchmark for Few-Shot Summarization
The high annotation costs and diverse demands of various summarization tasks motivate the development of few-shot summarization.
Empowering Language Models with Knowledge Graph Reasoning for Question Answering
Answering open-domain questions requires world knowledge about in-context entities.
Global Contrastive Batch Sampling via Optimization on Sample Permutations
Contrastive Learning has recently achieved state-of-the-art performance in a wide range of tasks.
Language Models with Image Descriptors are Strong Few-Shot Video-Language Learners
The goal of this work is to build flexible video-language models that can generalize to various video-to-text tasks from few examples, such as domain-specific captioning, question answering, and future event prediction.
ODBO: Bayesian Optimization with Search Space Prescreening for Directed Protein Evolution
Directed evolution is a versatile technique in protein engineering that mimics the process of natural selection by iteratively alternating between mutagenesis and screening in order to search for sequences that optimize a given property of interest, such as catalytic activity and binding affinity to a specified target.
Automatic Rule Induction for Interpretable Semi-Supervised Learning
Semi-supervised learning has shown promise in allowing NLP models to generalize from small amounts of labeled data.
i-Code: An Integrative and Composable Multimodal Learning Framework
Human intelligence is multimodal; we integrate visual, linguistic, and acoustic signals to maintain a holistic worldview.
GASCN: Graph Attention Shape Completion Network
Shape completion, the problem of inferring the complete geometry of an object given a partial point cloud, is an important problem in robotics and computer vision.
SPLDExtraTrees: Robust machine learning approach for predicting kinase inhibitor resistance
Drug resistance is a major threat to the global health and a significant concern throughout the clinical treatment of diseases and drug development.
Natural Language for Human-Robot Collaboration: Problems Beyond Language Grounding
To enable robots to instruct humans in collaborations, we identify several aspects of language processing that are not commonly studied in this context.
A Simple and Effective Method To Eliminate the Self Language Bias in Multilingual Representations
A simple but highly effective method "Language Information Removal (LIR)" factors out language identity information from semantic related components in multilingual representations pre-trained on multi-monolingual data.
Universal Sentence Representations Learning with Conditional Masked Language Model
This paper presents a novel training method, Conditional Masked Language Modeling (CMLM), to effectively learn sentence representations on large scale unlabeled corpora.
Universal Sentence Representation Learning with Conditional Masked Language Model
This paper presents a novel training method, Conditional Masked Language Modeling (CMLM), to effectively learn sentence representations on large scale unlabeled corpora.
Multi-Constitutive Neural Network for Large Deformation Poromechanics Problem
We propose a novel method "multi-constitutive neural network" (MCNN) such that one model can solve several different constitutive laws.
Select-ProtoNet: Learning to Select for Few-Shot Disease Subtype Prediction
Current machine learning has made great progress on computer vision and many other fields attributed to the large amount of high-quality training samples, while it does not work very well on genomic data analysis, since they are notoriously known as small data.
Filtered Inner Product Projection for Crosslingual Embedding Alignment
Due to widespread interest in machine translation and transfer learning, there are numerous algorithms for mapping multiple embeddings to a shared representation space.
Anomaly Detection with Domain Adaptation
We study the problem of semi-supervised anomaly detection with domain adaptation.
Memory Augmented Generative Adversarial Networks for Anomaly Detection
Decoded memory units in MEMGAN are more interpretable and disentangled than previous methods, which further demonstrates the effectiveness of the memory mechanism.
Regularized Cycle Consistent Generative Adversarial Network for Anomaly Detection
In this paper, we investigate algorithms for anomaly detection.
TED: A Pretrained Unsupervised Summarization Model with Theme Modeling and Denoising
Text summarization aims to extract essential information from a piece of text and transform the text into a concise version.
Leveraging Lead Bias for Zero-shot Abstractive News Summarization
A typical journalistic convention in news articles is to deliver the most salient information in the beginning, also known as the lead bias.
Make Lead Bias in Your Favor: A Simple and Effective Method for News Summarization
For example, the pretrained model without finetuning outperforms pointer-generator network on CNN/DailyMail dataset.
Embedding Imputation with Grounded Language Information
In this paper, we propose an approach for embedding imputation which uses grounded information in the form of a knowledge graph.
Out-of-Vocabulary Embedding Imputation with Grounded Language Information by Graph Convolutional Networks
In this paper, we propose an approach for embedding imputation which uses grounded information in the form of a knowledge graph.
Zero-training Sentence Embedding via Orthogonal Basis
We model the semantic meaning of a word in a sentence based on two aspects.
Parameter-free Sentence Embedding via Orthogonal Basis
Inspired by the Gram-Schmidt Process in geometric theory, we build an orthogonal basis of the subspace spanned by a word and its surrounding context in a sentence.
Language Distribution Prediction based on Batch Markov Monte Carlo Simulation with Migration
To model the randomness of language spreading, we propose the Batch Markov Monte Carlo Simulation with Migration(BMMCSM) algorithm, in which each agent is treated as a language stack.
