Search Results for author:
Found 48 papers, 17 papers with code
Phi-3 Technical Report: A Highly Capable Language Model Locally on Your Phone
We introduce phi-3-mini, a 3. 8 billion parameter language model trained on 3. 3 trillion tokens, whose overall performance, as measured by both academic benchmarks and internal testing, rivals that of models such as Mixtral 8x7B and GPT-3. 5 (e. g., phi-3-mini achieves 69% on MMLU and 8. 38 on MT-bench), despite being small enough to be deployed on a phone.
3DGSR: Implicit Surface Reconstruction with 3D Gaussian Splatting
In this paper, we present an implicit surface reconstruction method with 3D Gaussian Splatting (3DGS), namely 3DGSR, that allows for accurate 3D reconstruction with intricate details while inheriting the high efficiency and rendering quality of 3DGS.
Training Small Multimodal Models to Bridge Biomedical Competency Gap: A Case Study in Radiology Imaging
Frontier models such as GPT-4V still have major competency gaps in multimodal capabilities for biomedical applications.
FuseChat: Knowledge Fusion of Chat Models
Recently, \textsc{FuseLLM} introduced the concept of knowledge fusion to transfer the collective knowledge of multiple structurally varied LLMs into a target LLM through lightweight continual training.
Spec-Gaussian: Anisotropic View-Dependent Appearance for 3D Gaussian Splatting
The recent advancements in 3D Gaussian splatting (3D-GS) have not only facilitated real-time rendering through modern GPU rasterization pipelines but have also attained state-of-the-art rendering quality.
SC-GS: Sparse-Controlled Gaussian Splatting for Editable Dynamic Scenes
During learning, the location and number of control points are adaptively adjusted to accommodate varying motion complexities in different regions, and an ARAP loss following the principle of as rigid as possible is developed to enforce spatial continuity and local rigidity of learned motions.
CoDi-2: In-Context, Interleaved, and Interactive Any-to-Any Generation
We present CoDi-2, a versatile and interactive Multimodal Large Language Model (MLLM) that can follow complex multimodal interleaved instructions, conduct in-context learning (ICL), reason, chat, edit, etc., in an any-to-any input-output modality paradigm.
SIRe-IR: Inverse Rendering for BRDF Reconstruction with Shadow and Illumination Removal in High-Illuminance Scenes
Implicit neural representation has opened up new possibilities for inverse rendering.
Soft Convex Quantization: Revisiting Vector Quantization with Convex Optimization
SCQ works like a differentiable convex optimization (DCO) layer: in the forward pass, we solve for the optimal convex combination of codebook vectors that quantize the inputs.
Deformable 3D Gaussians for High-Fidelity Monocular Dynamic Scene Reconstruction
Implicit neural representation has paved the way for new approaches to dynamic scene reconstruction and rendering.
Plug in the Safety Chip: Enforcing Constraints for LLM-driven Robot Agents
Recent advancements in large language models (LLMs) have enabled a new research domain, LLM agents, for solving robotics and planning tasks by leveraging the world knowledge and general reasoning abilities of LLMs obtained during pretraining.
A General Implicit Framework for Fast NeRF Composition and Rendering
Mainly, our work introduces a new surface representation known as Neural Depth Fields (NeDF) that quickly determines the spatial relationship between objects by allowing direct intersection computation between rays and implicit surfaces.
Multi-task Bioassay Pre-training for Protein-ligand Binding Affinity Prediction
By introducing multi-task pre-training to treat the prediction of different affinity labels as different tasks and classifying relative rankings between samples from the same bioassay, MBP learns robust and transferrable structural knowledge from our new ChEMBL-Dock dataset with varied and noisy labels.
i-Code Studio: A Configurable and Composable Framework for Integrative AI
Artificial General Intelligence (AGI) requires comprehensive understanding and generation capabilities for a variety of tasks spanning different modalities and functionalities.
i-Code V2: An Autoregressive Generation Framework over Vision, Language, and Speech Data
The convergence of text, visual, and audio data is a key step towards human-like artificial intelligence, however the current Vision-Language-Speech landscape is dominated by encoder-only models which lack generative abilities.
Any-to-Any Generation via Composable Diffusion
We present Composable Diffusion (CoDi), a novel generative model capable of generating any combination of output modalities, such as language, image, video, or audio, from any combination of input modalities.
Ranked #7 on
Audio Generation
on AudioCaps
Real-Time Audio-Visual End-to-End Speech Enhancement
In this paper, we propose a low-latency real-time audio-visual end-to-end enhancement (AV-E3Net) model based on the recently proposed end-to-end enhancement network (E3Net).
Grounding Complex Natural Language Commands for Temporal Tasks in Unseen Environments
We propose Lang2LTL, a modular system and a software package that leverages large language models (LLMs) to ground temporal navigational commands to LTL specifications in environments without prior language data.
APOLLO: A Simple Approach for Adaptive Pretraining of Language Models for Logical Reasoning
Logical reasoning of text is an important ability that requires understanding the information present in the text, their interconnections, and then reasoning through them to infer new conclusions.
Unifying Vision, Text, and Layout for Universal Document Processing
UDOP leverages the spatial correlation between textual content and document image to model image, text, and layout modalities with one uniform representation.
Ranked #5 on
Visual Question Answering (VQA)
on InfographicVQA
(using extra training data)
UniSumm and SummZoo: Unified Model and Diverse Benchmark for Few-Shot Summarization
The high annotation costs and diverse demands of various summarization tasks motivate the development of few-shot summarization.
Empowering Language Models with Knowledge Graph Reasoning for Question Answering
Answering open-domain questions requires world knowledge about in-context entities.
MACSum: Controllable Summarization with Mixed Attributes
We propose two simple and effective parameter-efficient approaches for the new task of mixed controllable summarization based on hard prompt tuning and soft prefix tuning.
Global Contrastive Batch Sampling via Optimization on Sample Permutations
Contrastive Learning has recently achieved state-of-the-art performance in a wide range of tasks.
Language Models with Image Descriptors are Strong Few-Shot Video-Language Learners
The goal of this work is to build flexible video-language models that can generalize to various video-to-text tasks from few examples, such as domain-specific captioning, question answering, and future event prediction.
ODBO: Bayesian Optimization with Search Space Prescreening for Directed Protein Evolution
Directed evolution is a versatile technique in protein engineering that mimics the process of natural selection by iteratively alternating between mutagenesis and screening in order to search for sequences that optimize a given property of interest, such as catalytic activity and binding affinity to a specified target.
Automatic Rule Induction for Interpretable Semi-Supervised Learning
Semi-supervised learning has shown promise in allowing NLP models to generalize from small amounts of labeled data.
i-Code: An Integrative and Composable Multimodal Learning Framework
Human intelligence is multimodal; we integrate visual, linguistic, and acoustic signals to maintain a holistic worldview.
GASCN: Graph Attention Shape Completion Network
Shape completion, the problem of inferring the complete geometry of an object given a partial point cloud, is an important problem in robotics and computer vision.
SPLDExtraTrees: Robust machine learning approach for predicting kinase inhibitor resistance
Drug resistance is a major threat to the global health and a significant concern throughout the clinical treatment of diseases and drug development.
Natural Language for Human-Robot Collaboration: Problems Beyond Language Grounding
To enable robots to instruct humans in collaborations, we identify several aspects of language processing that are not commonly studied in this context.
A Simple and Effective Method To Eliminate the Self Language Bias in Multilingual Representations
A simple but highly effective method "Language Information Removal (LIR)" factors out language identity information from semantic related components in multilingual representations pre-trained on multi-monolingual data.
Universal Sentence Representations Learning with Conditional Masked Language Model
This paper presents a novel training method, Conditional Masked Language Modeling (CMLM), to effectively learn sentence representations on large scale unlabeled corpora.
Universal Sentence Representation Learning with Conditional Masked Language Model
This paper presents a novel training method, Conditional Masked Language Modeling (CMLM), to effectively learn sentence representations on large scale unlabeled corpora.
Multi-Constitutive Neural Network for Large Deformation Poromechanics Problem
We propose a novel method "multi-constitutive neural network" (MCNN) such that one model can solve several different constitutive laws.
Select-ProtoNet: Learning to Select for Few-Shot Disease Subtype Prediction
Current machine learning has made great progress on computer vision and many other fields attributed to the large amount of high-quality training samples, while it does not work very well on genomic data analysis, since they are notoriously known as small data.
Filtered Inner Product Projection for Crosslingual Embedding Alignment
Due to widespread interest in machine translation and transfer learning, there are numerous algorithms for mapping multiple embeddings to a shared representation space.
Anomaly Detection with Domain Adaptation
We study the problem of semi-supervised anomaly detection with domain adaptation.
Memory Augmented Generative Adversarial Networks for Anomaly Detection
Decoded memory units in MEMGAN are more interpretable and disentangled than previous methods, which further demonstrates the effectiveness of the memory mechanism.
Regularized Cycle Consistent Generative Adversarial Network for Anomaly Detection
In this paper, we investigate algorithms for anomaly detection.
TED: A Pretrained Unsupervised Summarization Model with Theme Modeling and Denoising
Text summarization aims to extract essential information from a piece of text and transform the text into a concise version.
Leveraging Lead Bias for Zero-shot Abstractive News Summarization
A typical journalistic convention in news articles is to deliver the most salient information in the beginning, also known as the lead bias.
Make Lead Bias in Your Favor: A Simple and Effective Method for News Summarization
For example, the pretrained model without finetuning outperforms pointer-generator network on CNN/DailyMail dataset.
Embedding Imputation with Grounded Language Information
In this paper, we propose an approach for embedding imputation which uses grounded information in the form of a knowledge graph.
Out-of-Vocabulary Embedding Imputation with Grounded Language Information by Graph Convolutional Networks
In this paper, we propose an approach for embedding imputation which uses grounded information in the form of a knowledge graph.
Zero-training Sentence Embedding via Orthogonal Basis
We model the semantic meaning of a word in a sentence based on two aspects.
Parameter-free Sentence Embedding via Orthogonal Basis
Inspired by the Gram-Schmidt Process in geometric theory, we build an orthogonal basis of the subspace spanned by a word and its surrounding context in a sentence.
Language Distribution Prediction based on Batch Markov Monte Carlo Simulation with Migration
To model the randomness of language spreading, we propose the Batch Markov Monte Carlo Simulation with Migration(BMMCSM) algorithm, in which each agent is treated as a language stack.
