Search Results for author:
Found 31 papers, 14 papers with code
Segmenting Text and Learning Their Rewards for Improved RLHF in Language Model
Reinforcement learning from human feedback (RLHF) has been widely adopted to align language models (LMs) with human preference.
Developing a Reliable, Fast, General-Purpose Hallucination Detection and Mitigation Service
Hallucination, a phenomenon where large language models (LLMs) produce output that is factually incorrect or unrelated to the input, is a major challenge for LLM applications that require accuracy and dependability.
Diffusion-RPO: Aligning Diffusion Models through Relative Preference Optimization
Aligning large language models with human preferences has emerged as a critical focus in language modeling research.
Self-Augmented Preference Optimization: Off-Policy Paradigms for Language Model Alignment
Traditional language model alignment methods, such as Direct Preference Optimization (DPO), are limited by their dependence on static, pre-collected paired preference data, which hampers their adaptability and practical applicability.
Towards a clinically accessible radiology foundation model: open-access and lightweight, with automated evaluation
Frontier general-domain models such as GPT-4V still have significant performance gaps in multimodal biomedical applications.
Video-Bench: A Comprehensive Benchmark and Toolkit for Evaluating Video-based Large Language Models
Video-based large language models (Video-LLMs) have been recently introduced, targeting both fundamental improvements in perception and comprehension, and a diverse range of user inquiries.
DoLa: Decoding by Contrasting Layers Improves Factuality in Large Language Models
Despite their impressive capabilities, large language models (LLMs) are prone to hallucinations, i. e., generating content that deviates from facts seen during pretraining.
Predictive Sparse Manifold Transform
PSMT incorporates two layers where the first sparse coding layer represents the input sequence as sparse coefficients over an overcomplete dictionary and the second manifold learning layer learns a geometric embedding space that captures topological similarity and dynamic temporal linearity in sparse coefficients.
Interactive Editing for Text Summarization
Summarizing lengthy documents is a common and essential task in our daily lives.
Album Storytelling with Iterative Story-aware Captioning and Large Language Models
One natural approach is to use caption models to describe each photo in the album, and then use LLMs to summarize and rewrite the generated captions into an engaging story.
Summarization with Precise Length Control
Many applications of text generation such as summarization benefit from accurately controlling the text length.
Personalized Abstractive Summarization by Tri-agent Generation Pipeline
The inference-only large language model (ChatGPT) serves as both the generator and editor, with a smaller model acting as the instructor to guide output generation.
Check Your Facts and Try Again: Improving Large Language Models with External Knowledge and Automated Feedback
Large language models (LLMs), such as ChatGPT, are able to generate human-like, fluent responses for many downstream tasks, e. g., task-oriented dialog and question answering.
MADAv2: Advanced Multi-Anchor Based Active Domain Adaptation Segmentation
Unsupervised domain adaption has been widely adopted in tasks with scarce annotated data.
Look Before You Match: Instance Understanding Matters in Video Object Segmentation
Towards this goal, we present a two-branch network for VOS, where the query-based instance segmentation (IS) branch delves into the instance details of the current frame and the VOS branch performs spatial-temporal matching with the memory bank.
Ranked #1 on
Semi-Supervised Video Object Segmentation
on Long Video Dataset
(using extra training data)
Improving Commonsense in Vision-Language Models via Knowledge Graph Riddles
Through our analysis, we find one important reason is that existing large-scale VL datasets do not contain much commonsense knowledge, which motivates us to improve the commonsense of VL-models from the data perspective.
OmniVL:One Foundation Model for Image-Language and Video-Language Tasks
This paper presents OmniVL, a new foundation model to support both image-language and video-language tasks using one universal architecture.
Ranked #4 on
Cross-Modal Retrieval
on Flickr30k
(using extra training data)
Visual Clues: Bridging Vision and Language Foundations for Image Paragraph Captioning
Thanks to the strong zero-shot capability of foundation models, we start by constructing a rich semantic representation of the image (e. g., image tags, object attributes / locations, captions) as a structured textual prompt, called visual clues, using a vision foundation model.
REVIVE: Regional Visual Representation Matters in Knowledge-Based Visual Question Answering
Specifically, we observe that in most state-of-the-art knowledge-based VQA methods: 1) visual features are extracted either from the whole image or in a sliding window manner for retrieving knowledge, and the important relationship within/among object regions is neglected; 2) visual features are not well utilized in the final answering model, which is counter-intuitive to some extent.
Ranked #11 on
Visual Question Answering (VQA)
on OK-VQA
i-Code: An Integrative and Composable Multimodal Learning Framework
Human intelligence is multimodal; we integrate visual, linguistic, and acoustic signals to maintain a holistic worldview.
K-LITE: Learning Transferable Visual Models with External Knowledge
We propose K-LITE, a simple strategy to leverage external knowledge for building transferable visual systems: In training, it enriches entities in text with WordNet and Wiktionary knowledge, leading to an efficient and scalable approach to learning image representations that uses knowledge about the visual concepts.
Differentiable Top-k with Optimal Transport
Finding the k largest or smallest elements from a collection of scores, i. e., top-k operation, is an important model component widely used in information retrieval, machine learning, and data mining.
A Hypergradient Approach to Robust Regression without Correspondence
Due to the combinatorial nature of the problem, most existing methods are only applicable when the sample size is small, and limited to linear regression models.
Differentiable Top-$k$ with Optimal Transport
The top-$k$ operation, i. e., finding the $k$ largest or smallest elements from a collection of scores, is an important model component, which is widely used in information retrieval, machine learning, and data mining.
Conditional Self-Attention for Query-based Summarization
Thereby, the contextual dependencies modeled by CSA will be highly relevant to the query.
Differentiable Top-k Operator with Optimal Transport
The top-k operation, i. e., finding the k largest or smallest elements from a collection of scores, is an important model component, which is widely used in information retrieval, machine learning, and data mining.
Meta Learning with Relational Information for Short Sequences
This paper proposes a new meta-learning method -- named HARMLESS (HAwkes Relational Meta LEarning method for Short Sequences) for learning heterogeneous point process models from short event sequence data along with a relational network.
On Scalable and Efficient Computation of Large Scale Optimal Transport
Optimal Transport (OT) naturally arises in many machine learning applications, yet the heavy computational burden limits its wide-spread uses.
Active Image Synthesis for Efficient Labeling
The great success achieved by deep neural networks attracts increasing attention from the manufacturing and healthcare communities.
Generative Invertible Networks (GIN): Pathophysiology-Interpretable Feature Mapping and Virtual Patient Generation
Machine learning methods play increasingly important roles in pre-procedural planning for complex surgeries and interventions.
A Fast Proximal Point Method for Computing Exact Wasserstein Distance
However, as we will demonstrate, regularized variations with large regularization parameter will degradate the performance in several important machine learning applications, and small regularization parameter will fail due to numerical stability issues with existing algorithms.
