Search Results for author:
Found 45 papers, 24 papers with code
InfiGUI-R1: Advancing Multimodal GUI Agents from Reactive Actors to Deliberative Reasoners
Recent works have begun exploring reasoning in GUI tasks with encouraging results.
Speculative Thinking: Enhancing Small-Model Reasoning with Large Model Guidance at Inference Time
Moreover, when applied to a non-reasoning model (Qwen-2. 5-7B-Instruct), our framework boosts its accuracy from 74. 0% to 81. 8% on the same benchmark, achieving a relative improvement of 7. 8%.
CausalRAG: Integrating Causal Graphs into Retrieval-Augmented Generation
Large language models (LLMs) have revolutionized natural language processing (NLP), particularly through Retrieval-Augmented Generation (RAG), which enhances LLM capabilities by integrating external knowledge.
You Only Debias Once: Towards Flexible Accuracy-Fairness Trade-offs at Inference Time
Instead of pursuing one individual fixed point (fairness-optimum) in the weight space, we aim to find a "line" in the weight space that connects the accuracy-optimum and fairness-optimum points using a single model.
More for Keys, Less for Values: Adaptive KV Cache Quantization
Although prior work has underscored the distinct roles of key and value cache during inference, our systematic analysis -- examining singular value distributions, spectral norms, and Frobenius norms -- reveals, for the first time, that key matrices consistently exhibit higher norm values and are more sensitive to quantization than value matrices.
Thinking Preference Optimization
To further boost the performance with the SFT data, we propose Thinking Preference Optimization (ThinkPO), a simple yet effective post-SFT method that enhances long CoT reasoning without requiring new long CoT responses.
InfiR : Crafting Effective Small Language Models and Multimodal Small Language Models in Reasoning
Large Language Models (LLMs) and Multimodal Large Language Models (MLLMs) have made significant advancements in reasoning capabilities.
InfiGUIAgent: A Multimodal Generalist GUI Agent with Native Reasoning and Reflection
Graphical User Interface (GUI) Agents, powered by multimodal large language models (MLLMs), have shown great potential for task automation on computing devices such as computers and mobile phones.
NeuralPLexer3: Accurate Biomolecular Complex Structure Prediction with Flow Models
Structure determination is essential to a mechanistic understanding of diseases and the development of novel therapeutics.
Dynamic Self-Distillation via Previous Mini-batches for Fine-tuning Small Language Models
Knowledge distillation (KD) has become a widely adopted approach for compressing large language models (LLMs) to reduce computational costs and memory footprints.
DreamClear: High-Capacity Real-World Image Restoration with Privacy-Safe Dataset Curation
Our second contribution, DreamClear, is a DiT-based image restoration model.
Gradient Rewiring for Editable Graph Neural Network Training
In this paper, we first observe the gradient of cross-entropy loss for the target node and training nodes with significant inconsistency, which indicates that directly fine-tuning the base model using the loss on the target node deteriorates the performance on training nodes.
BabelBench: An Omni Benchmark for Code-Driven Analysis of Multimodal and Multistructured Data
BabelBench incorporates a dataset comprising 247 meticulously curated problems that challenge the models with tasks in perception, commonsense reasoning, logical reasoning, and so on.
InfiMM-WebMath-40B: Advancing Multimodal Pre-Training for Enhanced Mathematical Reasoning
Pre-training on large-scale, high-quality datasets is crucial for enhancing the reasoning capabilities of Large Language Models (LLMs), especially in specialized domains such as mathematics.
Visual Anchors Are Strong Information Aggregators For Multimodal Large Language Model
In the realm of Multimodal Large Language Models (MLLMs), vision-language connector plays a crucial role to link the pre-trained vision encoders with Large Language Models (LLMs).
ViTAR: Vision Transformer with Any Resolution
Firstly, we propose a novel module for dynamic resolution adjustment, designed with a single Transformer block, specifically to achieve highly efficient incremental token integration.
InfiMM-HD: A Leap Forward in High-Resolution Multimodal Understanding
Multimodal Large Language Models (MLLMs) have experienced significant advancements recently.
Ranked #120 on
Visual Question Answering
on MM-Vet
COCO is "ALL'' You Need for Visual Instruction Fine-tuning
We argue that datasets with diverse and high-quality detailed instruction following annotations are essential and adequate for MLLMs IFT.
Ranked #132 on
Visual Question Answering
on MM-Vet
Exploring the Reasoning Abilities of Multimodal Large Language Models (MLLMs): A Comprehensive Survey on Emerging Trends in Multimodal Reasoning
In this survey, we comprehensively review the existing evaluation protocols of multimodal reasoning, categorize and illustrate the frontiers of MLLMs, introduce recent trends in applications of MLLMs on reasoning-intensive tasks, and finally discuss current practices and future directions.
LLM Maybe LongLM: Self-Extend LLM Context Window Without Tuning
To achieve this goal, we propose SelfExtend to extend the context window of LLMs by constructing bi-level attention information: the grouped attention and the neighbor attention.
PokeMQA: Programmable knowledge editing for Multi-hop Question Answering
Multi-hop question answering (MQA) is one of the challenging tasks to evaluate machine's comprehension and reasoning abilities, where large language models (LLMs) have widely achieved the human-comparable performance.
Chasing Fairness in Graphs: A GNN Architecture Perspective
To this end, we aim to achieve fairness via a new GNN architecture.
InfiMM-Eval: Complex Open-Ended Reasoning Evaluation For Multi-Modal Large Language Models
To mitigate this issue, we manually curate a benchmark dataset specifically designed for MLLMs, with a focus on complex reasoning tasks.
Marginal Nodes Matter: Towards Structure Fairness in Graphs
Specifically, we first analyzed multiple graphs and observed that marginal nodes in graphs have a worse performance of downstream tasks than others in graph neural networks.
GrowLength: Accelerating LLMs Pretraining by Progressively Growing Training Length
Our method progressively increases the training length throughout the pretraining phase, thereby mitigating computational costs and enhancing efficiency.
On the Equivalence of Graph Convolution and Mixup
We establish this equivalence mathematically by demonstrating that graph convolution networks (GCN) and simplified graph convolution (SGC) can be expressed as a form of Mixup.
Beyond Fairness: Age-Harmless Parkinson's Detection via Voice
Through our investigation, the discrepancy between the elderly and the young arises due to 1) an imbalanced dataset and 2) the milder symptoms often seen in early-onset patients.
Towards Assumption-free Bias Mitigation
Those studies face challenges, either in inaccurate predictions of sensitive attributes or the need to mitigate unequal distribution of manually defined non-sensitive attributes related to bias.
FFB: A Fair Fairness Benchmark for In-Processing Group Fairness Methods
This paper introduces the Fair Fairness Benchmark (\textsf{FFB}), a benchmarking framework for in-processing group fairness methods.
Harnessing the Power of LLMs in Practice: A Survey on ChatGPT and Beyond
This paper presents a comprehensive and practical guide for practitioners and end-users working with Large Language Models (LLMs) in their downstream natural language processing (NLP) tasks.
Does Synthetic Data Generation of LLMs Help Clinical Text Mining?
Our method has resulted in significant improvements in the performance of downstream tasks, improving the F1-score from 23. 37% to 63. 99% for the named entity recognition task and from 75. 86% to 83. 59% for the relation extraction task.
Chasing Fairness Under Distribution Shift: A Model Weight Perturbation Approach
Motivated by these sufficient conditions, we propose robust fairness regularization (RFR) by considering the worst case within the model weight perturbation ball for each sensitive attribute group.
Retiring $Δ$DP: New Distribution-Level Metrics for Demographic Parity
Unfortunately, in this paper, we reveal that the fairness metric $\Delta DP$ can not precisely measure the violation of demographic parity, because it inherently has the following drawbacks: i) zero-value $\Delta DP$ does not guarantee zero violation of demographic parity, ii) $\Delta DP$ values can vary with different classification thresholds.
MLPInit: Embarrassingly Simple GNN Training Acceleration with MLP Initialization
Training graph neural networks (GNNs) on large graphs is complex and extremely time consuming.
Auto-PINN: Understanding and Optimizing Physics-Informed Neural Architecture
Physics-informed neural networks (PINNs) are revolutionizing science and engineering practice by bringing together the power of deep learning to bear on scientific computation.
G-Mixup: Graph Data Augmentation for Graph Classification
To this end, we propose $\mathcal{G}$-Mixup to augment graphs for graph classification by interpolating the generator (i. e., graphon) of different classes of graphs.
Geometric Graph Representation Learning via Maximizing Rate Reduction
Learning discriminative node representations benefits various downstream tasks in graph analysis such as community detection and node classification.
FMP: Toward Fair Graph Message Passing against Topology Bias
Despite recent advances in achieving fair representations and predictions through regularization, adversarial debiasing, and contrastive learning in graph neural networks (GNNs), the working mechanism (i. e., message passing) behind GNNs inducing unfairness issue remains unknown.
MMPTRACK: Large-scale Densely Annotated Multi-camera Multiple People Tracking Benchmark
This dataset provides a more reliable benchmark of multi-camera, multi-object tracking systems in cluttered and crowded environments.
Ranked #2 on
Object Tracking
on MMPTRACK
G-Mixup: Graph Augmentation for Graph Classification
To this end, we propose $\mathcal{G}$-Mixup to augment graphs for graph classification by interpolating the generator (i. e., graphon) of different classes of graphs.
Generalized Demographic Parity for Group Fairness
We show the understanding of GDP from the probability perspective and theoretically reveal the connection between GDP regularizer and adversarial debiasing.
Image Scene Graph Generation (SGG) Benchmark
There is a surge of interest in image scene graph generation (object, attribute and relationship detection) due to the need of building fine-grained image understanding models that go beyond object detection.
AutoRec: An Automated Recommender System
To address the need, we present AutoRec, an open-source automated machine learning (AutoML) platform extended from the TensorFlow ecosystem and, to our knowledge, the first framework to leverage AutoML for model search and hyperparameter tuning in deep recommendation models.
Deep Collaborative Filtering with Multi-Aspect Information in Heterogeneous Networks
However, the characteristics of users and the properties of items may stem from different aspects, e. g., the brand-aspect and category-aspect of items.
DMGroup at EmoInt-2017: Emotion Intensity Using Ensemble Method
In stage1, we employ both linear and nonlinear regression models to obtain a more diverse emotion intensity representation.
