Search Results for author:
Found 425 papers, 184 papers with code
Leveraging Capsule Routing to Associate Knowledge with Medical Literature Hierarchically
Integrating knowledge into text is a promising way to enrich text representation, especially in the medical field.
"What's Up, Doc?": Analyzing How Users Seek Health Information in Large-Scale Conversational AI Datasets
People are increasingly seeking healthcare information from large language models (LLMs) via interactive chatbots, yet the nature and inherent risks of these conversations remain largely unexplored.
FixCLR: Negative-Class Contrastive Learning for Semi-Supervised Domain Generalization
These experiments include benchmarking different improvements to semi-supervised methods, evaluating the performance of pretrained versus non-pretrained models, and testing on datasets with many domains.
SwiftSpec: Ultra-Low Latency LLM Decoding by Scaling Asynchronous Speculative Decoding
Low-latency decoding for large language models (LLMs) is crucial for applications like chatbots and code assistants, yet generating long outputs remains slow in single-query settings.
DUN-SRE: Deep Unrolling Network with Spatiotemporal Rotation Equivariance for Dynamic MRI Reconstruction
The DUN-SRE establishes spatiotemporal equivariance through a (2+1)D equivariant convolutional architecture.
Non-Contact Health Monitoring During Daily Personal Care Routines
Remote photoplethysmography (rPPG) enables non-contact, continuous monitoring of physiological signals and offers a practical alternative to traditional health sensing methods.
RADAR: Benchmarking Language Models on Imperfect Tabular Data
Language models (LMs) are increasingly being deployed to perform autonomous data analyses.
LSM-2: Learning from Incomplete Wearable Sensor Data
Foundation models, a cornerstone of recent advancements in machine learning, have predominantly thrived on complete and well-structured data.
Simplifying Root Cause Analysis in Kubernetes with StateGraph and LLM
We evaluate SynergyRCA using datasets from two production Kubernetes clusters, highlighting its capacity to identify numerous root causes, including novel ones, with high efficiency and precision.
On the Necessity of Multi-Domain Explanation: An Uncertainty Principle Approach for Deep Time Series Models
This principle establishes a lower bound on how much a signal can be simultaneously localized in both the time and frequency domains.
Period-LLM: Extending the Periodic Capability of Multimodal Large Language Model
Periodic or quasi-periodic phenomena reveal intrinsic characteristics in various natural processes, such as weather patterns, movement behaviors, traffic flows, and biological signals.
BehaviorSFT: Behavioral Token Conditioning for Clinical Agents Across the Proactivity Spectrum
To address this, we propose BehaviorSFT, a novel training strategy using behavioral tokens to explicitly condition LLMs for dynamic behavioral selection along this spectrum.
Think-RM: Enabling Long-Horizon Reasoning in Generative Reward Models
We then further improve the model's long-horizon abilities by rule-based reinforcement learning (RL).
EcomScriptBench: A Multi-task Benchmark for E-commerce Script Planning via Step-wise Intention-Driven Product Association
We propose a novel framework that enables the scalable generation of product-enriched scripts by associating products with each step based on the semantic similarity between the actions and their purchase intentions.
OmniFC: Rethinking Federated Clustering via Lossless and Secure Distance Reconstruction
Federated clustering (FC) aims to discover global cluster structures across decentralized clients without sharing raw data, making privacy preservation a fundamental requirement.
JNLP at SemEval-2025 Task 11: Cross-Lingual Multi-Label Emotion Detection Using Generative Models
Our paper focuses on two sub-tracks of this task: (1) Track A: Multi-label emotion detection, and (2) Track B: Emotion intensity.
VocalAgent: Large Language Models for Vocal Health Diagnostics with Safety-Aware Evaluation
Vocal health plays a crucial role in peoples' lives, significantly impacting their communicative abilities and interactions.
FEALLM: Advancing Facial Emotion Analysis in Multimodal Large Language Models with Emotional Synergy and Reasoning
Facial Emotion Analysis (FEA) plays a crucial role in visual affective computing, aiming to infer a person's emotional state based on facial data.
MegaScale-MoE: Large-Scale Communication-Efficient Training of Mixture-of-Experts Models in Production
We present MegaScale-MoE, a production system tailored for the efficient training of large-scale mixture-of-experts (MoE) models.
EmotionHallucer: Evaluating Emotion Hallucinations in Multimodal Large Language Models
By evaluating 38 LLMs and MLLMs on EmotionHallucer, we reveal that: i) most current models exhibit substantial issues with emotion hallucinations; ii) closed-source models outperform open-source ones in detecting emotion hallucinations, and reasoning capability provides additional advantages; iii) existing models perform better in emotion psychology knowledge than in multimodal emotion perception.
VeriFact: Enhancing Long-Form Factuality Evaluation with Refined Fact Extraction and Reference Facts
Large language models (LLMs) excel at generating long-form responses, but evaluating their factuality remains challenging due to complex inter-sentence dependencies within the generated facts.
Denoising and Alignment: Rethinking Domain Generalization for Multimodal Face Anti-Spoofing
We also design a \textbf{U}-shaped \textbf{D}ual \textbf{S}pace \textbf{A}daptation (\textbf{U-DSA}) module to enhance the adaptability of representations while maintaining generalization performance.
Implet: A Post-hoc Subsequence Explainer for Time Series Models
Explainability in time series models is crucial for fostering trust, facilitating debugging, and ensuring interpretability in real-world applications.
Seed1.5-VL Technical Report
We present Seed1. 5-VL, a vision-language foundation model designed to advance general-purpose multimodal understanding and reasoning.
Ranked #1 on
Video Question Answering
on TVBench
Understanding Stragglers in Large Model Training Using What-if Analysis
Large language model (LLM) training is one of the most demanding distributed computations today, often requiring thousands of GPUs with frequent synchronization across machines.
Passive Measurement of Autonomic Arousal in Real-World Settings
The autonomic nervous system (ANS) is activated during stress, which can have negative effects on cardiovascular health, sleep, the immune system, and mental health.
MMHCL: Multi-Modal Hypergraph Contrastive Learning for Recommendation
For a comprehensive information exploration from user-product relations, we construct two hypergraphs, i. e. a user-to-user (u2u) hypergraph and an item-to-item (i2i) hypergraph, to mine shared preferences among users and intricate multimodal semantic resemblance among items, respectively.
NTIRE 2025 Challenge on Image Super-Resolution ($\times$4): Methods and Results
This paper presents the NTIRE 2025 image super-resolution ($\times$4) challenge, one of the associated competitions of the 10th NTIRE Workshop at CVPR 2025.
KeepKV: Eliminating Output Perturbation in KV Cache Compression for Efficient LLMs Inference
Efficient inference of large language models (LLMs) is hindered by an ever-growing key-value (KV) cache, making KV cache compression a critical research direction.
OVERLORD: Ultimate Scaling of DataLoader for Multi-Source Large Foundation Model Training
This also hinders dynamic data mixing (e. g., curriculum learning) and causes redundant access/memory overhead in hybrid parallelism.
HM-RAG: Hierarchical Multi-Agent Multimodal Retrieval Augmented Generation
While Retrieval-Augmented Generation (RAG) augments Large Language Models (LLMs) with external knowledge, conventional single-agent RAG remains fundamentally limited in resolving complex queries demanding coordinated reasoning across heterogeneous data ecosystems.
Multi-Modal Hypergraph Enhanced LLM Learning for Recommendation
To this end, we propose a novel framework, Hypergraph Enhanced LLM Learning for multimodal Recommendation (HeLLM), designed to equip LLMs with the capability to capture intricate higher-order semantic correlations by fusing graph-level contextual signals with sequence-level behavioral patterns.
Seed1.5-Thinking: Advancing Superb Reasoning Models with Reinforcement Learning
We introduce Seed1. 5-Thinking, capable of reasoning through thinking before responding, resulting in improved performance on a wide range of benchmarks.
VAPO: Efficient and Reliable Reinforcement Learning for Advanced Reasoning Tasks
We present VAPO, Value-based Augmented Proximal Policy Optimization framework for reasoning models., a novel framework tailored for reasoning models within the value-based paradigm.
MegaScale-Infer: Serving Mixture-of-Experts at Scale with Disaggregated Expert Parallelism
Combined with distinct model parallelism for each module, MegaScale-Infer effectively hides communication overhead and maximizes GPU utilization.
AU-TTT: Vision Test-Time Training model for Facial Action Unit Detection
Facial Action Units (AUs) detection is a cornerstone of objective facial expression analysis and a critical focus in affective computing.
A Scalable Framework for Evaluating Health Language Models
Our approach is based on recent work in more general evaluation settings that contrasts a smaller set of complex evaluation targets with a larger set of more precise, granular targets answerable with simple boolean responses.
Towards Fully Automated Decision-Making Systems for Greenhouse Control: Challenges and Opportunities
Machine learning has been successful in building control policies to drive a complex system to desired states in various applications (e. g. games, robotics, etc.).
Look Before Leap: Look-Ahead Planning with Uncertainty in Reinforcement Learning
In the policy optimization phase, we leverage an uncertainty-driven exploratory policy to actively collect diverse training samples, resulting in improved model accuracy and overall performance of the RL agent.
Substance over Style: Evaluating Proactive Conversational Coaching Agents
While NLP research has made strides in conversational tasks, many approaches focus on single-turn responses with well-defined objectives or evaluation criteria.
Adventurer: Exploration with BiGAN for Deep Reinforcement Learning
Unfortunately, no single algorithm outperforms all others in all tasks and most of them struggle with tasks with high-dimensional and complex observations.
DAPO: An Open-Source LLM Reinforcement Learning System at Scale
Inference scaling empowers LLMs with unprecedented reasoning ability, with reinforcement learning as the core technique to elicit complex reasoning.
ZeroMerge: Parameter-Free KV Cache Compression for Memory-Efficient Long-Context LLMs
The linear growth of key-value (KV) cache memory and quadratic computational complexity pose significant bottlenecks for large language models (LLMs) in long-context processing.
CATANet: Efficient Content-Aware Token Aggregation for Lightweight Image Super-Resolution
SR typically leverages the redundancy of images for reconstruction, and this redundancy appears not only in local regions but also in long-range regions.
A Graph-Partitioning Based Continuous Optimization Approach to Semi-supervised Clustering Problems
In this work, we view the semi-supervised clustering task as a partitioning problem on a graph associated with the given dataset, where the similarity matrix includes a scaling parameter to reflect the must-link constraints.
The Distributionally Robust Optimization Model of Sparse Principal Component Analysis
We prove the Riemannian gradient consistency and global convergence of our algorithm to a stationary point of the nonsmooth minimization problem.
Passive Heart Rate Monitoring During Smartphone Use in Everyday Life
Resting heart rate (RHR) is an important biomarker of cardiovascular health and mortality, but tracking it longitudinally generally requires a wearable device, limiting its availability.
AutoLUT: LUT-Based Image Super-Resolution with Automatic Sampling and Adaptive Residual Learning
Moreover, their reliance on fixed sampling patterns limits both accuracy and the ability to capture fine details in low-resolution images.
DADM: Dual Alignment of Domain and Modality for Face Anti-spoofing
We identify two main types of misalignment: (1) \textbf{Intra-domain modality misalignment}, where the importance of each modality varies across different attacks.
ByteScale: Efficient Scaling of LLM Training with a 2048K Context Length on More Than 12,000 GPUs
The mismatch between data heterogeneity and static mesh causes redundant communication and imbalanced computation, degrading the training efficiency.
ReCon: Enhancing True Correspondence Discrimination through Relation Consistency for Robust Noisy Correspondence Learning
To address this problem, we propose a general Relation Consistency learning framework, namely ReCon, to accurately discriminate the true correspondences among the multimodal data and thus effectively mitigate the adverse impact caused by mismatches.
Cross-modal retrieval with noisy correspondence
Image-text matching
+2
Comet: Fine-grained Computation-communication Overlapping for Mixture-of-Experts
The inter-device communication of a MoE layer can occupy 47% time of the entire model execution with popular models and frameworks.
BIG-Bench Extra Hard
One particular exception is the BIG-Bench dataset, which has served as a crucial benchmark for evaluating the general reasoning capabilities of LLMs, thanks to its diverse set of challenging tasks that allowed for a comprehensive assessment of general reasoning across various skills within a unified framework.
Talking to the brain: Using Large Language Models as Proxies to Model Brain Semantic Representation
Traditional psychological experiments utilizing naturalistic stimuli face challenges in manual annotation and ecological validity.
Medical Hallucinations in Foundation Models and Their Impact on Healthcare
Our contributions include (1) a taxonomy for understanding and addressing medical hallucinations, (2) benchmarking models using medical hallucination dataset and physician-annotated LLM responses to real medical cases, providing direct insight into the clinical impact of hallucinations, and (3) a multi-national clinician survey on their experiences with medical hallucinations.
Discriminative Finetuning of Generative Large Language Models without Reward Models and Human Preference Data
To address its limitations, the existing common strategy is to follow SFT with a separate phase of preference optimization (PO), which relies on either human-labeled preference data or a strong reward model to guide the learning process.
VLM-E2E: Enhancing End-to-End Autonomous Driving with Multimodal Driver Attention Fusion
Our method integrates textual representations into Bird's-Eye-View (BEV) features for semantic supervision, which enables the model to learn richer feature representations that explicitly capture the driver's attentional semantics.
PlanGEN: A Multi-Agent Framework for Generating Planning and Reasoning Trajectories for Complex Problem Solving
Recent agent frameworks and inference-time algorithms often struggle with complex planning problems due to limitations in verifying generated plans or reasoning and varying complexity of instances within a single task.
IHEval: Evaluating Language Models on Following the Instruction Hierarchy
The instruction hierarchy, which establishes a priority order from system messages to user messages, conversation history, and tool outputs, is essential for ensuring consistent and safe behavior in language models (LMs).
CodePhys: Robust Video-based Remote Physiological Measurement through Latent Codebook Querying
In this paper, we propose a novel method named CodePhys, which innovatively treats rPPG measurement as a code query task in a noise-free proxy space (i. e., codebook) constructed by ground-truth PPG signals.
Hephaestus: Improving Fundamental Agent Capabilities of Large Language Models through Continual Pre-Training
Due to the scarcity of agent-oriented pre-training data, LLM-based autonomous agents typically rely on complex prompting or extensive fine-tuning, which often fails to introduce new capabilities while preserving strong generalizability.
Confidence intervals for intentionally biased estimators
We propose and study three confidence intervals (CIs) centered at an estimator that is intentionally biased to reduce mean squared error.
DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning
We introduce our first-generation reasoning models, DeepSeek-R1-Zero and DeepSeek-R1.
Ranked #2 on
Mathematical Reasoning
on AIME24
UI-TARS: Pioneering Automated GUI Interaction with Native Agents
This paper introduces UI-TARS, a native GUI agent model that solely perceives the screenshots as input and performs human-like interactions (e. g., keyboard and mouse operations).
Advancing ALS Applications with Large-Scale Pre-training: Dataset Development and Downstream Assessment
In this study, we address this gap by constructing a large-scale ALS point cloud dataset and evaluating its impact on downstream applications.
Microservice Deployment in Space Computing Power Networks via Robust Reinforcement Learning
With the growing demand for Earth observation, it is important to provide reliable real-time remote sensing inference services to meet the low-latency requirements.
No Pains, More Gains: Recycling Sub-Salient Patches for Efficient High-Resolution Image Recognition
Over the last decade, many notable methods have emerged to tackle the computational resource challenge of the high resolution image recognition (HRIR).
STDD: Spatio-Temporal Dual Diffusion for Video Generation
In this work, we propose an explicit Spatio-Temporal Dual Diffusion (STDD) method by principledly extending the standard diffusion model to a spatio-temporal diffusion model for joint spatial and temporal noise propagation/reduction.
Distribution Prototype Diffusion Learning for Open-set Supervised Anomaly Detection
In Open-set Supervised Anomaly Detection (OSAD), the existing methods typically generate pseudo anomalies to compensate for the scarcity of observed anomaly samples, while overlooking critical priors of normal samples, leading to less effective discriminative boundaries.
FSBench: A Figure Skating Benchmark for Advancing Artistic Sports Understanding
Figure skating, known as the "Art on Ice," is among the most artistic sports, challenging to understand due to its blend of technical elements (like jumps and spins) and overall artistic expression.
DeepSeek-V3 Technical Report
We present DeepSeek-V3, a strong Mixture-of-Experts (MoE) language model with 671B total parameters with 37B activated for each token.
Do Multimodal Language Models Really Understand Direction? A Benchmark for Compass Direction Reasoning
Direction reasoning is essential for intelligent systems to understand the real world.
The Impact of Cut Layer Selection in Split Federated Learning
Split Federated Learning (SFL) is a distributed machine learning paradigm that combines federated learning and split learning.
Intention Knowledge Graph Construction for User Intention Relation Modeling
Understanding user intentions is challenging for online platforms.
Re-Attentional Controllable Video Diffusion Editing
Therefore, the controllability of video editing remains a formidable challenge.
Sample-efficient Unsupervised Policy Cloning from Ensemble Self-supervised Labeled Videos
In this paper, we try to let machines replicate this efficient watching-and-learning process through Unsupervised Policy from Ensemble Self-supervised labeled Videos (UPESV), a novel framework to efficiently learn policies from videos without any other expert supervision.
SpecFuse: Ensembling Large Language Models via Next-Segment Prediction
In this paper, we propose SpecFuse, a novel ensemble framework that outputs the fused result by iteratively producing the next segment through collaboration among LLMs.
Hyperspectral Image Spectral-Spatial Feature Extraction via Tensor Principal Component Analysis
This paper addresses the challenge of spectral-spatial feature extraction for hyperspectral image classification by introducing a novel tensor-based framework.
Hyperspectral image analysis
Hyperspectral Image Classification
+1
Safe and Efficient Online Convex Optimization with Linear Budget Constraints and Partial Feedback
This paper studies online convex optimization with unknown linear budget constraints, where only the gradient information of the objective and the bandit feedback of constraint functions are observed.
Distributed Sign Momentum with Local Steps for Training Transformers
Pre-training Transformer models is resource-intensive, and recent studies have shown that sign momentum is an efficient technique for training large-scale deep learning models, particularly Transformers.
GMFlow: Global Motion-Guided Recurrent Flow for 6D Object Pose Estimation
6D object pose estimation is crucial for robotic perception and precise manipulation.
A Plug-and-Play Temporal Normalization Module for Robust Remote Photoplethysmography
Remote photoplethysmography (rPPG) extracts PPG signals from subtle color changes in facial videos, showing strong potential for health applications.
SEMPose: A Single End-to-end Network for Multi-object Pose Estimation
On the LM-O and YCB-V datasets, our method outperforms other RGB-based single-model methods, achieving higher accuracy.
MLAN: Language-Based Instruction Tuning Improves Zero-Shot Generalization of Multimodal Large Language Models
With a small number of visual instructions, this emerging language instruction following ability transfers well to the unseen vision datasets, outperforming the state of the art with greater training efficiency.
PICZL: Image-based Photometric Redshifts for AGN
Instead, we employ readily available data products from the 10th Data Release of the Imaging Legacy Survey for DESI, covering > 20, 000 deg$^{2}$ with deep images and catalog-based photometry in the grizW1-W4 bands.
Shopping MMLU: A Massive Multi-Task Online Shopping Benchmark for Large Language Models
Shopping MMLU consists of 57 tasks covering 4 major shopping skills: concept understanding, knowledge reasoning, user behavior alignment, and multi-linguality, and can thus comprehensively evaluate the abilities of LLMs as general shop assistants.
ShadowKV: KV Cache in Shadows for High-Throughput Long-Context LLM Inference
By evaluating ShadowKV on a broad range of benchmarks, including RULER, LongBench, and Needle In A Haystack, and models like Llama-3. 1-8B, Llama-3-8B-1M, GLM-4-9B-1M, Yi-9B-200K, Phi-3-Mini-128K, and Qwen2-7B-128K, we demonstrate that it can support up to 6$\times$ larger batch sizes and boost throughput by up to 3. 04$\times$ on an A100 GPU without sacrificing accuracy, even surpassing the performance achievable with infinite batch size under the assumption of infinite GPU memory.
A Stock Price Prediction Approach Based on Time Series Decomposition and Multi-Scale CNN using OHLCT Images
By utilizing CNN to learn sequential features and combining them with image features, we improve the accuracy of stock trend prediction on the A-share market stock dataset.
Enhancing Safety in Reinforcement Learning with Human Feedback via Rectified Policy Optimization
To address this issue, we propose Rectified Policy Optimization (RePO), which replaces the expected safety constraint with critical safety constraints imposed on every prompt.
SDP4Bit: Toward 4-bit Communication Quantization in Sharded Data Parallelism for LLM Training
Recent years have witnessed a clear trend towards language models with an ever-increasing number of parameters, as well as the growing training overhead and memory usage.
HACSurv: A Hierarchical Copula-Based Approach for Survival Analysis with Dependent Competing Risks
In this paper, we introduce HACSurv, a survival analysis method that learns Hierarchical Archimedean Copulas structures and cause-specific survival functions from data with competing risks.
GeSubNet: Gene Interaction Inference for Disease Subtype Network Generation
Retrieving gene functional networks from knowledge databases presents a challenge due to the mismatch between disease networks and subtype-specific variations.
Scaling Wearable Foundation Models
Wearable sensors have become ubiquitous thanks to a variety of health tracking features.
Interpreting Inflammation Prediction Model via Tag-based Cohort Explanation
Machine learning is revolutionizing nutrition science by enabling systems to learn from data and make intelligent decisions.
CohEx: A Generalized Framework for Cohort Explanation
eXplainable Artificial Intelligence (XAI) has garnered significant attention for enhancing transparency and trust in machine learning models.
Explainable artificial intelligence
Explainable Artificial Intelligence (XAI)
Evidence of Cognitive Deficits andDevelopmental Advances in Generative AI: A Clock Drawing Test Analysis
Generative AI's rapid advancement sparks interest in its cognitive abilities, especially given its capacity for tasks like language understanding and code generation.
MoE-Pruner: Pruning Mixture-of-Experts Large Language Model using the Hints from Its Router
Experimental results demonstrate that the Mixtral-8x7B model with 50% sparsity maintains 99% of the performance of the original model after the expert-wise knowledge distillation.
The Cognitive Capabilities of Generative AI: A Comparative Analysis with Human Benchmarks
There is increasing interest in tracking the capabilities of general intelligence foundation models.
Persona Knowledge-Aligned Prompt Tuning Method for Online Debate
Debate is the process of exchanging viewpoints or convincing others on a particular issue.
Improving Arabic Multi-Label Emotion Classification using Stacked Embeddings and Hybrid Loss Function
In multi-label emotion classification, particularly for low-resource languages like Arabic, the challenges of class imbalance and label correlation hinder model performance, especially in accurately predicting minority emotions.
Constrained Reasoning Chains for Enhancing Theory-of-Mind in Large Language Models
Afterward, CCoToM prompts LLMs to infer the queried ToM dimension based on the generated related ToM dimensions and corresponding causal relations.
Self-Updating Vehicle Monitoring Framework Employing Distributed Acoustic Sensing towards Real-World Settings
We conducted comparative experiments with multiple update strategies for self-updating and identified an optimal approach.
Infrared and Visible Image Fusion with Hierarchical Human Perception
Existing methods take pixel intensity, texture and high-level vision task information as the standards to determine preservation of information, lacking enhancement for human perception.
DiffFAS: Face Anti-Spoofing via Generative Diffusion Models
Based on our analysis, we propose DiffFAS framework, which quantifies quality as prior information input into the network to counter image quality shift, and performs diffusion-based high-fidelity cross-domain and cross-attack types generation to counter image style shift.
A Double Tracking Method for Optimization with Decentralized Generalized Orthogonality Constraints
In this paper, we consider the decentralized optimization problems with generalized orthogonality constraints, where both the objective function and the constraint exhibit a distributed structure.
Explanation Space: A New Perspective into Time Series Interpretability
Human understandable explanation of deep learning models is necessary for many critical and sensitive applications.
Fire-Flyer AI-HPC: A Cost-Effective Software-Hardware Co-Design for Deep Learning
For DL training, we deployed the Fire-Flyer 2 with 10, 000 PCIe A100 GPUs, achieved performance approximating the DGX-A100 while reducing costs by half and energy consumption by 40%.
Benchmarking Counterfactual Interpretability in Deep Learning Models for Time Series Classification
We specifically redesign the metrics for sparsity and plausibility and introduce a new metric for consistency.
EMO-LLaMA: Enhancing Facial Emotion Understanding with Instruction Tuning
However, current FER paradigms face challenges in generalization, lack semantic information aligned with natural language, and struggle to process both images and videos within a unified framework, making their application in multimodal emotion understanding and human-computer interaction difficult.
Facial Expression Recognition
Facial Expression Recognition (FER)
Attention Is Not What You Need: Revisiting Multi-Instance Learning for Whole Slide Image Classification
From the standard MIL assumptions, we propose a surprisingly simple yet effective instance-based MIL method for WSI classification (FocusMIL) based on max-pooling and forward amortized variational inference.
MU-MAE: Multimodal Masked Autoencoders-Based One-Shot Learning
Mu-MAE integrates a multimodal masked autoencoder with a synchronized masking strategy tailored for wearable sensors.
FacialPulse: An Efficient RNN-based Depression Detection via Temporal Facial Landmarks
Depression is a prevalent mental health disorder that significantly impacts individuals' lives and well-being.
Diffusion Model Meets Non-Exemplar Class-Incremental Learning and Beyond
Non-exemplar class-incremental learning (NECIL) is to resist catastrophic forgetting without saving old class samples.
From Recognition to Prediction: Leveraging Sequence Reasoning for Action Anticipation
ARR decomposes the action anticipation task into action recognition and sequence reasoning tasks, and effectively learns the statistical relationship between actions by next action prediction (NAP).
ByteCheckpoint: A Unified Checkpointing System for Large Foundation Model Development
In production, different LFMs are trained with various frameworks and storage backends, depending on model sizes and training scales.
Adversarial Robustness in RGB-Skeleton Action Recognition: Leveraging Attention Modality Reweighter
We initially conducted empirical analysis on the robustness of different modalities and observed that the skeleton modality is more robust than the RGB modality.
Facial Affect Recognition based on Multi Architecture Encoder and Feature Fusion for the ABAW7 Challenge
In this paper, we present our approach to addressing the challenges of the 7th ABAW competition.
UGNCL: Uncertainty-Guided Noisy Correspondence Learning for Efficient Cross-Modal Matching
Cross-modal matching has recently gained significant popularity to facilitate retrieval across multi-modal data, and existing works are highly relied on an implicit assumption that the training data pairs are perfectly aligned.
Cross-modal retrieval with noisy correspondence
Image-text matching
+1
Tuning Vision-Language Models with Candidate Labels by Prompt Alignment
In order to improve its robustness, we propose a simple yet effective framework that better leverages the prior knowledge of VLMs to guide the learning process with candidate labels.
Zero-Shot Video Restoration and Enhancement Using Pre-Trained Image Diffusion Model
In this paper, we propose the first framework for zero-shot video restoration and enhancement based on the pre-trained image diffusion model.
MMRo: Are Multimodal LLMs Eligible as the Brain for In-Home Robotics?
It is fundamentally challenging for robots to serve as useful assistants in human environments because this requires addressing a spectrum of sub-problems across robotics, including perception, language understanding, reasoning, and planning.
Enhancing Language Model Factuality via Activation-Based Confidence Calibration and Guided Decoding
Built on top of ActCab, we further propose CoDec, a confidence-guided decoding strategy to elicit truthful answers with high confidence from LMs.
What Are the Odds? Language Models Are Capable of Probabilistic Reasoning
Language models (LM) are capable of remarkably complex linguistic tasks; however, numerical reasoning is an area in which they frequently struggle.
WeatherQA: Can Multimodal Language Models Reason about Severe Weather?
In this work, we introduce WeatherQA, the first multimodal dataset designed for machines to reason about complex combinations of weather parameters (a. k. a., ingredients) and predict severe weather in real-world scenarios.
Balancing Embedding Spectrum for Recommendation
In this work, we shed light on an issue in the existing pair-wise learning paradigm (i. e., the embedding collapse problem), that the representations tend to span a subspace of the whole embedding space, leading to a suboptimal solution and reducing the model capacity.
On the Role of Entity and Event Level Conceptualization in Generalizable Reasoning: A Survey of Tasks, Methods, Applications, and Future Directions
Entity- and event-level conceptualization, as fundamental elements of human cognition, plays a pivotal role in generalizable reasoning.
MIND: Multimodal Shopping Intention Distillation from Large Vision-language Models for E-commerce Purchase Understanding
Improving user experience and providing personalized search results in E-commerce platforms heavily rely on understanding purchase intention.
IntentionQA: A Benchmark for Evaluating Purchase Intention Comprehension Abilities of Language Models in E-commerce
Enhancing Language Models' (LMs) ability to understand purchase intentions in E-commerce scenarios is crucial for their effective assistance in various downstream tasks.
How Powerful is Graph Filtering for Recommendation
Based on this observation, we propose a generalized graph normalization G^2N to adjust the sharpness of spectral distribution in order to redistribute data noise to assure that it can be removed by graph filtering without training.
FLUX: Fast Software-based Communication Overlap On GPUs Through Kernel Fusion
Overall, it can achieve up to 1. 24x speedups for training over Megatron-LM on a cluster of 128 GPUs with various GPU generations and interconnects, and up to 1. 66x and 1. 30x speedups for prefill and decoding inference over vLLM on a cluster with 8 GPUs with various GPU generations and interconnects.
Towards a Personal Health Large Language Model
Here we present Personal Health Large Language Model (PH-LLM), fine-tuned from Gemini for understanding and reasoning over numerical time-series personal health data.
Transforming Wearable Data into Health Insights using Large Language Model Agents
Despite the proliferation of wearable health trackers and the importance of sleep and exercise to health, deriving actionable personalized insights from wearable data remains a challenge because doing so requires non-trivial open-ended analysis of these data.
Identity-free Artificial Emotional Intelligence via Micro-Gesture Understanding
The second is whether micro-gestures, as supplementary data, can provide additional insights for emotional understanding.
Learning Future Representation with Synthetic Observations for Sample-efficient Reinforcement Learning
Employing auxiliary tasks allows the agent to enhance visual representation in a targeted manner, thereby improving the sample efficiency and performance of downstream RL.
A Comprehensive Survey of Large Language Models and Multimodal Large Language Models in Medicine
Finally, the survey addresses the challenges confronting medical LLMs and MLLMs and proposes practical strategies and future directions for their integration into medicine.
A Decoupling and Aggregating Framework for Joint Extraction of Entities and Relations
Second, information interaction mainly focuses on the two subtasks, leaving the fine-grained informtion interaction among the subtask-specific features of encoding subjects, relations, and objects unexplored.
Towards Subgraph Isomorphism Counting with Graph Kernels
Subgraph isomorphism counting is known as #P-complete and requires exponential time to find the accurate solution.
Disttack: Graph Adversarial Attacks Toward Distributed GNN Training
In this study, we introduce Disttack, the first framework of adversarial attacks for distributed GNN training that leverages the characteristics of frequent gradient updates in a distributed system.
DeepSeek-V2: A Strong, Economical, and Efficient Mixture-of-Experts Language Model
MLA guarantees efficient inference through significantly compressing the Key-Value (KV) cache into a latent vector, while DeepSeekMoE enables training strong models at an economical cost through sparse computation.
A quantile-based nonadditive fixed effects model
I propose a quantile-based nonadditive fixed effects panel model to study heterogeneous causal effects.
Enhancing Micro Gesture Recognition for Emotion Understanding via Context-aware Visual-Text Contrastive Learning
In addition, instead of using handcrafted prompts for visual-text contrastive learning, we propose a novel module called Adaptive prompting to generate context-aware prompts.
Enhanced Language Model Truthfulness with Learnable Intervention and Uncertainty Expression
The adaptive nature of LITO counters the limitations of one-size-fits-all intervention methods, maximizing truthfulness by reflecting the model's internal knowledge only when it is confident.
EALD-MLLM: Emotion Analysis in Long-sequential and De-identity videos with Multi-modal Large Language Model
In contrast, long sequential videos can reveal authentic emotions; 2) Previous studies commonly utilize various signals such as facial, speech, and even sensitive biological signals (e. g., electrocardiogram).
Real-Time 4K Super-Resolution of Compressed AVIF Images. AIS 2024 Challenge Survey
This paper introduces a novel benchmark as part of the AIS 2024 Real-Time Image Super-Resolution (RTSR) Challenge, which aims to upscale compressed images from 540p to 4K resolution (4x factor) in real-time on commercial GPUs.
NegotiationToM: A Benchmark for Stress-testing Machine Theory of Mind on Negotiation Surrounding
Large Language Models (LLMs) have sparked substantial interest and debate concerning their potential emergence of Theory of Mind (ToM) ability.
The Ninth NTIRE 2024 Efficient Super-Resolution Challenge Report
In sub-track 1, the practical runtime performance of the submissions was evaluated, and the corresponding score was used to determine the ranking.
DEGNN: Dual Experts Graph Neural Network Handling Both Edge and Node Feature Noise
Leveraging these modified representations, DEGNN subsequently addresses downstream tasks, ensuring robustness against noise present in both edges and node features of real-world graphs.
Future-Proofing Class Incremental Learning
Exemplar-Free Class Incremental Learning is a highly challenging setting where replay memory is unavailable.
EventGround: Narrative Reasoning by Grounding to Eventuality-centric Knowledge Graphs
In this work, we propose an initial comprehensive framework called EventGround, which aims to tackle the problem of grounding free-texts to eventuality-centric KGs for contextualized narrative reasoning.
Convergence of Decentralized Stochastic Subgradient-based Methods for Nonsmooth Nonconvex functions
In this paper, we focus on the decentralized stochastic subgradient-based methods in minimizing nonsmooth nonconvex functions without Clarke regularity, especially in the decentralized training of nonsmooth neural networks.
Generation is better than Modification: Combating High Class Homophily Variance in Graph Anomaly Detection
We find that in graph anomaly detection, the homophily distribution differences between different classes are significantly greater than those in homophilic and heterophilic graphs.
Benchmarking Zero-Shot Robustness of Multimodal Foundation Models: A Pilot Study
We show that CLIP leads to a significant robustness drop compared to supervised ImageNet models on our benchmark, especially under synthetic distribution shift and adversarial attacks.
Answering Diverse Questions via Text Attached with Key Audio-Visual Clues
Audio-visual question answering (AVQA) requires reference to video content and auditory information, followed by correlating the question to predict the most precise answer.
Audio-visual Question Answering
Audio-Visual Question Answering (AVQA)
+3
Advancing Generalizable Remote Physiological Measurement through the Integration of Explicit and Implicit Prior Knowledge
Remote photoplethysmography (rPPG) is a promising technology that captures physiological signals from face videos, with potential applications in medical health, emotional computing, and biosecurity recognition.
ACT-MNMT Auto-Constriction Turning for Multilingual Neural Machine Translation
Large language model (LLM) has achieved promising performance in multilingual machine translation tasks through zero/few-shot prompts or prompt-tuning.
Revisiting Edge Perturbation for Graph Neural Network in Graph Data Augmentation and Attack
It can be categorized into two veins based on their effects on the performance of graph neural networks (GNNs), i. e., graph data augmentation and attack.
Mipha: A Comprehensive Overhaul of Multimodal Assistant with Small Language Models
Multimodal Large Language Models (MLLMs) have showcased impressive skills in tasks related to visual understanding and reasoning.
Ranked #185 on
Visual Question Answering
on MM-Vet
AUFormer: Vision Transformers are Parameter-Efficient Facial Action Unit Detectors
Facial Action Units (AU) is a vital concept in the realm of affective computing, and AU detection has always been a hot research topic.
Recent Advances, Applications, and Open Challenges in Machine Learning for Health: Reflections from Research Roundtables at ML4H 2023 Symposium
The organization of the research roundtables at the conference involved 17 Senior Chairs and 19 Junior Chairs across 11 tables.
Multi-modal Attribute Prompting for Vision-Language Models
To address this issue, we propose a Multi-modal Attribute Prompting method (MAP) by jointly exploring textual attribute prompting, visual attribute prompting, and attribute-level alignment.
A Simple yet Effective Network based on Vision Transformer for Camouflaged Object and Salient Object Detection
Camouflaged object detection (COD) and salient object detection (SOD) are two distinct yet closely-related computer vision tasks widely studied during the past decades.
FedUV: Uniformity and Variance for Heterogeneous Federated Learning
Based on this finding, we introduce two regularization terms for local training to continuously emulate IID settings: (1) variance in the dimension-wise probability distribution of the classifier and (2) hyperspherical uniformity of representations of the encoder.
OncoGPT: A Medical Conversational Model Tailored with Oncology Domain Expertise on a Large Language Model Meta-AI (LLaMA)
The primary aim of this research was to develop a specialized language model that demonstrates improved accuracy in providing advice related to oncology.
Learning to Schedule Online Tasks with Bandit Feedback
These render reward and cost hard to model thus unknown before decision making.
MegaScale: Scaling Large Language Model Training to More Than 10,000 GPUs
Training LLMs at this scale brings unprecedented challenges to training efficiency and stability.
Convergence Analysis of Split Federated Learning on Heterogeneous Data
We further extend the analysis to non-convex objectives and the scenario where some clients may be unavailable during training.
Demographic Bias of Expert-Level Vision-Language Foundation Models in Medical Imaging
Such demographic biases present over a wide range of pathologies and demographic attributes.
AbsInstruct: Eliciting Abstraction Ability from LLMs through Explanation Tuning with Plausibility Estimation
Abstraction ability is crucial in human intelligence, which can also benefit various tasks in NLP study.
Safety of Multimodal Large Language Models on Images and Texts
In this paper, we systematically survey current efforts on the evaluation, attack, and defense of MLLMs' safety on images and text.
Fast Adversarial Training against Textual Adversarial Attacks
Based on the observation that the adversarial perturbations crafted by single-step and multi-step gradient ascent are similar, FAT uses single-step gradient ascent to craft adversarial examples in the embedding space to expedite the training process.
CANDLE: Iterative Conceptualization and Instantiation Distillation from Large Language Models for Commonsense Reasoning
The sequential process of conceptualization and instantiation is essential to generalizable commonsense reasoning as it allows the application of existing knowledge to unfamiliar scenarios.
DeepSeek LLM: Scaling Open-Source Language Models with Longtermism
The rapid development of open-source large language models (LLMs) has been truly remarkable.
Adversarially Trained Weighted Actor-Critic for Safe Offline Reinforcement Learning
In theory, we demonstrate that when the actor employs a no-regret optimization oracle, WSAC achieves a number of guarantees: (i) For the first time in the safe offline RL setting, we establish that WSAC can produce a policy that outperforms any reference policy while maintaining the same level of safety, which is critical to designing a safe algorithm for offline RL.
Advancing Abductive Reasoning in Knowledge Graphs through Complex Logical Hypothesis Generation
Although many applications require the use of knowledge for explanations, the utilization of abductive reasoning in conjunction with structured knowledge, such as a knowledge graph, remains largely unexplored.
Enhancing User Intent Capture in Session-Based Recommendation with Attribute Patterns
The goal of session-based recommendation in E-commerce is to predict the next item that an anonymous user will purchase based on the browsing and purchase history.
Safe Reinforcement Learning with Instantaneous Constraints: The Role of Aggressive Exploration
This paper studies safe Reinforcement Learning (safe RL) with linear function approximation and under hard instantaneous constraints where unsafe actions must be avoided at each step.
AutoAugment Input Transformation for Highly Transferable Targeted Attacks
However, few of them are dedicated to input transformation. In this work, we observe a positive correlation between the logit/probability of the target class and diverse input transformation methods in targeted attacks.
MM-SafetyBench: A Benchmark for Safety Evaluation of Multimodal Large Language Models
The security concerns surrounding Large Language Models (LLMs) have been extensively explored, yet the safety of Multimodal Large Language Models (MLLMs) remains understudied.
ALPHA: AnomaLous Physiological Health Assessment Using Large Language Models
Our findings reveal that LLMs exhibit exceptional performance in determining medical indicators, including a Mean Absolute Error (MAE) of less than 1 beat per minute for heart rate and less than 1% for oxygen saturation (SpO2).
From Classification to Clinical Insights: Towards Analyzing and Reasoning About Mobile and Behavioral Health Data With Large Language Models
Passively collected behavioral health data from ubiquitous sensors holds significant promise to provide mental health professionals insights from patient's daily lives; however, developing analysis tools to use this data in clinical practice requires addressing challenges of generalization across devices and weak or ambiguous correlations between the measured signals and an individual's mental health.
Improving Zero-shot Visual Question Answering via Large Language Models with Reasoning Question Prompts
This results in a set of candidate answers.
AbsPyramid: Benchmarking the Abstraction Ability of Language Models with a Unified Entailment Graph
Cognitive research indicates that abstraction ability is essential in human intelligence, which remains under-explored in language models.
Training Robust Deep Physiological Measurement Models with Synthetic Video-based Data
However, there exists a significant gap between synthetic and real-world data, which hinders the generalization of neural models trained on these synthetic datasets.
PrivLM-Bench: A Multi-level Privacy Evaluation Benchmark for Language Models
Lastly, PrivLM-Bench performs existing privacy attacks on LMs with pre-defined privacy objectives as the empirical evaluation results.
IBADR: an Iterative Bias-Aware Dataset Refinement Framework for Debiasing NLU models
Then, we pair each sample with a bias indicator representing its bias degree, and use these extended samples to train a sample generator.
Recaptured Raw Screen Image and Video Demoiréing via Channel and Spatial Modulations
The spatial modulation utilizes the feature with large receptive field to modulate the feature with small receptive field.
LitCab: Lightweight Language Model Calibration over Short- and Long-form Responses
For evaluation, we construct CaT, a benchmark consisting of eight text generation tasks, covering responses ranging from short phrases to paragraphs.
Gold: A Global and Local-aware Denoising Framework for Commonsense Knowledge Graph Noise Detection
Commonsense Knowledge Graphs (CSKGs) are crucial for commonsense reasoning, yet constructing them through human annotations can be costly.
QADYNAMICS: Training Dynamics-Driven Synthetic QA Diagnostic for Zero-Shot Commonsense Question Answering
Zero-shot commonsense Question-Answering (QA) requires models to reason about general situations beyond specific benchmarks.
On the Convergence of Federated Averaging under Partial Participation for Over-parameterized Neural Networks
Federated learning (FL) is a widely employed distributed paradigm for collaboratively training machine learning models from multiple clients without sharing local data.
Predicting Emergent Abilities with Infinite Resolution Evaluation
With PassUntil, we conduct a quantitative investigation into the scaling law of task performance.
ComSD: Balancing Behavioral Quality and Diversity in Unsupervised Skill Discovery
It contains a particle-based exploration reward to make agents access far-reaching states for exploratory skill acquisition, and a novel contrastive diversity reward to promote the discriminability between different skills.
Self-Consistent Narrative Prompts on Abductive Natural Language Inference
However, the inter-sentential coherence and the model consistency have not been well exploited in the previous works on this task.
Federated Linear Bandit Learning via Over-the-Air Computation
In this context, we propose a customized federated linear bandits scheme, where each device transmits an analog signal, and the server receives a superposition of these signals distorted by channel noise.
Video BagNet: short temporal receptive fields increase robustness in long-term action recognition
Previous work on long-term video action recognition relies on deep 3D-convolutional models that have a large temporal receptive field (RF).
Federated Reinforcement Learning for Electric Vehicles Charging Control on Distribution Networks
To effectively learn the optimal EV charging control strategy, a federated deep reinforcement learning algorithm named FedSAC is further proposed.
