Search Results for author:
Found 555 papers, 166 papers with code
A Structure-Aware Argument Encoder for Literature Discourse Analysis
Existing research for argument representation learning mainly treats tokens in the sentence equally and ignores the implied structure information of argumentative context.
EARL: Informative Knowledge-Grounded Conversation Generation with Entity-Agnostic Representation Learning
Generating informative and appropriate responses is challenging but important for building human-like dialogue systems.
Dual Refinement Underwater Object Detection Network
Due to the complex underwater environment, underwater imaging often encounters some problems such as blur, scale variation, color shift, and texture distortion.
Combinatorial Pure Exploration for Dueling Bandit
For Borda winner, we establish a reduction of the problem to the original CPE-MAB setting and design PAC and exact algorithms that achieve both the sample complexity similar to that in the CPE-MAB setting (which is nearly optimal for a subclass of problems) and polynomial running time per round.
The Sogou-TIIC Speech Translation System for IWSLT 2018
Our final average result on speech translation is 31. 02 BLEU.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+6
Uplink Assisted Joint Channel Estimation and CSI Feedback: An Approach Based on Deep Joint Source-Channel Coding
In frequency division duplex (FDD) multiple-input multiple-output (MIMO) wireless communication systems, the acquisition of downlink channel state information (CSI) is essential for maximizing spatial resource utilization and improving system spectral efficiency.
AutoStyle-TTS: Retrieval-Augmented Generation based Automatic Style Matching Text-to-Speech Synthesis
This scheme uses embeddings, extracted by Llama, PER-LLM-Embedder, and Moka, to match with samples in the knowledge database, selecting the most appropriate speech style for synthesis.
MiMu: Mitigating Multiple Shortcut Learning Behavior of Transformers
Then, we further design self-improvement strategy in target model to reduce the reliance on multiple shortcuts.
Hyperbolic Diffusion Recommender Model
To gain deeper insights into the limitations of diffusion models in recommender systems, we investigate the fundamental structural disparities between images and items.
Correlation-Attention Masked Temporal Transformer for User Identity Linkage Using Heterogeneous Mobility Data
With the rise of social media and Location-Based Social Networks (LBSN), check-in data across platforms has become crucial for User Identity Linkage (UIL).
Coeff-Tuning: A Graph Filter Subspace View for Tuning Attention-Based Large Models
In this paper, we propose to tune the large pre-trained transformers by learning a small set of combination coefficients that construct a more expressive filter subspace from the original multi-head attention maps.
Hierarchy-Aware and Channel-Adaptive Semantic Communication for Bandwidth-Limited Data Fusion
Obtaining high-resolution hyperspectral images (HR-HSI) is costly and data-intensive, making it necessary to fuse low-resolution hyperspectral images (LR-HSI) with high-resolution RGB images (HR-RGB) for practical applications.
Growing a Twig to Accelerate Large Vision-Language Models
Despite the success of these token pruning methods, they still suffer from two major shortcomings: (i) considerable accuracy drop due to insensitive attention signals in early layers, and (ii) limited speedup when generating long responses (e. g., 30 tokens).
Bridging Social Psychology and LLM Reasoning: Conflict-Aware Meta-Review Generation via Cognitive Alignment
The rapid growth of scholarly submissions has overwhelmed traditional peer review systems, driving the need for intelligent automation to preserve scientific rigor.
Step-Video-TI2V Technical Report: A State-of-the-Art Text-Driven Image-to-Video Generation Model
We present Step-Video-TI2V, a state-of-the-art text-driven image-to-video generation model with 30B parameters, capable of generating videos up to 102 frames based on both text and image inputs.
Semantic Latent Motion for Portrait Video Generation
Second, in the Reasoning step, long-term modeling and efficient reasoning are performed in this latent space to generate motion sequences.
R1-Onevision: Advancing Generalized Multimodal Reasoning through Cross-Modal Formalization
Existing visual-language models often struggle to effectively analyze and reason visual content, resulting in suboptimal performance on complex reasoning tasks.
Extra Clients at No Extra Cost: Overcome Data Heterogeneity in Federated Learning with Filter Decomposition
Data heterogeneity is one of the major challenges in federated learning (FL), which results in substantial client variance and slow convergence.
FedEM: A Privacy-Preserving Framework for Concurrent Utility Preservation in Federated Learning
Federated Learning (FL) enables collaborative training of models across distributed clients without sharing local data, addressing privacy concerns in decentralized systems.
Deep Joint CSI Estimation-Feedback-Precoding for MU-MIMO OFDM Systems
Furthermore, compared to conventional separate architecture, the proposed network architecture with joint architecture reduces the computational burden and model storage overhead at the UE side, facilitating the deployment of low-overhead multi-module joint architectures in practice.
GraphGarment: Learning Garment Dynamics for Bimanual Cloth Manipulation Tasks
In simulation experiments, GraphGarment achieves better garment state prediction performance, with a prediction error 0. 46 cm lower than the best baseline.
CLIP-Optimized Multimodal Image Enhancement via ISP-CNN Fusion for Coal Mine IoVT under Uneven Illumination
Clear monitoring images are crucial for the safe operation of coal mine Internet of Video Things (IoVT) systems.
LDGen: Enhancing Text-to-Image Synthesis via Large Language Model-Driven Language Representation
In this paper, we introduce LDGen, a novel method for integrating large language models (LLMs) into existing text-to-image diffusion models while minimizing computational demands.
CLIPure: Purification in Latent Space via CLIP for Adversarially Robust Zero-Shot Classification
We propose two variants for our CLIPure approach: CLIPure-Diff which models the likelihood of images' latent vectors with the DiffusionPrior module in DaLLE-2 (modeling the generation process of CLIP's latent vectors), and CLIPure-Cos which models the likelihood with the cosine similarity between the embeddings of an image and ``a photo of a.''.
A Deep Learning Framework with Geographic Information Adaptive Loss for Remote Sensing Images based UAV Self-Positioning
Some methods obtain geolocation information in GPS-denied environments by matching ground objects in the UAV viewpoint with remote sensing images.
Continuous K-Max Bandits
The continuous $K$-Max bandits introduce unique challenges, including discretization error from continuous-to-discrete conversion, non-deterministic tie-breaking under limited feedback, and biased estimation due to partial observability.
FedEAT: A Robustness Optimization Framework for Federated LLMs
Significant advancements have been made by Large Language Models (LLMs) in the domains of natural language understanding and automated content creation.
A Framework for Supervised and Unsupervised Segmentation and Classification of Materials Microstructure Images
Microstructure of materials is often characterized through image analysis to understand processing-structure-properties linkages.
NOMANet: A Graph Neural Network Enabled Power Allocation Scheme for NOMA
This paper proposes a graph neural network (GNN) enabled power allocation scheme for non-orthogonal multiple access (NOMA) networks.
Singing Voice Conversion with Accompaniment Using Self-Supervised Representation-Based Melody Features
Melody preservation is crucial in singing voice conversion (SVC).
Graph Neural Network Enabled Fluid Antenna Systems: A Two-Stage Approach
An emerging fluid antenna system (FAS) brings a new dimension, i. e., the antenna positions, to deal with the deep fading, but simultaneously introduces challenges related to the transmit design.
SWIPTNet: A Unified Deep Learning Framework for SWIPT based on GNN and Transfer Learning
The quality-of-service (QoS) constrained sum-rate maximization problems are, respectively, formulated for power-splitting (PS) receivers and time-switching (TS) receivers and solved by a unified graph neural network (GNN) based model termed SWIPT net (SWIPTNet).
Offline Learning for Combinatorial Multi-armed Bandits
The combinatorial multi-armed bandit (CMAB) is a fundamental sequential decision-making framework, extensively studied over the past decade.
InsQABench: Benchmarking Chinese Insurance Domain Question Answering with Large Language Models
The application of large language models (LLMs) has achieved remarkable success in various fields, but their effectiveness in specialized domains like the Chinese insurance industry remains underexplored.
Uncertainty-Aware Digital Twins: Robust Model Predictive Control using Time-Series Deep Quantile Learning
This proactive while uncertainty-aware control capability positions the proposed method as a potent tool for future Digital Twin applications and real-time process control in engineering systems.
What Limits LLM-based Human Simulation: LLMs or Our Design?
We argue that advancing LLM-based human simulation requires addressing both LLM's inherent limitations and simulation framework design challenges.
ACORD: An Expert-Annotated Retrieval Dataset for Legal Contract Drafting
Information retrieval, specifically contract clause retrieval, is foundational to contract drafting because lawyers rarely draft contracts from scratch; instead, they locate and revise the most relevant precedent.
Real-Time Decision-Making for Digital Twin in Additive Manufacturing with Model Predictive Control using Time-Series Deep Neural Networks
Using Directed Energy Deposition (DED) additive manufacturing as a case study, we demonstrate the effectiveness of the proposed MPC in achieving melt pool temperature tracking to ensure part quality, while reducing porosity defects by regulating laser power to maintain melt pool depth constraints.
COLOR: A compositional linear operation-based representation of protein sequences for identification of monomer contributions to properties
The properties of biological materials like proteins and nucleic acids are largely determined by their primary sequence.
Noise-Tolerant Hybrid Prototypical Learning with Noisy Web Data
On the other hand, the relation modeling between noisy and clean images is not learned for the class prototype generation in an end-to-end manner, which results in a suboptimal class prototype.
Make Domain Shift a Catastrophic Forgetting Alleviator in Class-Incremental Learning
This paper discovers a counter-intuitive observation: by incorporating domain shift into CIL tasks, the forgetting rate is significantly reduced.
OCRBench v2: An Improved Benchmark for Evaluating Large Multimodal Models on Visual Text Localization and Reasoning
Scoring the Optical Character Recognition (OCR) capabilities of Large Multimodal Models (LMMs) has witnessed growing interest recently.
Do Current Video LLMs Have Strong OCR Abilities? A Preliminary Study
With the rise of multimodal large language models, accurately extracting and understanding textual information from video content, referred to as video based optical character recognition (Video OCR), has become a crucial capability.
UniAvatar: Taming Lifelike Audio-Driven Talking Head Generation with Comprehensive Motion and Lighting Control
Recently, animating portrait images using audio input is a popular task.
Efficient Detection Framework Adaptation for Edge Computing: A Plug-and-play Neural Network Toolbox Enabling Edge Deployment
These results highlight ED-TOOLBOX as a superior solution for edge object detection.
Emerging Microelectronic Materials by Design: Navigating Combinatorial Design Space with Scarce and Dispersed Data
In this Account, we review a team effort toward establishing a framework that integrates data-driven and physics-based methods to address these challenges and accelerate materials design.
STKDRec: Spatial-Temporal Knowledge Distillation for Takeaway Recommendation
During the second STKD stage, a spatial-temporal Transformer is employed to comprehensively model dynamic user preferences on various types of fine-grained geospatial information from a sequence perspective.
CognTKE: A Cognitive Temporal Knowledge Extrapolation Framework
Reasoning future unknowable facts on temporal knowledge graphs (TKGs) is a challenging task, holding significant academic and practical values for various fields.
Security and Privacy of Digital Twins for Advanced Manufacturing: A Survey
In Industry 4. 0, the digital twin is one of the emerging technologies, offering simulation abilities to predict, refine, and interpret conditions and operations, where it is crucial to emphasize a heightened concentration on the associated security and privacy risks.
OmniVLM: A Token-Compressed, Sub-Billion-Parameter Vision-Language Model for Efficient On-Device Inference
We present OmniVLM, a sub-billion-parameter vision-language model for efficient on-device inference.
LLMs Can Simulate Standardized Patients via Agent Coevolution
Training medical personnel using standardized patients (SPs) remains a complex challenge, requiring extensive domain expertise and role-specific practice.
Smoothness Really Matters: A Simple Yet Effective Approach for Unsupervised Graph Domain Adaptation
Given the sensitivity of GNNs to local structural features, even slight discrepancies between source and target graphs could lead to significant shifts in node embeddings, thereby reducing the effectiveness of knowledge transfer.
Asynchronous Random Access in Massive MIMO Systems Facilitated by the Delay-Angle Domain
This paper explores contention-based random access (CBRA) schemes for asynchronous access in massive multiple-input multiple-output (MIMO) systems.
DataLab: A Unified Platform for LLM-Powered Business Intelligence
To achieve this unification, we design a domain knowledge incorporation module tailored for enterprise-specific BI tasks, an inter-agent communication mechanism to facilitate information sharing across the BI workflow, and a cell-based context management strategy to enhance context utilization efficiency in BI notebooks.
Combinatorial Rising Bandit
However, we provide the sub-linear regret lower bound for combinatorial rising bandit and show that CRUCB is provably efficient by showing that the regret upper bound is close to the regret lower bound.
Learning Rate-Compatible Linear Block Codes: An Auto-Encoder Based Approach
The coding process associated with AI or non-AI decoders and multiple puncturing patterns is optimized in a data-driven manner.
Cross Space and Time: A Spatio-Temporal Unitized Model for Traffic Flow Forecasting
In this paper, we introduce the Spatio-Temporal Unitized Model (STUM), a unified framework designed to capture both spatial and temporal dependencies while addressing spatio-temporal heterogeneity through techniques such as distribution alignment and feature fusion.
FedDTPT: Federated Discrete and Transferable Prompt Tuning for Black-Box Large Language Models
To address these challenges, we propose for the first time a federated discrete and transferable prompt tuning, namely FedDTPT, for black-box large language models.
Fast and scalable Wasserstein-1 neural optimal transport solver for single-cell perturbation prediction
Our experiments demonstrate that the proposed $W_1$ neural optimal transport solver can mimic the $W_2$ OT solvers in finding a unique and ``monotonic" map on 2D datasets.
Beyond Content Relevance: Evaluating Instruction Following in Retrieval Models
Instruction-following capabilities in LLMs have progressed significantly, enabling more complex user interactions through detailed prompts.
RSL-SQL: Robust Schema Linking in Text-to-SQL Generation
Text-to-SQL generation aims to translate natural language questions into SQL statements.
CausalDiff: Causality-Inspired Disentanglement via Diffusion Model for Adversarial Defense
Despite ongoing efforts to defend neural classifiers from adversarial attacks, they remain vulnerable, especially to unseen attacks.
KANsformer for Scalable Beamforming
This paper proposes an unsupervised deep-learning (DL) approach by integrating transformer and Kolmogorov-Arnold networks (KAN) termed KANsformer to realize scalable beamforming for mobile communication systems.
FairDgcl: Fairness-aware Recommendation with Dynamic Graph Contrastive Learning
Specifically, we propose FairDgcl, a dynamic graph adversarial contrastive learning framework aiming at improving fairness in recommender system.
Expand and Compress: Exploring Tuning Principles for Continual Spatio-Temporal Graph Forecasting
The widespread deployment of sensing devices leads to a surge in data for spatio-temporal forecasting applications such as traffic flow, air quality, and wind energy.
TV-3DG: Mastering Text-to-3D Customized Generation with Visual Prompt
To address this, we propose a novel algorithm, Classifier Score Matching (CSM), which removes the difference term in SDS and uses a deterministic noise addition process to reduce noise during optimization, effectively overcoming the low-quality limitations of SDS in our customized generation framework.
Learning to Customize Text-to-Image Diffusion In Diverse Context
This often results in the model becoming overfitted to these training images and unable to generalize to new contexts in future text prompts.
ChartKG: A Knowledge-Graph-Based Representation for Chart Images
We present four cases to illustrate how our knowledge-graph-based representation can model the detailed visual elements and semantic relations in charts, and further demonstrate how our approach can benefit downstream applications such as semantic-aware chart retrieval and chart question answering.
PDF-WuKong: A Large Multimodal Model for Efficient Long PDF Reading with End-to-End Sparse Sampling
In this paper, we introduce PDF-WuKong, a multimodal large language model (MLLM) which is designed to enhance multimodal question-answering (QA) for long PDF documents.
LoTLIP: Improving Language-Image Pre-training for Long Text Understanding
Then, after incorporating corner tokens to aggregate diverse textual information, we manage to help the model catch up to its original level of short text understanding yet greatly enhance its capability of long text understanding.
MC-CoT: A Modular Collaborative CoT Framework for Zero-shot Medical-VQA with LLM and MLLM Integration
In recent advancements, multimodal large language models (MLLMs) have been fine-tuned on specific medical image datasets to address medical visual question answering (Med-VQA) tasks.
Structural-Entropy-Based Sample Selection for Efficient and Effective Learning
Based on the decomposition, we present $\textbf{S}$tructural-$\textbf{E}$ntropy-based sample $\textbf{S}$election ($\textbf{SES}$), a method that integrates both global and local information to select informative and representative samples.
GNN-Enabled Optimization of Placement and Transmission Design for UAV Communications
This paper applies graph neural networks (GNN) in UAV communications to optimize the placement and transmission design.
Model-Based GNN Enabled Energy-Efficient Beamforming for Ultra-Dense Wireless Networks
This paper investigates deep learning enabled beamforming design for ultra-dense wireless networks by integrating prior knowledge and graph neural network (GNN), named model-based GNN.
Generative Retrieval Meets Multi-Graded Relevance
GR$^2$ focuses on two key components: ensuring relevant and distinct identifiers, and implementing multi-graded constrained contrastive training.
FaceVid-1K: A Large-Scale High-Quality Multiracial Human Face Video Dataset
We obtain the corresponding performance benchmarks and compared them with those trained on public datasets to demonstrate the superiority of our dataset.
From Yes-Men to Truth-Tellers: Addressing Sycophancy in Large Language Models with Pinpoint Tuning
Recent works propose to employ supervised fine-tuning (SFT) to mitigate the sycophancy issue, while it typically leads to the degeneration of LLMs' general capability.
Generic Objects as Pose Probes for Few-shot View Synthesis
COLMAP is frequently employed for preprocessing to estimate poses, while it necessitates a large number of feature matches to operate effectively, and it struggles with scenes characterized by sparse features, large baselines between images, or a limited number of input images.
Do Graph Neural Networks Work for High Entropy Alloys?
Graph neural networks (GNNs) have excelled in predictive modeling for both crystals and molecules, owing to the expressiveness of graph representations.
GRPose: Learning Graph Relations for Human Image Generation with Pose Priors
In this paper, we propose a framework that delves into the graph relations of pose priors to provide control information for human image generation.
Squid: Long Context as a New Modality for Energy-Efficient On-Device Language Models
Our work contributes to the development of more sustainable and scalable language models for on-device applications, addressing the critical need for energy-efficient and responsive AI technologies in resource-constrained environments while maintaining the accuracy to understand long contexts.
On-Device Language Models: A Comprehensive Review
For a comprehensive review of research work and educational resources on on-device large language models (LLMs), please visit https://github. com/NexaAI/Awesome-LLMs-on-device.
Histology Virtual Staining with Mask-Guided Adversarial Transfer Learning for Tertiary Lymphoid Structure Detection
Capitalizing on the prevalence of H&E staining slides, we introduce a novel Mask-Guided Adversarial Transfer Learning method designed for virtual pathological staining.
FlexEdit: Marrying Free-Shape Masks to VLLM for Flexible Image Editing
Our approach employs a VLLM in comprehending the image content, mask, and user instructions.
PartGS:Learning Part-aware 3D Representations by Fusing 2D Gaussians and Superquadrics
In this paper, we introduce $\textbf{PartGS}$, $\textbf{part}$-aware 3D reconstruction by a hybrid representation of 2D $\textbf{G}$aussians and $\textbf{S}$uperquadrics, which parses objects or scenes into semantic parts, digging 3D structural clues from multi-view image inputs.
Navigating Spatio-Temporal Heterogeneity: A Graph Transformer Approach for Traffic Forecasting
To tackle these challenges, we introduce the Spatio-Temporal Graph Transformer (STGormer), which effectively integrates attribute and structure information inherent in traffic data for learning spatio-temporal correlations, and a mixture-of-experts module for capturing heterogeneity along spaital and temporal axes.
MePT: Multi-Representation Guided Prompt Tuning for Vision-Language Model
Recent advancements in pre-trained Vision-Language Models (VLMs) have highlighted the significant potential of prompt tuning for adapting these models to a wide range of downstream tasks.
Towards Boosting LLMs-driven Relevance Modeling with Progressive Retrieved Behavior-augmented Prompting
In response, we propose ProRBP, a novel Progressive Retrieved Behavior-augmented Prompting framework for integrating search scenario-oriented knowledge with LLMs effectively.
Classifier Guidance Enhances Diffusion-based Adversarial Purification by Preserving Predictive Information
Adversarial purification is one of the promising approaches to defend neural networks against adversarial attacks.
A Digital Twin Framework Utilizing Machine Learning for Robust Predictive Maintenance: Enhancing Tire Health Monitoring
We introduce a novel digital twin framework for predictive maintenance of long-term physical systems.
Impacts of Darwinian Evolution on Pre-trained Deep Neural Networks
The empirical results show that the proposed framework has positive impacts on the network, with reduced over-fitting and an order of magnitude lower time complexity compared to BP.
An Efficient and Effective Transformer Decoder-Based Framework for Multi-Task Visual Grounding
In this paper, we propose an efficient and effective multi-task visual grounding (EEVG) framework based on Transformer Decoder to address this issue, which reduces the cost in both language and visual aspects.
A Backbone for Long-Horizon Robot Task Understanding
In the online testing stage, after a one-shot demonstration of a new task is collected, our MGSF network extracts high-level knowledge, which is then encoded into the image using Action Registration (ActionREG).
Paying More Attention to Image: A Training-Free Method for Alleviating Hallucination in LVLMs
However, the scale disparity between vision encoder and language model may led to LLMs assuming a predominant role in multi-modal comprehension.
Diffusion-Driven Semantic Communication for Generative Models with Bandwidth Constraints
This paper introduces a diffusion-driven semantic communication framework with advanced VAE-based compression for bandwidth-constrained generative model.
Deep learning for predicting the occurrence of tipping points
The ability to predict the occurrence of tipping points from time series data remains an outstanding challenge and a major interest in a broad range of research fields.
Theoretical Analysis of Privacy Leakage in Trustworthy Federated Learning: A Perspective from Linear Algebra and Optimization Theory
From the optimization theory perspective, we establish an upper bound on the privacy leakage in terms of the batch size, the distortion extent, and several other factors.
Norface: Improving Facial Expression Analysis by Identity Normalization
First, the carefully designed normalization network struggles to directly remove the above task-irrelevant noise, by maintaining facial expression consistency but normalizing all original images to a common identity with consistent pose, and background.
Ranked #1 on
Facial Expression Recognition (FER)
on DISFA
Real-Time 3D Occupancy Prediction via Geometric-Semantic Disentanglement
In the BEV branch, a BEV-level temporal fusion module and a U-Net encoder is introduced to extract dense semantic features.
Heterogenous Multi-Source Data Fusion Through Input Mapping and Latent Variable Gaussian Process
In the second stage, a multi-source data fusion model enabled by LVGP is leveraged to build a single source-aware surrogate model on the transformed reference space.
Synergistic Multi-Agent Framework with Trajectory Learning for Knowledge-Intensive Tasks
Recent advancements in Large Language Models (LLMs) have led to significant breakthroughs in various natural language processing tasks.
One-Shot Pose-Driving Face Animation Platform
The objective of face animation is to generate dynamic and expressive talking head videos from a single reference face, utilizing driving conditions derived from either video or audio inputs.
Tissue-Contrastive Semi-Masked Autoencoders for Segmentation Pretraining on Chest CT
Existing Masked Image Modeling (MIM) depends on a spatial patch-based masking-reconstruction strategy to perceive objects'features from unlabeled images, which may face two limitations when applied to chest CT: 1) inefficient feature learning due to complex anatomical details presented in CT images, and 2) suboptimal knowledge transfer owing to input disparity between upstream and downstream models.
Entropy-Informed Weighting Channel Normalizing Flow
Normalizing Flows (NFs) have gained popularity among deep generative models due to their ability to provide exact likelihood estimation and efficient sampling.
A Unified Learn-to-Distort-Data Framework for Privacy-Utility Trade-off in Trustworthy Federated Learning
In this paper, we first give an introduction to the theoretical basis of the privacy-utility equilibrium in federated learning based on Bayesian privacy definitions and total variation distance privacy definitions.
Seed-ASR: Understanding Diverse Speech and Contexts with LLM-based Speech Recognition
Modern automatic speech recognition (ASR) model is required to accurately transcribe diverse speech signals (from different domains, languages, accents, etc) given the specific contextual information in various application scenarios.
Ranked #2 on
Speech Recognition
on AISHELL-1
(using extra training data)
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+4
AI-Driven Mobility Management for High-Speed Railway Communications: Compressed Measurements and Proactive Handover
Compared with the traditional event A3-based handover mechanism, the proposed approach significantly reduces the RLF rates which saves 50% beam measurement overhead.
CURLS: Causal Rule Learning for Subgroups with Significant Treatment Effect
In this paper, we propose CURLS, a novel rule learning method leveraging HTE, which can effectively describe subgroups with significant treatment effects.
Octo-planner: On-device Language Model for Planner-Action Agents
In this paper, we present an efficient on-device Planner-Action framework that separates planning and action execution into two distinct components: a planner agent based on Phi-3 Mini, a 3. 8 billion parameter LLM optimized for edge devices, and an action agent using the Octopus model for function execution.
Evaluating Implicit Bias in Large Language Models by Attacking From a Psychometric Perspective
As Large Language Models (LLMs) become an important way of information seeking, there have been increasing concerns about the unethical content LLMs may generate.
Distributed Stochastic Gradient Descent with Staleness: A Stochastic Delay Differential Equation Based Framework
It is interestingly shown that increasing the number of activated workers does not necessarily accelerate distributed SGD due to staleness.
CoMM: A Coherent Interleaved Image-Text Dataset for Multimodal Understanding and Generation
To address this gap, we introduce CoMM, a high-quality Coherent interleaved image-text MultiModal dataset designed to enhance the coherence, consistency, and alignment of generated multimodal content.
Technique Report of CVPR 2024 PBDL Challenges
The intersection of physics-based vision and deep learning presents an exciting frontier for advancing computer vision technologies.
Common and Rare Fundus Diseases Identification Using Vision-Language Foundation Model with Knowledge of Over 400 Diseases
Here we introduce RetiZero, a vision-language foundation model that leverages knowledge from over 400 fundus diseases.
MobileAgentBench: An Efficient and User-Friendly Benchmark for Mobile LLM Agents
Large language model (LLM)-based mobile agents are increasingly popular due to their capability to interact directly with mobile phone Graphic User Interfaces (GUIs) and their potential to autonomously manage daily tasks.
CLDTA: Contrastive Learning based on Diagonal Transformer Autoencoder for Cross-Dataset EEG Emotion Recognition
Recent advances in non-invasive EEG technology have broadened its application in emotion recognition, yielding a multitude of related datasets.
VulDetectBench: Evaluating the Deep Capability of Vulnerability Detection with Large Language Models
However, sound comprehensive research on detecting program vulnerabilities, a more specific task related to code, and evaluating the performance of LLMs in this more specialized scenario is still lacking.
Global Parameterization-based Texture Space Optimization
Experiments show the effectiveness of the proposed method and the potency in improving the storage and rendering efficiency.
A Combination Model for Time Series Prediction using LSTM via Extracting Dynamic Features Based on Spatial Smoothing and Sequential General Variational Mode Decomposition
In order to solve the problems such as difficult to extract effective features and low accuracy of sales volume prediction caused by complex relationships such as market sales volume in time series prediction, we proposed a time series prediction method of market sales volume based on Sequential General VMD and spatial smoothing Long short-term memory neural network (SS-LSTM) combination model.
A Combination Model Based on Sequential General Variational Mode Decomposition Method for Time Series Prediction
Within the prediction interval, our proposed combination model has improved advantages over traditional decomposition prediction control group models.
Combinatorial Multivariant Multi-Armed Bandits with Applications to Episodic Reinforcement Learning and Beyond
We introduce a novel framework of combinatorial multi-armed bandits (CMAB) with multivariant and probabilistically triggering arms (CMAB-MT), where the outcome of each arm is a $d$-dimensional multivariant random variable and the feedback follows a general arm triggering process.
No Free Lunch Theorem for Privacy-Preserving LLM Inference
Therefore, it is essential to evaluate the balance between the risk of privacy leakage and loss of utility when conducting effective protection mechanisms.
Identifying Functional Brain Networks of Spatiotemporal Wide-Field Calcium Imaging Data via a Long Short-Term Memory Autoencoder
The goal of this study is to elucidate and illustrate, qualitatively and quantitatively, the FBNs identified by use of the LSTM-AER method and compare them to those from traditional SBC and ICA.
Can Graph Learning Improve Planning in LLM-based Agents?
This theoretical insight led us to integrate GNNs with LLMs to enhance overall performance.
Fully Exploiting Every Real Sample: SuperPixel Sample Gradient Model Stealing
With the basic idea of imitating the victim model's low-variance patch-level gradients instead of pixel-level gradients, SPSG achieves efficient sample gradient estimation through two steps.
VideoQA-SC: Adaptive Semantic Communication for Video Question Answering
The difficulty in such system design lies in the extraction of task-related compact semantic representations and their accurate delivery over noisy channels.
Feature-based Low-Rank Compression of Large Language Models via Bayesian Optimization
To address the challenges of low-rank compression in LLMs, we conduct empirical research on the low-rank characteristics of large models.
ALPINE: Unveiling the Planning Capability of Autoregressive Learning in Language Models
Furthermore, our theoretical analysis of gradient-based learning dynamics reveals that LLMs can learn both the adjacency and a limited form of the reachability matrices.
Deep Learning for CSI Feedback: One-Sided Model and Joint Multi-Module Learning Perspectives
The use of deep learning (DL) for channel state information (CSI) feedback has garnered widespread attention across academia and industry.
ProbRadarM3F: mmWave Radar based Human Skeletal Pose Estimation with Probability Map Guided Multi-Format Feature Fusion
However, mmWave radar relies on the collection of reflected signals from the target, and the radar signals containing information is difficult to be fully applied.
Octopus v4: Graph of language models
Additionally, we explore the use of graph as a versatile data structure that effectively coordinates multiple open-source models by harnessing the capabilities of the Octopus model and \textit{functional tokens}.
Beyond ESM2: Graph-Enhanced Protein Sequence Modeling with Efficient Clustering
Proteins are essential to life's processes, underpinning evolution and diversity.
ControlTraj: Controllable Trajectory Generation with Topology-Constrained Diffusion Model
Generating trajectory data is among promising solutions to addressing privacy concerns, collection costs, and proprietary restrictions usually associated with human mobility analyses.
Towards Robust Trajectory Representations: Isolating Environmental Confounders with Causal Learning
Trajectory modeling refers to characterizing human movement behavior, serving as a pivotal step in understanding mobility patterns.
decoupleQ: Towards 2-bit Post-Training Uniform Quantization via decoupling Parameters into Integer and Floating Points
However, existing quantization schemes suffer from significant accuracy degradation at very low bits, or require some additional computational overhead when deployed, making it difficult to be applied to large-scale applications in industry.
Adaptive Catalyst Discovery Using Multicriteria Bayesian Optimization with Representation Learning
The results demonstrate that our approach achieves high prediction accuracy, facilitates interpretable feature extraction, and enables multicriteria design optimization, leading to significant reduction of computing power and time (10x reduction of required DFT calculations) in high-performance catalyst discovery.
Graph Neural Networks for Wireless Networks: Graph Representation, Architecture and Evaluation
Graph neural networks (GNNs) have been regarded as the basic model to facilitate deep learning (DL) to revolutionize resource allocation in wireless networks.
Octopus v3: Technical Report for On-device Sub-billion Multimodal AI Agent
A multimodal AI agent is characterized by its ability to process and learn from various types of data, including natural language, visual, and audio inputs, to inform its actions.
Interpolating neural network: A novel unification of machine learning and interpolation theory
Artificial intelligence (AI) has revolutionized software development, shifting from task-specific codes (Software 1. 0) to neural network-based approaches (Software 2. 0).
Building Semantic Communication System via Molecules: An End-to-End Training Approach
Furthermore, we propose a channel network to enable the E2E learning over the non-differentiable molecular channel.
Bridging Data Islands: Geographic Heterogeneity-Aware Federated Learning for Collaborative Remote Sensing Semantic Segmentation
Our framework consists of three modules, including the Global Insight Enhancement (GIE) module, the Essential Feature Mining (EFM) module and the Local-Global Balance (LoGo) module.
JailbreakLens: Visual Analysis of Jailbreak Attacks Against Large Language Models
Addressing these concerns necessitates a comprehensive analysis of jailbreak prompts to evaluate LLMs' defensive capabilities and identify potential weaknesses.
Low-rank Adaptation for Spatio-Temporal Forecasting
Spatio-temporal forecasting is crucial in real-world dynamic systems, predicting future changes using historical data from diverse locations.
Simulation-Free Determination of Microstructure Representative Volume Element Size via Fisher Scores
We propose a simulation-free alternative that determines RVE size based only on a micrograph.
Octopus: On-device language model for function calling of software APIs
In the rapidly evolving domain of artificial intelligence, Large Language Models (LLMs) play a crucial role due to their advanced text processing and generation abilities.
Octopus v2: On-device language model for super agent
Current on-device models for function calling face issues with latency and accuracy.
A Correction of Pseudo Log-Likelihood Method
Pseudo log-likelihood is a type of maximum likelihood estimation (MLE) method used in various fields including contextual bandits, influence maximization of social networks, and causal bandits.
Invisible Gas Detection: An RGB-Thermal Cross Attention Network and A New Benchmark
The widespread use of various chemical gases in industrial processes necessitates effective measures to prevent their leakage during transportation and storage, given their high toxicity.
UrbanVLP: Multi-Granularity Vision-Language Pretraining for Urban Socioeconomic Indicator Prediction
Our UrbanVLP seamlessly integrates multi-granularity information from both macro (satellite) and micro (street-view) levels, overcoming the limitations of prior pretrained models.
DreamLIP: Language-Image Pre-training with Long Captions
Motivated by this, we propose to dynamically sample sub-captions from the text label to construct multiple positive pairs, and introduce a grouping loss to match the embeddings of each sub-caption with its corresponding local image patches in a self-supervised manner.
Deciphering the Interplay between Local Differential Privacy, Average Bayesian Privacy, and Maximum Bayesian Privacy
The swift evolution of machine learning has led to emergence of various definitions of privacy due to the threats it poses to privacy, including the concept of local differential privacy (LDP).
Heterogeneous Federated Learning with Splited Language Model
Federated Split Learning (FSL) is a promising distributed learning paradigm in practice, which gathers the strengths of both Federated Learning (FL) and Split Learning (SL) paradigms, to ensure model privacy while diminishing the resource overhead of each client, especially on large transformer models in a resource-constrained environment, e. g., Internet of Things (IoT).
On the Convergence of Adam under Non-uniform Smoothness: Separability from SGDM and Beyond
This paper aims to clearly distinguish between Stochastic Gradient Descent with Momentum (SGDM) and Adam in terms of their convergence rates.
Deep Learning for Trajectory Data Management and Mining: A Survey and Beyond
In this paper, we present a comprehensive review of the development and recent advances in deep learning for trajectory computing (DL4Traj).
Listwise Generative Retrieval Models via a Sequential Learning Process
Specifically, we view the generation of a ranked docid list as a sequence learning process: at each step we learn a subset of parameters that maximizes the corresponding generation likelihood of the $i$-th docid given the (preceding) top $i-1$ docids.
Hyper-3DG: Text-to-3D Gaussian Generation via Hypergraph
Our framework is anchored by a well-established mainflow and an essential module, named ``Geometry and Texture Hypergraph Refiner (HGRefiner)''.
CarbonNet: How Computer Vision Plays a Role in Climate Change? Application: Learning Geomechanics from Subsurface Geometry of CCS to Mitigate Global Warming
We introduce a new approach using computer vision to predict the land surface displacement from subsurface geometry images for Carbon Capture and Sequestration (CCS).
Gemini 1.5: Unlocking multimodal understanding across millions of tokens of context
In this report, we introduce the Gemini 1. 5 family of models, representing the next generation of highly compute-efficient multimodal models capable of recalling and reasoning over fine-grained information from millions of tokens of context, including multiple long documents and hours of video and audio.
Large Convolutional Model Tuning via Filter Subspace
Furthermore, each filter atom can be recursively decomposed as a combination of another set of atoms, which naturally expands the number of tunable parameters in the filter subspace.
Towards a Digital Twin Framework in Additive Manufacturing: Machine Learning and Bayesian Optimization for Time Series Process Optimization
Our work presents a digital twin (DT) framework for real-time predictive control of DED process parameters to meet specific design objectives.
Graph Diffusion Policy Optimization
Recent research has made significant progress in optimizing diffusion models for downstream objectives, which is an important pursuit in fields such as graph generation for drug design.
Self-Distillation Bridges Distribution Gap in Language Model Fine-Tuning
The surge in Large Language Models (LLMs) has revolutionized natural language processing, but fine-tuning them for specific tasks often encounters challenges in balancing performance and preserving general instruction-following abilities.
Attractor Memory for Long-Term Time Series Forecasting: A Chaos Perspective
In long-term time series forecasting (LTSF) tasks, an increasing number of models have acknowledged that discrete time series originate from continuous dynamic systems and have attempted to model their dynamical structures.
AI Hospital: Benchmarking Large Language Models in a Multi-agent Medical Interaction Simulator
Artificial intelligence has significantly advanced healthcare, particularly through large language models (LLMs) that excel in medical question answering benchmarks.
Model Compression and Efficient Inference for Large Language Models: A Survey
However, Large language models have two prominent characteristics compared to smaller models: (1) Most of compression algorithms require finetuning or even retraining the model after compression.
Prismatic: Interactive Multi-View Cluster Analysis of Concept Stocks
Financial cluster analysis allows investors to discover investment alternatives and avoid undertaking excessive risks.
AgentLens: Visual Analysis for Agent Behaviors in LLM-based Autonomous Systems
Recently, Large Language Model based Autonomous system(LLMAS) has gained great popularity for its potential to simulate complicated behaviors of human societies.
A Unified Causal View of Instruction Tuning
The key idea is to learn task-required causal factors and only use those to make predictions for a given task.
FedAA: A Reinforcement Learning Perspective on Adaptive Aggregation for Fair and Robust Federated Learning
Federated Learning (FL) has emerged as a promising approach for privacy-preserving model training across decentralized devices.
Learning by Doing: An Online Causal Reinforcement Learning Framework with Causal-Aware Policy
As a key component to intuitive cognition and reasoning solutions in human intelligence, causal knowledge provides great potential for reinforcement learning (RL) agents' interpretability towards decision-making by helping reduce the searching space.
Interpretable Multi-Source Data Fusion Through Latent Variable Gaussian Process
Additionally, a dissimilarity metric based on the latent variables of LVGP is introduced to study and understand the differences in the sources of data.
Position: What Can Large Language Models Tell Us about Time Series Analysis
Time series analysis is essential for comprehending the complexities inherent in various realworld systems and applications.
Delving into Multi-modal Multi-task Foundation Models for Road Scene Understanding: From Learning Paradigm Perspectives
Foundation models have indeed made a profound impact on various fields, emerging as pivotal components that significantly shape the capabilities of intelligent systems.
EasyFS: an Efficient Model-free Feature Selection Framework via Elastic Transformation of Features
Traditional model-free feature selection methods treat each feature independently while disregarding the interrelationships among features, which leads to relatively poor performance compared with the model-aware methods.
EMN: Brain-inspired Elastic Memory Network for Quick Domain Adaptive Feature Mapping
Inspired by the memory mechanism and powerful generalization ability of biological neural networks in human brains, we propose a novel gradient-free Elastic Memory Network, namely EMN, to support quick fine-tuning of the mapping between features and prediction without heavy optimization of deep features.
Optimize Individualized Energy Delivery for Septic Patients Using Predictive Deep Learning Models: A Real World Study
Excessive energy intake increased mortality rapidly in the early period of the acute phase.
Shapelet-based Model-agnostic Counterfactual Local Explanations for Time Series Classification
In this work, we propose a model-agnostic instance-based post-hoc explainability method for time series classification.
Query-Efficient Correlation Clustering with Noisy Oracle
We study a general clustering setting in which we have $n$ elements to be clustered, and we aim to perform as few queries as possible to an oracle that returns a noisy sample of the weighted similarity between two elements.
CauESC: A Causal Aware Model for Emotional Support Conversation
Emotional Support Conversation aims at reducing the seeker's emotional distress through supportive response.
Synthetic data enables faster annotation and robust segmentation for multi-object grasping in clutter
In this work, we propose a synthetic data generation method that minimizes human intervention and makes downstream image segmentation algorithms more robust by combining a generated synthetic dataset with a smaller real-world dataset (hybrid dataset).
Cascading Reinforcement Learning
In the cascading bandit model, at each timestep, an agent recommends an ordered subset of items (called an item list) from a pool of items, each associated with an unknown attraction probability.
Attention-Based CNN-BiLSTM for Sleep State Classification of Spatiotemporal Wide-Field Calcium Imaging Data
Comparison with Existing Method: On a 3-hour WFCI recording, the CNN-BiLSTM achieved a kappa of 0. 67, comparable to a kappa of 0. 65 corresponding to the human EEG/EMG-based scoring.
The NPU-ASLP-LiAuto System Description for Visual Speech Recognition in CNVSRC 2023
This paper delineates the visual speech recognition (VSR) system introduced by the NPU-ASLP-LiAuto (Team 237) in the first Chinese Continuous Visual Speech Recognition Challenge (CNVSRC) 2023, engaging in the fixed and open tracks of Single-Speaker VSR Task, and the open track of Multi-Speaker VSR Task.
ICMC-ASR: The ICASSP 2024 In-Car Multi-Channel Automatic Speech Recognition Challenge
To promote speech processing and recognition research in driving scenarios, we build on the success of the Intelligent Cockpit Speech Recognition Challenge (ICSRC) held at ISCSLP 2022 and launch the ICASSP 2024 In-Car Multi-Channel Automatic Speech Recognition (ICMC-ASR) Challenge.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+3
Large Language Models for Generative Information Extraction: A Survey
Information extraction (IE) aims to extract structural knowledge from plain natural language texts.
MolSets: Molecular Graph Deep Sets Learning for Mixture Property Modeling
This complexity is particularly evident in molecular mixtures, a frequently explored space for materials such as battery electrolytes.
TagAlign: Improving Vision-Language Alignment with Multi-Tag Classification
The crux of learning vision-language models is to extract semantically aligned information from visual and linguistic data.
Identification of Causal Structure with Latent Variables Based on Higher Order Cumulants
Despite the emergence of numerous methods aimed at addressing this challenge, they are not fully identified to the structure that two observed variables are influenced by one latent variable and there might be a directed edge in between.
Perturbation-Invariant Adversarial Training for Neural Ranking Models: Improving the Effectiveness-Robustness Trade-Off
We decompose the robust ranking error into two components, i. e., a natural ranking error for effectiveness evaluation and a boundary ranking error for assessing adversarial robustness.
K-ESConv: Knowledge Injection for Emotional Support Dialogue Systems via Prompt Learning
Automatic psychological counseling requires mass of professional knowledge that can be found in online counseling forums.
Enlighten-Your-Voice: When Multimodal Meets Zero-shot Low-light Image Enhancement
Low-light image enhancement is a crucial visual task, and many unsupervised methods tend to overlook the degradation of visible information in low-light scenes, which adversely affects the fusion of complementary information and hinders the generation of satisfactory results.
Generative Inverse Design of Metamaterials with Functional Responses by Interpretable Learning
Unlike data-intensive and non-interpretable deep-learning-based methods, we propose the Random-forest-based Interpretable Generative Inverse Design (RIGID), a single-shot inverse design method for fast generation of metamaterial designs with on-demand functional behaviors.
Local-Global History-aware Contrastive Learning for Temporal Knowledge Graph Reasoning
Temporal knowledge graphs (TKGs) have been identified as a promising approach to represent the dynamics of facts along the timeline.
A Cyclic Small Phase Theorem
This paper introduces a brand-new phase definition called the segmental phase for multi-input multi-output linear time-invariant systems.
Phase Preservation of N-Port Networks under General Connections
In addition, the inverse operations of the considered connections, that is, network subtractions with correspondences are examined.
Statistical Parameterized Physics-Based Machine Learning Digital Twin Models for Laser Powder Bed Fusion Process
In the context of laser powder bed fusion (LPBF) additive manufacturing, a digital twin of the manufacturing process can offer predictions for the produced parts, diagnostics for manufacturing defects, as well as control capabilities.
CAME: Competitively Learning a Mixture-of-Experts Model for First-stage Retrieval
The first-stage retrieval aims to retrieve a subset of candidate documents from a huge collection both effectively and efficiently.
Closing the Gap Between the Upper Bound and the Lower Bound of Adam's Iteration Complexity
Recently, Arjevani et al. [1] established a lower bound of iteration complexity for the first-order optimization under an $L$-smooth condition and a bounded noise variance assumption.
DISC-FinLLM: A Chinese Financial Large Language Model based on Multiple Experts Fine-tuning
We propose Multiple Experts Fine-tuning Framework to build a financial large language model (LLM), DISC-FinLLM.
