Search Results for author:
Found 88 papers, 32 papers with code
DialogueEIN: Emotion Interaction Network for Dialogue Affective Analysis
Emotion Recognition in Conversation (ERC) has attracted increasing attention in the affective computing research field.
InverTune: Removing Backdoors from Multimodal Contrastive Learning Models via Trigger Inversion and Activation Tuning
Multimodal contrastive learning models like CLIP have demonstrated remarkable vision-language alignment capabilities, yet their vulnerability to backdoor attacks poses critical security risks.
ChromFound: Towards A Universal Foundation Model for Single-Cell Chromatin Accessibility Data
The advent of single-cell Assay for Transposase-Accessible Chromatin using sequencing (scATAC-seq) offers an innovative perspective for deciphering regulatory mechanisms by assembling a vast repository of single-cell chromatin accessibility data.
Rethinking Repetition Problems of LLMs in Code Generation
Specifically, RPG first leverages grammar rules to identify repetition problems during code generation, and then strategically decays the likelihood of critical tokens that contribute to repetitions, thereby mitigating them in code generation.
Collection: Datasets from AFAR Challenge
For each team, the UGV was placed at three different positions, resulting in a total of 30 datasets, 15 collected in a DT simulation environment and 15 in a physical outdoor testbed.
ZipIR: Latent Pyramid Diffusion Transformer for High-Resolution Image Restoration
Recent progress in generative models has significantly improved image restoration capabilities, particularly through powerful diffusion models that offer remarkable recovery of semantic details and local fidelity.
Context-Aware Human Behavior Prediction Using Multimodal Large Language Models: Challenges and Insights
In this paper, we present a systematic analysis of applying pre-trained MLLMs for context-aware human behavior prediction.
Generating, Fast and Slow: Scalable Parallel Video Generation with Video Interface Networks
At each diffusion step, VINs encode global semantics from the noisy input of local chunks and the encoded representations, in turn, guide DiTs in denoising chunks in parallel.
Movable Cell-Free Massive MIMO For High-Speed Train Communications: A PPO-Based Antenna Position Optimization
Simulation results demonstrate that movable CF massive MIMO effectively suppresses the negative impact of the Doppler effect in HST communications.
Synergizing AI and Digital Twins for Next-Generation Network Optimization, Forecasting, and Security
This article presents a comprehensive analysis of the synergy of DNTs, FL, and RL techniques, showcasing their collective potential to address critical challenges in 6G networks.
SemiSAM+: Rethinking Semi-Supervised Medical Image Segmentation in the Era of Foundation Models
SemiSAM+ consists of one or multiple promptable foundation models as generalist models, and a trainable task-specific segmentation model as specialist model.
SegAnyPET: Universal Promptable Segmentation from Positron Emission Tomography Images
Positron Emission Tomography (PET) imaging plays a crucial role in modern medical diagnostics by revealing the metabolic processes within a patient's body, which is essential for quantification of therapy response and monitoring treatment progress.
Optimizing Wireless Resource Management and Synchronization in Digital Twin Networks
Since the DNT can predict the physical network status based on its historical status, the BSs may not need to send their physical network information at each time slot, allowing them to conserve spectrum resources to serve the users.
Exploit Gradient Skewness to Circumvent Byzantine Defenses for Federated Learning
This gradient skew phenomenon allows Byzantine gradients to hide within the densely distributed skewed gradients.
BRIDLE: Generalized Self-supervised Learning with Quantization
To address these limitations, we introduce BRIDLE (Bidirectional Residual Quantization Interleaved Discrete Learning Encoder), a self-supervised encoder pretraining framework that incorporates residual quantization (RQ) into the bidirectional training process, and is generalized for pretraining with audio, image, and video.
INSIGHT: Enhancing Autonomous Driving Safety through Vision-Language Models on Context-Aware Hazard Detection and Edge Case Evaluation
Autonomous driving systems face significant challenges in handling unpredictable edge-case scenarios, such as adversarial pedestrian movements, dangerous vehicle maneuvers, and sudden environmental changes.
Poisoning Attacks and Defenses to Federated Unlearning
In BadUnlearn, malicious clients send specifically designed local model updates to the server during the unlearning process, aiming to ensure that the resulting unlearned model remains poisoned.
Face-Human-Bench: A Comprehensive Benchmark of Face and Human Understanding for Multi-modal Assistants
Faces and humans are crucial elements in social interaction and are widely included in everyday photos and videos.
DOLLAR: Few-Step Video Generation via Distillation and Latent Reward Optimization
In this work, we introduce a distillation method that combines variational score distillation and consistency distillation to achieve few-step video generation, maintaining both high quality and diversity.
Lingma SWE-GPT: An Open Development-Process-Centric Language Model for Automated Software Improvement
The results demonstrate that Lingma SWE-GPT 72B successfully resolves 30. 20% of the GitHub issues, marking a significant improvement in automatic issue resolution (22. 76% relative improvement compared to Llama 3. 1 405B), approaching the performance of closed-source models (31. 80\% issues of GPT-4o resolved).
FlexMol: A Flexible Toolkit for Benchmarking Molecular Relational Learning
Molecular relational learning (MRL) is crucial for understanding the interaction behaviors between molecular pairs, a critical aspect of drug discovery and development.
Mixture of Efficient Diffusion Experts Through Automatic Interval and Sub-Network Selection
Finally, with a selected configuration, we fine-tune our pruned experts to obtain our mixture of efficient experts.
Joint Vehicle Connection and Beamforming Optimization in Digital Twin Assisted Integrated Sensing and Communication Vehicular Networks
To solve the simplified problem, this paper introduces both greedy and heuristic algorithms through optimizing both vehicle assignments and predictive beamforming.
Detecting, Explaining, and Mitigating Memorization in Diffusion Models
In this work, we introduce a straightforward yet effective method for detecting memorized prompts by inspecting the magnitude of text-conditional predictions.
Digital Twin-Assisted Data-Driven Optimization for Reliable Edge Caching in Wireless Networks
To bridge this gap, we introduce a novel digital twin-assisted optimization framework, called D-REC, which integrates reinforcement learning (RL) with diverse intervention modules to ensure reliable caching in nextG wireless networks.
Byzantine-Robust Decentralized Federated Learning
However, due to its fully decentralized nature, DFL is highly vulnerable to poisoning attacks, where malicious clients could manipulate the system by sending carefully-crafted local models to their neighboring clients.
Seed-TTS: A Family of High-Quality Versatile Speech Generation Models
Seed-TTS offers superior controllability over various speech attributes such as emotion and is capable of generating highly expressive and diverse speech for speakers in the wild.
Self-Modifying State Modeling for Simultaneous Machine Translation
Besides, building decision paths requires unidirectional encoders to simulate streaming source inputs, which impairs the translation quality of SiMT models.
Online Analytic Exemplar-Free Continual Learning with Large Models for Imbalanced Autonomous Driving Task
One of these scenarios can be formulated as an online continual learning (OCL) problem.
Personalized Residuals for Concept-Driven Text-to-Image Generation
We present personalized residuals and localized attention-guided sampling for efficient concept-driven generation using text-to-image diffusion models.
Attention-Driven Training-Free Efficiency Enhancement of Diffusion Models
Specifically, for single-denoising-step pruning, we develop a novel ranking algorithm, Generalized Weighted Page Rank (G-WPR), to identify redundant tokens, and a similarity-based recovery method to restore tokens for the convolution operation.
Poisoning Attacks on Federated Learning-based Wireless Traffic Prediction
Federated Learning (FL) offers a distributed framework to train a global control model across multiple base stations without compromising the privacy of their local network data.
Mapping Wireless Networks into Digital Reality through Joint Vertical and Horizontal Learning
In recent years, the complexity of 5G and beyond wireless networks has escalated, prompting a need for innovative frameworks to facilitate flexible management and efficient deployment.
F-MALLOC: Feed-forward Memory Allocation for Continual Learning in Neural Machine Translation
In the evolving landscape of Neural Machine Translation (NMT), the pretrain-then-finetune paradigm has yielded impressive results.
DELTA: Decomposed Efficient Long-Term Robot Task Planning using Large Language Models
In our extensive evaluation, we show that DELTA enables an efficient and fully automatic task planning pipeline, achieving higher planning success rates and significantly shorter planning times compared to the state of the art.
Spikewhisper: Temporal Spike Backdoor Attacks on Federated Neuromorphic Learning over Low-power Devices
Federated neuromorphic learning (FedNL) leverages event-driven spiking neural networks and federated learning frameworks to effectively execute intelligent analysis tasks over amounts of distributed low-power devices but also perform vulnerability to poisoning attacks.
F-OAL: Forward-only Online Analytic Learning with Fast Training and Low Memory Footprint in Class Incremental Learning
In this paper, we propose an exemplar-free approach--Forward-only Online Analytic Learning (F-OAL).
ToolNet: Connecting Large Language Models with Massive Tools via Tool Graph
While achieving remarkable progress in a broad range of tasks, large language models (LLMs) remain significantly limited in properly using massive external tools.
Collaborative Reinforcement Learning Based Unmanned Aerial Vehicle (UAV) Trajectory Design for 3D UAV Tracking
Meanwhile, since the accuracy of the distance estimation depends on the signal-to-noise ratio of the transmission signals, the active UAV must optimize its transmit power.
Self-supervised Learning for Electroencephalogram: A Systematic Survey
2) We provide a comprehensive review of SSL for EEG analysis, including taxonomy, methodology, and technique details of the existing EEG-based SSL frameworks, and discuss the difference between these methods.
SNED: Superposition Network Architecture Search for Efficient Video Diffusion Model
Our method employs a supernet training paradigm that targets various model cost and resolution options using a weight-sharing method.
SemiSAM: Enhancing Semi-Supervised Medical Image Segmentation via SAM-Assisted Consistency Regularization
Semi-supervised learning has attracted much attention due to its less dependence on acquiring abundant annotations from experts compared to fully supervised methods, which is especially important for medical image segmentation which typically requires intensive pixel/voxel-wise labeling by domain experts.
A Joint Gradient and Loss Based Clustered Federated Learning Design
In particular, our designed clustered FL algorithm must overcome two challenges associated with FL training.
From CLIP to DINO: Visual Encoders Shout in Multi-modal Large Language Models
By simply equipping it with an MLP layer for alignment, DINO surpasses CLIP in fine-grained related perception tasks.
BLSP: Bootstrapping Language-Speech Pre-training via Behavior Alignment of Continuation Writing
One is a cascaded approach where outputs (tokens or states) of a separately trained speech recognition system are used as inputs for LLMs, which limits their potential in modeling alignment between speech and text.
Gentopia: A Collaborative Platform for Tool-Augmented LLMs
We present gentopia, an ALM framework enabling flexible customization of agents through simple configurations, seamlessly integrating various language models, task formats, prompting modules, and plugins into a unified paradigm.
NBMOD: Find It and Grasp It in Noisy Background
In the past few years, researchers have proposed many methods to address the above-mentioned issues and achieved very good results on publicly available datasets such as the Cornell dataset and the Jacquard dataset.
Ranked #1 on
Robotic Grasping
on NBMOD
(using extra training data)
Few-shot Multi-domain Knowledge Rearming for Context-aware Defence against Advanced Persistent Threats
Secondly, to rearm current security strategies, an finetuning-based deployment mechanism is proposed to transfer learned knowledge into the student model, while minimizing the defense cost.
ReWOO: Decoupling Reasoning from Observations for Efficient Augmented Language Models
Augmented Language Models (ALMs) blend the reasoning capabilities of Large Language Models (LLMs) with tools that allow for knowledge retrieval and action execution.
MMViT: Multiscale Multiview Vision Transformers
Our model encodes different views of the input signal and builds several channel-resolution feature stages to process the multiple views of the input at different resolutions in parallel.
Ethylene Leak Detection Based on Infrared Imaging: A Benchmark
We find that the detection criteria used in infrared imaging ethylene leakage detection research cannot fully reflect real-world production conditions, which is not conducive to evaluate the performance of current image-based target detection methods.
Multi-modal Expression Recognition with Ensemble Method
This paper presents our submission to the Expression Classification Challenge of the fifth Affective Behavior Analysis in-the-wild (ABAW) Competition.
Byzantine-Robust Learning on Heterogeneous Data via Gradient Splitting
In order to address this issue, we propose GAS, a \shorten approach that can successfully adapt existing robust AGRs to non-IID settings.
A Composite T60 Regression and Classification Approach for Speech Dereverberation
More specifically, we develop a joint learning approach that uses a composite T60 module and a separate dereverberation module to simultaneously perform reverberation time estimation and dereverberation.
Promoting Semantic Connectivity: Dual Nearest Neighbors Contrastive Learning for Unsupervised Domain Generalization
Then, we propose a novel unsupervised domain generalization approach, namely Dual Nearest Neighbors contrastive learning with strong Augmentation (DN^2A).
Towards Unsupervised Domain Generalization for Face Anti-Spoofing
In this paper, we propose the first Unsupervised Domain Generalization framework for Face Anti-Spoofing, namely UDG-FAS, which could exploit large amounts of easily accessible unlabeled data to learn generalizable features for enhancing the low-data regime of FAS.
Adapting Shortcut With Normalizing Flow: An Efficient Tuning Framework for Visual Recognition
To mitigate the computational and storage demands, recent research has explored Parameter-Efficient Fine-Tuning (PEFT), which focuses on tuning a minimal number of parameters for efficient adaptation.
Improving Training and Inference of Face Recognition Models via Random Temperature Scaling
Intuitively, FR algorithms can benefit from both the estimation of uncertainty and the detection of out-of-distribution (OOD) samples.
CCATMos: Convolutional Context-aware Transformer Network for Non-intrusive Speech Quality Assessment
Speech quality assessment has been a critical component in many voice communication related applications such as telephony and online conferencing.
CodePAD: Sequence-based Code Generation with Pushdown Automaton
CodePAD can leverage existing sequence-based models, and we show that it can achieve 100\% grammatical correctness percentage on these benchmark datasets.
Discrete Cross-Modal Alignment Enables Zero-Shot Speech Translation
Existing zero-shot methods fail to align the two modalities of speech and text into a shared semantic space, resulting in much worse performance compared to the supervised ST methods.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+5
3D-FM GAN: Towards 3D-Controllable Face Manipulation
While concatenating GAN inversion and a 3D-aware, noise-to-image GAN is a straight-forward solution, it is inefficient and may lead to noticeable drop in editing quality.
Court Judgement Labeling Using Topic Modeling and Syntactic Parsing
In regions that practice common law, relevant historical cases are essential references for sentencing.
SdAE: Self-distillated Masked Autoencoder
We also analyze how to build good views for the teacher branch to produce latent representation from the perspective of information bottleneck.
Multi-Task Learning Framework for Emotion Recognition in-the-wild
This paper presents our system for the Multi-Task Learning (MTL) Challenge in the 4th Affective Behavior Analysis in-the-wild (ABAW) competition.
Scalable Model-based Policy Optimization for Decentralized Networked Systems
Reinforcement learning algorithms require a large amount of samples; this often limits their real-world applications on even simple tasks.
CalFAT: Calibrated Federated Adversarial Training with Label Skewness
In this paper, we study the problem of FAT under label skewness, and reveal one root cause of the training instability and natural accuracy degradation issues: skewed labels lead to non-identical class probabilities and heterogeneous local models.
M3ED: Multi-modal Multi-scene Multi-label Emotional Dialogue Database
In this work, we propose a Multi-modal Multi-scene Multi-label Emotional Dialogue dataset, M3ED, which contains 990 dyadic emotional dialogues from 56 different TV series, a total of 9, 082 turns and 24, 449 utterances.
Subspace Nonnegative Matrix Factorization for Feature Representation
Traditional nonnegative matrix factorization (NMF) learns a new feature representation on the whole data space, which means treating all features equally.
Multi-modal Emotion Estimation for in-the-wild Videos
In this paper, we briefly introduce our submission to the Valence-Arousal Estimation Challenge of the 3rd Affective Behavior Analysis in-the-wild (ABAW) competition.
Evolving Transferable Neural Pruning Functions
While most prior works in evolutionary learning aim at directly searching the structure of a network, few attempts have been made on another promising track, channel pruning, which recently has made major headway in designing efficient deep learning models.
Class-Discriminative CNN Compression
We then propose a novel layer-adaptive hierarchical pruning approach, where we use a coarse class discrimination scheme for early layers and a fine one for later layers.
Fully Decentralized Model-based Policy Optimization with Networked Agents
Model-based RL is an effective approach for reducing sample complexity.
Model-based Reinforcement Learning
Reinforcement Learning (RL)
MMGCN: Multimodal Fusion via Deep Graph Convolution Network for Emotion Recognition in Conversation
Emotion recognition in conversation (ERC) is a crucial component in affective dialogue systems, which helps the system understand users' emotions and generate empathetic responses.
Content-Aware GAN Compression
We then propose a novel content-aware method to guide the processes of both pruning and distillation.
Finite generation for valuations computing stability thresholds and applications to K-stability
We prove that on any log Fano pair of dimension $n$ whose stability threshold is less than $\frac{n+1}{n}$, any valuation computing the stability threshold has a finitely generated associated graded ring.
Algebraic Geometry Differential Geometry
Learning Latent Architectural Distribution in Differentiable Neural Architecture Search via Variational Information Maximization
Existing differentiable neural architecture search approaches simply assume the architectural distribution on each edge is independent of each other, which conflicts with the intrinsic properties of architecture.
Emotions in Online Content Diffusion
We apply a partial-linear instrumental variable approach with a double machine learning framework to causally identify the impact of the negative discrete emotions on online content diffusion.
Bridging the Modality Gap for Speech-to-Text Translation
End-to-end speech translation aims to translate speech in one language into text in another language via an end-to-end way.
CASIA's System for IWSLT 2020 Open Domain Translation
This paper describes the CASIA{'}s system for the IWSLT 2020 open domain translation task.
Rethinking Class-Discrimination Based CNN Channel Pruning
To this end, we initiate the first study on the effectiveness of a broad range of discriminant functions on channel pruning.
Synchronous Speech Recognition and Speech-to-Text Translation with Interactive Decoding
Speech-to-text translation (ST), which translates source language speech into target language text, has attracted intensive attention in recent years.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+5
Synchronously Generating Two Languages with Interactive Decoding
In this paper, we introduce a novel interactive approach to translate a source language into two different languages simultaneously and interactively.
End-to-End Speech Translation with Knowledge Distillation
End-to-end speech translation (ST), which directly translates from source language speech into target language text, has attracted intensive attentions in recent years.
Language-Independent Representor for Neural Machine Translation
Current Neural Machine Translation (NMT) employs a language-specific encoder to represent the source sentence and adopts a language-specific decoder to generate target translation.
Unsupervised Learning of Neural Networks to Explain Neural Networks
Given feature maps of a certain conv-layer of the CNN, the explainer performs like an auto-encoder, which first disentangles the feature maps into object-part features and then inverts object-part features back to features of higher conv-layers of the CNN.
Pixie: A System for Recommending 3+ Billion Items to 200+ Million Users in Real-Time
Furthermore, we develop a graph pruning strategy at that leads to an additional 58% improvement in recommendations.
Human Curation and Convnets: Powering Item-to-Item Recommendations on Pinterest
This paper presents Pinterest Related Pins, an item-to-item recommendation system that combines collaborative filtering with content-based ranking.
