Search Results for author:
Found 278 papers, 81 papers with code
DocEE: A Large-Scale and Fine-grained Benchmark for Document-level Event Extraction
Event extraction aims to identify an event and then extract the arguments participating in the event.
Automatic recognition of abdominal lymph nodes from clinical text
We then introduce an end-to-end approach based on the combination of rules and transformer-based methods to detect these abdominal lymph node mentions and classify their types from the MRI radiology reports.
Pre-training vs. Fine-tuning: A Reproducibility Study on Dense Retrieval Knowledge Acquisition
Recent research has questioned the role of fine-tuning vs. that of pre-training within dense retrievers, specifically arguing that retrieval knowledge is primarily gained during pre-training, meaning knowledge not acquired during pre-training cannot be sub-sequentially acquired via fine-tuning.
Reassessing Large Language Model Boolean Query Generation for Systematic Reviews
Our results show that query effectiveness varies significantly across models and prompt designs, with guided query formulation benefiting from well-chosen seed studies.
ArtRAG: Retrieval-Augmented Generation with Structured Context for Visual Art Understanding
Understanding visual art requires reasoning across multiple perspectives -- cultural, historical, and stylistic -- beyond mere object recognition.
FedEMA: Federated Exponential Moving Averaging with Negative Entropy Regularizer in Autonomous Driving
Street Scene Semantic Understanding (denoted as S3U) is a crucial but complex task for autonomous driving (AD) vehicles.
Black-Box Visual Prompt Engineering for Mitigating Object Hallucination in Large Vision Language Models
Large Vision Language Models (LVLMs) often suffer from object hallucination, which undermines their reliability.
Opportunistic Collaborative Planning with Large Vision Model Guided Control and Joint Query-Service Optimization
Existing solutions either rely on small models that struggle with generalization or large models that are resource-intensive.
BadMoE: Backdooring Mixture-of-Experts LLMs via Optimizing Routing Triggers and Infecting Dormant Experts
Accordingly, our attack, namely BadMoE, exploits the unique architecture of MoE models by 1) identifying dormant experts unrelated to the target task, 2) constructing a routing-aware loss to optimize the activation triggers of these experts, and 3) promoting dormant experts to dominating roles via poisoned training data.
Seeing The Words: Evaluating AI-generated Biblical Art
This triggers the discussion of whether AI can generate accurate images using text from the Bible with respect to the corresponding biblical contexts and backgrounds.
Green Robotic Mixed Reality with Gaussian Splatting
Realizing green communication in robotic mixed reality (RoboMR) systems presents a challenge, due to the necessity of uploading high-resolution images at high frequencies through wireless channels.
DMM: Building a Versatile Image Generation Model via Distillation-Based Model Merging
The success of text-to-image (T2I) generation models has spurred a proliferation of numerous model checkpoints fine-tuned from the same base model on various specialized datasets.
GNN-ACLP: Graph Neural Networks based Analog Circuit Link Prediction
Circuit link prediction identifying missing component connections from incomplete netlists is crucial in automating analog circuit design.
Seed1.5-Thinking: Advancing Superb Reasoning Models with Reinforcement Learning
We introduce Seed1. 5-Thinking, capable of reasoning through thinking before responding, resulting in improved performance on a wide range of benchmarks.
DDT: Decoupled Diffusion Transformer
For ImageNet $256\times256$, Our DDT-XL/2 achieves a new state-of-the-art performance of {1. 31 FID}~(nearly $4\times$ faster training convergence compared to previous diffusion transformers).
Ranked #5 on
Image Generation
on ImageNet 512x512
Causal Self-supervised Pretrained Frontend with Predictive Code for Speech Separation
The pretrained frontend employs a transformer decoder network with a causal convolutional encoder as the backbone and is pretrained in a self-supervised manner with two innovative pretext tasks: autoregressive hybrid prediction and contextual knowledge distillation.
$C^2$AV-TSE: Context and Confidence-aware Audio Visual Target Speaker Extraction
Audio-Visual Target Speaker Extraction (AV-TSE) aims to mimic the human ability to enhance auditory perception using visual cues.
Advancing THz Radio Map Construction and Obstacle Sensing: An Integrated Generative Framework in ISAC
Integrated sensing and communication (ISAC) in the terahertz (THz) band enables obstacle detection, which in turn facilitates efficient beam management to mitigate THz signal blockage.
Generative Adversarial Network
Integrated sensing and communication
+1
GateLens: A Reasoning-Enhanced LLM Agent for Automotive Software Release Analytics
Ensuring the reliability and effectiveness of software release decisions is critical, particularly in safety-critical domains like automotive systems.
STShield: Single-Token Sentinel for Real-Time Jailbreak Detection in Large Language Models
Large Language Models (LLMs) have become increasingly vulnerable to jailbreak attacks that circumvent their safety mechanisms.
SDTrack: A Baseline for Event-based Tracking via Spiking Neural Networks
Spiking Neural Networks (SNNs) naturally complement event data through discrete spike signals, making them ideal for event-based tracking.
CHOP: Mobile Operating Assistant with Constrained High-frequency Optimized Subtask Planning
The subtask level, linking high-level goals with low-level executable actions, is crucial for task completion but faces two challenges: ineffective subtasks that lower-level agent cannot execute and inefficient subtasks that fail to contribute to the completion of the higher-level task.
OmniSQL: Synthesizing High-quality Text-to-SQL Data at Scale
Text-to-SQL, the task of translating natural language questions into SQL queries, plays a crucial role in enabling non-experts to interact with databases.
DiffRhythm: Blazingly Fast and Embarrassingly Simple End-to-End Full-Length Song Generation with Latent Diffusion
To address these challenges, we propose DiffRhythm, the first latent diffusion-based song generation model capable of synthesizing complete songs with both vocal and accompaniment for durations of up to 4m45s in only ten seconds, maintaining high musicality and intelligibility.
CLIP-Optimized Multimodal Image Enhancement via ISP-CNN Fusion for Coal Mine IoVT under Uneven Illumination
Clear monitoring images are crucial for the safe operation of coal mine Internet of Video Things (IoVT) systems.
Memory-Free and Parallel Computation for Quantized Spiking Neural Networks
In this study, we first identify a new underlying cause for this decline: the loss of historical information due to the quantized membrane potential.
A Systematic Survey of Automatic Prompt Optimization Techniques
Since the advent of large language models (LLMs), prompt engineering has been a crucial step for eliciting desired responses for various Natural Language Processing (NLP) tasks.
GuidedBench: Equipping Jailbreak Evaluation with Guidelines
Some jailbreak methods that claim to achieve over 90% attack success rate (ASR) on other benchmarks only reach a maximum of 30. 2% on our benchmark, providing a higher ceiling for more advanced jailbreak research; furthermore, using our scoring system reduces the variance of disagreements between different evaluator LLMs by up to 76. 33%.
Towards Accurate Binary Spiking Neural Networks: Learning with Adaptive Gradient Modulation Mechanism
Binary Spiking Neural Networks (BSNNs) inherit the eventdriven paradigm of SNNs, while also adopting the reduced storage burden of binarization techniques.
Exploring Large Language Models in Healthcare: Insights into Corpora Sources, Customization Strategies, and Evaluation Metrics
This study reviewed the use of Large Language Models (LLMs) in healthcare, focusing on their training corpora, customization techniques, and evaluation metrics.
Can't See the Forest for the Trees: Benchmarking Multimodal Safety Awareness for Multimodal LLMs
Multimodal Large Language Models (MLLMs) have expanded the capabilities of traditional language models by enabling interaction through both text and images.
LaRA: Benchmarking Retrieval-Augmented Generation and Long-Context LLMs - No Silver Bullet for LC or RAG Routing
However, the advancements in context window size for LLMs offer an alternative approach, raising the question of whether RAG remains necessary for effectively handling external knowledge.
Estimating Probabilities of Causation with Machine Learning Models
To estimate these probabilities for subpopulations with insufficient data, we propose using machine learning models that draw insights from subpopulations with sufficient data.
Towards THz-based Obstacle Sensing: A Generative Radio Environment Awareness Framework
To solve such a problem, we propose a THz-based generative radio environment awareness framework, in which obstacle information is obtained directly from the aware radio environment.
Automating a Complete Software Test Process Using LLMs: An Automotive Case Study
Vehicle API testing verifies whether the interactions between a vehicle's internal systems and external applications meet expectations, ensuring that users can access and control various vehicle functions and data.
Label Anything: An Interpretable, High-Fidelity and Prompt-Free Annotator
To mitigate this cost of manual labeling, we propose a Label Anything Model (denoted as LAM), serving as an interpretable, high-fidelity, and prompt-free data annotator.
High-Precision Fabric Defect Detection via Adaptive Shape Convolutions and Large Kernel Spatial Modeling
Detecting fabric defects in the textile industry remains a challenging task due to the diverse and complex nature of defect patterns.
ACEBench: Who Wins the Match Point in Tool Learning?
Normal evaluates function calls in basic scenarios; Special evaluates function calls in scenarios with vague or incomplete instructions; Agent introduces multi-agent interactions to simulate function calling evaluation in real-world multi-turn interactions.
Communication-Efficient Federated Learning by Quantized Variance Reduction for Heterogeneous Wireless Edge Networks
In this paper, we propose a novel communication-efficient FL algorithm, named FedQVR, which relies on a sophisticated variance-reduced scheme to achieve heterogeneity-robustness in the presence of quantized transmission and heterogeneous local updates among active edge devices.
How Should We Build A Benchmark? Revisiting 274 Code-Related Benchmarks For LLMs
We propose How2Bench, which is comprised of a 55-criteria checklist as a set of guidelines to govern the development of code-related benchmarks comprehensively.
Leveraging Metamemory Mechanisms for Enhanced Data-Free Code Generation in LLMs
Automated code generation using large language models (LLMs) has gained attention due to its efficiency and adaptability.
ExPO: Explainable Phonetic Trait-Oriented Network for Speaker Verification
Furthermore, we investigate phonetic traits from within-speaker and between-speaker variation perspectives to determine which trait is most effective for speaker verification, marking an important step towards explainable speaker verification.
Enhancing Large Vision Model in Street Scene Semantic Understanding through Leveraging Posterior Optimization Trajectory
Specifically, as the fitted data volume grows to exceed the the AD model's fitting capacities, the AD model is prone to under-fitting.
SongEditor: Adapting Zero-Shot Song Generation Language Model as a Multi-Task Editor
The emergence of novel generative modeling paradigms, particularly audio language models, has significantly advanced the field of song generation.
Surrealistic-like Image Generation with Vision-Language Models
Recent advances in generative AI make it convenient to create different types of content, including text, images, and code.
AI PERSONA: Towards Life-long Personalization of LLMs
In this work, we introduce the task of life-long personalization of large language models.
RecSys Arena: Pair-wise Recommender System Evaluation with Large Language Models
More specifically, for each sample we use LLM to generate a user profile description based on user behavior history or off-the-shelf profile features, which is used to guide LLM to play the role of this user and evaluate the relative preference for two recommendation results generated by different models.
MoMuSE: Momentum Multi-modal Target Speaker Extraction for Real-time Scenarios with Impaired Visual Cues
Audio-visual Target Speaker Extraction (AV-TSE) aims to isolate the speech of a specific target speaker from an audio mixture using time-synchronized visual cues.
Fab-ME: A Vision State-Space and Attention-Enhanced Framework for Fabric Defect Detection
Effective defect detection is critical for ensuring the quality, functionality, and economic value of textile products.
Adaptive Interactive Segmentation for Multimodal Medical Imaging via Selection Engine
To mitigate memory bottlenecks and optimize prompt frame selection during the inference of 2D image sequences, we developed an automated system, the Adaptive Frame Selection Engine (AFSE).
2D Matryoshka Training for Information Retrieval
In this reproducibility study, we implement and evaluate both versions of 2D Matryoshka Training on STS tasks and extend our analysis to retrieval tasks.
mmSpyVR: Exploiting mmWave Radar for Penetrating Obstacles to Uncover Privacy Vulnerability of Virtual Reality
The mmSpyVR demonstrates the capability to extract critical VR privacy from the mmWave signals that have penetrated through obstacles.
GUI Agents with Foundation Models: A Comprehensive Survey
Recent advances in foundation models, particularly Large Language Models (LLMs) and Multimodal Large Language Models (MLLMs), have facilitated the development of intelligent agents capable of performing complex tasks.
Speech Separation with Pretrained Frontend to Minimize Domain Mismatch
As a result, there exists a domain gap between real and synthetic data when deploying speech separation models in real-world applications.
Beyond Utility: Evaluating LLM as Recommender
We intend our evaluation framework and observations to benefit future research on the use of LLMs as recommenders.
The ISCSLP 2024 Conversational Voice Clone (CoVoC) Challenge: Tasks, Results and Findings
The ISCSLP 2024 Conversational Voice Clone (CoVoC) Challenge aims to benchmark and advance zero-shot spontaneous style voice cloning, particularly focusing on generating spontaneous behaviors in conversational speech.
FlowDCN: Exploring DCN-like Architectures for Fast Image Generation with Arbitrary Resolution
Arbitrary-resolution image generation still remains a challenging task in AIGC, as it requires handling varying resolutions and aspect ratios while maintaining high visual quality.
Prototype and Instance Contrastive Learning for Unsupervised Domain Adaptation in Speaker Verification
In this paper, we propose Prototype and Instance Contrastive Learning (PICL), a novel method for unsupervised domain adaptation in speaker verification through dual-level contrastive learning.
Multi-Level Speaker Representation for Target Speaker Extraction
In this work, we propose a multi-level speaker representation approach, from raw features to neural embeddings, to serve as the speaker reference cue.
SPA-Bench: A Comprehensive Benchmark for SmartPhone Agent Evaluation
SPA-Bench offers three key contributions: (1) A diverse set of tasks covering system and third-party apps in both English and Chinese, focusing on features commonly used in daily routines; (2) A plug-and-play framework enabling real-time agent interaction with Android devices, integrating over ten agents with the flexibility to add more; (3) A novel evaluation pipeline that automatically assesses agent performance across multiple dimensions, encompassing seven metrics related to task completion and resource consumption.
Starbucks: Improved Training for 2D Matryoshka Embeddings
Effective approaches that can scale embedding model depth (i. e. layers) and embedding size allow for the creation of models that are highly scalable across different computational resources and task requirements.
StepTool: A Step-grained Reinforcement Learning Framework for Tool Learning in LLMs
Despite having powerful reasoning and inference capabilities, Large Language Models (LLMs) still need external tools to acquire real-time information retrieval or domain-specific expertise to solve complex tasks, which is referred to as tool learning.
Chain-of-Jailbreak Attack for Image Generation Models via Editing Step by Step
To assess the safety of existing models, we introduce a novel jailbreaking method called Chain-of-Jailbreak (CoJ) attack, which compromises image generation models through a step-by-step editing process.
Fast-Convergent and Communication-Alleviated Heterogeneous Hierarchical Federated Learning in Autonomous Driving
Unfortunately, it suffers from slow convergence because data from different cities are with disparate statistical properties.
Metasurface-generated large and arbitrary analog convolution kernels for accelerated machine vision
Recently, to address the challenges in processing speed and power consumption of conventional digital convolution operations, many optical components have been suggested to replace the digital convolution layer in the neural network, accelerating various machine vision tasks.
M-Vec: Matryoshka Speaker Embeddings with Flexible Dimensions
Fixed-dimensional speaker embeddings have become the dominant approach in speaker modeling, typically spanning hundreds to thousands of dimensions.
WeSep: A Scalable and Flexible Toolkit Towards Generalizable Target Speaker Extraction
Target speaker extraction (TSE) focuses on isolating the speech of a specific target speaker from overlapped multi-talker speech, which is a typical setup in the cocktail party problem.
E1 TTS: Simple and Fast Non-Autoregressive TTS
This paper introduces Easy One-Step Text-to-Speech (E1 TTS), an efficient non-autoregressive zero-shot text-to-speech system based on denoising diffusion pretraining and distribution matching distillation.
DFDG: Data-Free Dual-Generator Adversarial Distillation for One-Shot Federated Learning
In dual-model distillation, the trained dual generators work together to provide the training data for updates of the global model.
Privacy-Preserving Federated Learning with Consistency via Knowledge Distillation Using Conditional Generator
Federated Learning (FL) is gaining popularity as a distributed learning framework that only shares model parameters or gradient updates and keeps private data locally.
Joint Input and Output Coordination for Class-Incremental Learning
This mechanism assigns different weights to different categories of data according to the gradient of the output score, and uses knowledge distillation (KD) to reduce the mutual interference between the outputs of old and new tasks.
$\mathbb{USCD}$: Improving Code Generation of LLMs by Uncertainty-Aware Selective Contrastive Decoding
Then, we selectively eliminate output noise induced by lame prompts based on the uncertainty of the prediction distribution from the standard prompt.
vec2wav 2.0: Advancing Voice Conversion via Discrete Token Vocoders
To amend the loss of speaker timbre in the content tokens, vec2wav 2. 0 utilizes the WavLM features to provide strong timbre-dependent information.
ESP-PCT: Enhanced VR Semantic Performance through Efficient Compression of Temporal and Spatial Redundancies in Point Cloud Transformers
Notably, ESP-PCT achieves a remarkable accuracy of 93. 2% while reducing the computational requirements (FLOPs) by 76. 9% and memory usage by 78. 2% compared to the existing Point Transformer model simultaneously.
ToolACE: Winning the Points of LLM Function Calling
Function calling significantly extends the application boundary of large language models, where high-quality and diverse training data is critical for unlocking this capability.
Drop the beat! Freestyler for Accompaniment Conditioned Rapping Voice Generation
Rap, a prominent genre of vocal performance, remains underexplored in vocal generation.
Multi-modal Intermediate Feature Interaction AutoEncoder for Overall Survival Prediction of Esophageal Squamous Cell Cancer
In this work, we propose a novel autoencoder-based deep learning model to predict the overall survival of the ESCC.
Tackling Data Heterogeneity in Federated Learning via Loss Decomposition
To mitigate the impact of data heterogeneity on FL performance, we start with analyzing how FL training influence FL performance by decomposing the global loss into three terms: local loss, distribution shift loss and aggregation loss.
API-guided Dataset Synthesis to Finetune Large Code Models
Inspired by APIs as high-level abstractions of code that encapsulate rich semantic information in a concise structure, we propose DataScope, an API-guided dataset synthesis framework designed to enhance the SFT process for LCMs in both general and domain-specific scenarios.
Multiple Contexts and Frequencies Aggregation Network forDeepfake Detection
Deepfake detection faces increasing challenges since the fast growth of generative models in developing massive and diverse Deepfake technologies.
Overview of Speaker Modeling and Its Applications: From the Lens of Deep Speaker Representation Learning
Speaker individuality information is among the most critical elements within speech signals.
STS MICCAI 2023 Challenge: Grand challenge on 2D and 3D semi-supervised tooth segmentation
The Semi-supervised Teeth Segmentation (STS) Challenge, a pioneering event in tooth segmentation, was held as a part of the MICCAI 2023 ToothFairy Workshop on the Alibaba Tianchi platform.
Context Embeddings for Efficient Answer Generation in RAG
We address this challenge by presenting COCOM, an effective context compression method, reducing long contexts to only a handful of Context Embeddings speeding up the generation time by a large margin.
Ternary Spike-based Neuromorphic Signal Processing System
QT-SNN, compatible with ternary spike trains from the TAE method, quantifies both membrane potentials and synaptic weights to reduce memory requirements while maintaining performance.
Generalized Robust Fundus Photography-based Vision Loss Estimation for High Myopia
High myopia significantly increases the risk of irreversible vision loss.
BERGEN: A Benchmarking Library for Retrieval-Augmented Generation
In response to the recent popularity of generative LLMs, many RAG approaches have been proposed, which involve an intricate number of different configurations such as evaluation datasets, collections, metrics, retrievers, and LLMs.
Symbolic Learning Enables Self-Evolving Agents
In this work, we introduce agent symbolic learning, a systematic framework that enables language agents to optimize themselves on their own in a data-centric way using symbolic optimizers.
SurgeMOD: Translating image-space tissue motions into vision-based surgical forces
Using internal movements generated by natural processes like breathing or the cardiac cycle, we infer the image-space basis of the motion on the frequency domain.
An Investigation of Prompt Variations for Zero-shot LLM-based Rankers
We provide a systematic understanding of the impact of specific components and wordings used in prompts on the effectiveness of rankers based on zero-shot Large Language Models (LLMs).
Global-Local Convolution with Spiking Neural Networks for Energy-efficient Keyword Spotting
Here, we take advantage of spiking neural networks' energy efficiency and propose an end-to-end lightweight KWS model.
Scalable Differentiable Causal Discovery in the Presence of Latent Confounders with Skeleton Posterior (Extended Version)
Differentiable causal discovery has made significant advancements in the learning of directed acyclic graphs.
DualVC 3: Leveraging Language Model Generated Pseudo Context for End-to-end Low Latency Streaming Voice Conversion
With speaker-independent semantic tokens to guide the training of the content encoder, the dependency on ASR is removed and the model can operate under extremely small chunks, with cascading errors eliminated.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+5
WenetSpeech4TTS: A 12,800-hour Mandarin TTS Corpus for Large Speech Generation Model Benchmark
Furthermore, we have created subsets of varying sizes, categorized by segment quality scores to allow for TTS model training and fine-tuning.
Autoregressive Diffusion Transformer for Text-to-Speech Synthesis
Our experiments reveal that employing Integral Kullback-Leibler (IKL) divergence for distillation at each autoregressive step significantly boosts the perceived quality of the samples.
SelfDefend: LLMs Can Defend Themselves against Jailbreaking in a Practical Manner
Jailbreaking is an emerging adversarial attack that bypasses the safety alignment deployed in off-the-shelf large language models (LLMs) and has evolved into multiple categories: human-based, optimization-based, generation-based, and the recent indirect and multilingual jailbreaks.
Ada-HGNN: Adaptive Sampling for Scalable Hypergraph Neural Networks
Hypergraphs serve as an effective model for depicting complex connections in various real-world scenarios, from social to biological networks.
MMFusion: Multi-modality Diffusion Model for Lymph Node Metastasis Diagnosis in Esophageal Cancer
In addition, efficient and effective interactions between multi-modal representations need to be further explored, lacking insightful exploration of prognostic correlation in multi-modality features.
Attention-Constrained Inference for Robust Decoder-Only Text-to-Speech
We call the attention maps of those heads Alignment-Emerged Attention Maps (AEAMs).
Mamba-FETrack: Frame-Event Tracking via State Space Model
Specifically, our Mamba-based tracker achieves 43. 5/55. 6 on the SR/PR metric, while the ViT-S based tracker (OSTrack) obtains 40. 0/50. 9.
Testing and Understanding Erroneous Planning in LLM Agents through Synthesized User Inputs
These synthesized inputs are natural language paragraphs that specify the requirements for completing a series of tasks.
Integrated Sensing and Communication for Edge Inference with End-to-End Multi-View Fusion
Integrated sensing and communication (ISAC) is a promising solution to accelerate edge inference via the dual use of wireless signals.
The X-LANCE Technical Report for Interspeech 2024 Speech Processing Using Discrete Speech Unit Challenge
Discrete speech tokens have been more and more popular in multiple speech processing fields, including automatic speech recognition (ASR), text-to-speech (TTS) and singing voice synthesis (SVS).
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+4
VHM: Versatile and Honest Vision Language Model for Remote Sensing Image Analysis
VHM is built on a large-scale remote sensing image-text dataset with rich-content captions (VersaD), and an honest instruction dataset comprising both factual and deceptive questions (HnstD).
Information Cascade Prediction under Public Emergencies: A Survey
With the advent of the era of big data, massive information, expert experience, and high-accuracy models bring great opportunities to the information cascade prediction of public emergencies.
Graph Image Prior for Unsupervised Dynamic Cardiac Cine MRI Reconstruction
A graph convolutional network is utilized for feature fusion and dynamic image generation.
Safeguarding Medical Image Segmentation Datasets against Unauthorized Training via Contour- and Texture-Aware Perturbations
This is particularly true for medical image segmentation (MIS) datasets, where the processes of collection and fine-grained annotation are time-intensive and laborious.
NN-Defined Modulator: Reconfigurable and Portable Software Modulator on IoT Gateways
A physical-layer modulator is a vital component for an IoT gateway to map the symbols to signals.
NeuPAN: Direct Point Robot Navigation with End-to-End Model-based Learning
This paper presents NeuPAN: a real-time, highly accurate, map-free, easy-to-deploy, and environment-invariant robot motion planner.
From Instructions to Constraints: Language Model Alignment with Automatic Constraint Verification
We investigate common constraints in NLP tasks, categorize them into three classes based on the types of their arguments, and propose a unified framework, ACT (Aligning to ConsTraints), to automatically produce supervision signals for user alignment with constraints.
Suppress and Rebalance: Towards Generalized Multi-Modal Face Anti-Spoofing
Face Anti-Spoofing (FAS) is crucial for securing face recognition systems against presentation attacks.
LLMs Can Defend Themselves Against Jailbreaking in a Practical Manner: A Vision Paper
Our key insight is that regardless of the kind of jailbreak strategies employed, they eventually need to include a harmful prompt (e. g., "how to make a bomb") in the prompt sent to LLMs, and we found that existing LLMs can effectively recognize such harmful prompts that violate their safety policies.
Large Language Models for Stemming: Promises, Pitfalls and Failures
With this respect, we identify three avenues, each characterised by different trade-offs in terms of computational cost, effectiveness and robustness : (1) use LLMs to stem the vocabulary for a collection, i. e., the set of unique words that appear in the collection (vocabulary stemming), (2) use LLMs to stem each document separately (contextual stemming), and (3) use LLMs to extract from each document entities that should not be stemmed, then use vocabulary stemming to stem the rest of the terms (entity-based contextual stemming).
FeB4RAG: Evaluating Federated Search in the Context of Retrieval Augmented Generation
Federated search systems aggregate results from multiple search engines, selecting appropriate sources to enhance result quality and align with user intent.
Eliminating Information Leakage in Hard Concept Bottleneck Models with Supervised, Hierarchical Concept Learning
This paper alleviates the information leakage issue by introducing label supervision in concept predication and constructing a hierarchical concept set.
ReSLLM: Large Language Models are Strong Resource Selectors for Federated Search
Our ReSLLM method exploits LLMs to drive the selection of resources in federated search in a zero-shot setting.
Weaver: Foundation Models for Creative Writing
Weaver is pre-trained on a carefully selected corpus that focuses on improving the writing capabilities of large language models.
An Empirical Study on Large Language Models in Accuracy and Robustness under Chinese Industrial Scenarios
In this paper, we present a comprehensive empirical study on the accuracy and robustness of LLMs in the context of the Chinese industrial production area.
VALL-T: Decoder-Only Generative Transducer for Robust and Decoding-Controllable Text-to-Speech
Recent TTS models with decoder-only Transformer architecture, such as SPEAR-TTS and VALL-E, achieve impressive naturalness and demonstrate the ability for zero-shot adaptation given a speech prompt.
Zero-shot Generative Large Language Models for Systematic Review Screening Automation
Systematic reviews are crucial for evidence-based medicine as they comprehensively analyse published research findings on specific questions.
OOP: Object-Oriented Programming Evaluation Benchmark for Large Language Models
Advancing automated programming necessitates robust and comprehensive code generation benchmarks, yet current evaluation frameworks largely neglect object-oriented programming (OOP) in favor of functional programming (FP), e. g., HumanEval and MBPP.
The NUS-HLT System for ICASSP2024 ICMC-ASR Grand Challenge
This paper summarizes our team's efforts in both tracks of the ICMC-ASR Challenge for in-car multi-channel automatic speech recognition.
VRPTEST: Evaluating Visual Referring Prompting in Large Multimodal Models
With recent advancements in Large Multimodal Models (LMMs) across various domains, a novel prompting method called visual referring prompting has emerged, showing significant potential in enhancing human-computer interaction within multimodal systems.
InstructTA: Instruction-Tuned Targeted Attack for Large Vision-Language Models
This practical setting poses challenges to the cross-prompt and cross-model transferability of targeted adversarial attack, which aims to confuse the LVLM to output a response that is semantically similar to the attacker's chosen target text.
Accurate Segmentation of Optic Disc And Cup from Multiple Pseudo-labels by Noise-aware Learning
Optic disc and cup segmentation plays a crucial role in automating the screening and diagnosis of optic glaucoma.
PMP-Swin: Multi-Scale Patch Message Passing Swin Transformer for Retinal Disease Classification
Specifically, we design a Patch Message Passing (PMP) module based on the Message Passing mechanism to establish global interaction for pathological semantic features and to exploit the subtle differences further between different diseases.
MSE-Nets: Multi-annotated Semi-supervised Ensemble Networks for Improving Segmentation of Medical Image with Ambiguous Boundaries
Medical image segmentation annotations exhibit variations among experts due to the ambiguous boundaries of segmented objects and backgrounds in medical images.
Unlock the Power: Competitive Distillation for Multi-Modal Large Language Models
The first stage pre-trains the student model on a large number of filtered multi-modal datasets.
CASTER: A Computer-Vision-Assisted Wireless Channel Simulator for Gesture Recognition
In the proposed CASTER simulator, however, the training dataset can be simulated via existing videos.
FDNet: Feature Decoupled Segmentation Network for Tooth CBCT Image
Precise Tooth Cone Beam Computed Tomography (CBCT) image segmentation is crucial for orthodontic treatment planning.
Evaluating Generative Ad Hoc Information Retrieval
To lay a foundation for developing new evaluation methods for generative retrieval systems, we survey the relevant literature from the fields of information retrieval and natural language processing, identify search tasks and system architectures in generative retrieval, develop a new user model, and study its operationalization.
Privacy-preserving Federated Primal-dual Learning for Non-convex and Non-smooth Problems with Model Sparsification
Federated learning (FL) has been recognized as a rapidly growing research area, where the model is trained over massively distributed clients under the orchestration of a parameter server (PS) without sharing clients' data.
SparseByteNN: A Novel Mobile Inference Acceleration Framework Based on Fine-Grained Group Sparsity
Our framework consists of two parts: (a) A fine-grained kernel sparsity schema with a sparsity granularity between structured pruning and unstructured pruning.
PC-bzip2: a phase-space continuity enhanced lossless compression algorithm for light field microscopy data
Moreover, by introducing the temporal continuity, our method shows the superior compression ratio on time series data of zebrafish blood vessels.
Benchmarking and Explaining Large Language Model-based Code Generation: A Causality-Centric Approach
We illustrate the insights that our framework can provide by studying over 3 popular LLMs with over 12 prompt adjustment strategies.
Forecasting Tropical Cyclones with Cascaded Diffusion Models
As tropical cyclones become more intense due to climate change, the rise of Al-based modelling provides a more affordable and accessible approach compared to traditional methods based on mathematical models.
Split and Merge: Aligning Position Biases in LLM-based Evaluators
Specifically, PORTIA splits the answers into multiple segments, aligns similar content across candidate answers, and then merges them back into a single prompt for evaluation by LLMs.
Exposing Image Splicing Traces in Scientific Publications via Uncertainty-guided Refinement
Compared to image duplication and synthesis, image splicing detection is more challenging due to the lack of reference images and the typically small tampered areas.
DualVC 2: Dynamic Masked Convolution for Unified Streaming and Non-Streaming Voice Conversion
Third, the model is unable to effectively address the noise in the unvoiced segments, lowering the sound quality.
AutoPrep: An Automatic Preprocessing Framework for In-the-Wild Speech Data
To address this issue, we introduce an automatic in-the-wild speech data preprocessing framework (AutoPrep) in this paper, which is designed to enhance speech quality, generate speaker labels, and produce transcriptions automatically.
Prototype-Enhanced Hypergraph Learning for Heterogeneous Information Networks
The variety and complexity of relations in multimedia data lead to Heterogeneous Information Networks (HINs).
Multiple Satellites Collaboration for Joint Code-aided CFOs and CPOs Estimation
Low Earth Orbit (LEO) satellites are being extensively researched in the development of secure Internet of Remote Things (IoRT).
Active Learning for Multilingual Fingerspelling Corpora
We apply active learning to help with data scarcity problems in sign languages.
Leveraging In-the-Wild Data for Effective Self-Supervised Pretraining in Speaker Recognition
Current speaker recognition systems primarily rely on supervised approaches, constrained by the scale of labeled datasets.
Agents: An Open-source Framework for Autonomous Language Agents
Recent advances on large language models (LLMs) enable researchers and developers to build autonomous language agents that can automatically solve various tasks and interact with environments, humans, and other agents using natural language interfaces.
Compiled Models, Built-In Exploits: Uncovering Pervasive Bit-Flip Attack Surfaces in DNN Executables
Defenses have also been proposed to guard model weights.
Enabling Runtime Verification of Causal Discovery Algorithms with Automated Conditional Independence Reasoning (Extended Version)
Based on the decision procedure to CIR, CICheck includes two variants: ED-CICheck and ED-CICheck, which detect erroneous CI tests (to enhance reliability) and prune excessive CI tests (to enhance privacy), respectively.
Generating Natural Language Queries for More Effective Systematic Review Screening Prioritisation
Our best approach is not only viable based on the information available at the time of screening, but also has similar effectiveness to the final title.
Integrated Robotics Networks with Co-optimization of Drone Placement and Air-Ground Communications
Terrestrial robots, i. e., unmanned ground vehicles (UGVs), and aerial robots, i. e., unmanned aerial vehicles (UAVs), operate in separate spaces.
Parsing is All You Need for Accurate Gait Recognition in the Wild
Furthermore, due to the lack of suitable datasets, we build the first parsing-based dataset for gait recognition in the wild, named Gait3D-Parsing, by extending the large-scale and challenging Gait3D dataset.
Least Squares Maximum and Weighted Generalization-Memorization Machines
Furthermore, we propose some different memory impact functions for the MIMM and WIMM.
Recursively Summarizing Enables Long-Term Dialogue Memory in Large Language Models
We evaluate our method on both open and closed LLMs, and the experiments on the widely-used public dataset show that our method can generate more consistent responses in a long-context conversation.
Compound Attention and Neighbor Matching Network for Multi-contrast MRI Super-resolution
Multi-contrast magnetic resonance imaging (MRI) reflects information about human tissue from different perspectives and has many clinical applications.
Communication Resources Constrained Hierarchical Federated Learning for End-to-End Autonomous Driving
While federated learning (FL) improves the generalization of end-to-end autonomous driving by model aggregation, the conventional single-hop FL (SFL) suffers from slow convergence rate due to long-range communications among vehicles and cloud server.
Wespeaker baselines for VoxSRC2023
This report showcases the results achieved using the wespeaker toolkit for the VoxSRC2023 Challenge.
Geometric Pooling: maintaining more useful information
However, by analyzing the statistical characteristic of activated units after pooling, we found that a large number of units dropped by sorting pooling are negative-value units that contain useful information and can contribute considerably to the final decision.
Towards Practical Federated Causal Structure Learning
Federated learning can solve this problem, but existing solutions for federated causal structure learning make unrealistic assumptions about data and lack convergence guarantees.
Precise and Generalized Robustness Certification for Neural Networks
We identify two key properties, independence and continuity, that convert the latent space into a precise and analysis-friendly input space representation for certification.
Enhancing Point Annotations with Superpixel and Confidence Learning Guided for Improving Semi-Supervised OCT Fluid Segmentation
Specifically, we use points to annotate fluid regions in unlabeled OCT images and the Superpixel-Guided Pseudo-Label Generation (SGPLG) module generates pseudo-labels and pixel-level label trust maps from the point annotations.
Taxonomy Expansion for Named Entity Recognition
We experiment with 6 diverse datasets and show that PLM consistently performs better than most other approaches (0. 5 - 2. 5 F1), including in novel settings for taxonomy expansion not considered in prior work.
Causality-Aided Trade-off Analysis for Machine Learning Fairness
This paper uses causality analysis as a principled method for analyzing trade-offs between fairness parameters and other crucial metrics in ML pipelines.
DualVC: Dual-mode Voice Conversion using Intra-model Knowledge Distillation and Hybrid Predictive Coding
Voice conversion is an increasingly popular technology, and the growing number of real-time applications requires models with streaming conversion capabilities.
A Weak Supervision Approach for Few-Shot Aspect Based Sentiment
We explore how weak supervision on abundant unlabeled data can be leveraged to improve few-shot performance in aspect-based sentiment analysis (ABSA) tasks.
Explain Any Concept: Segment Anything Meets Concept-Based Explanation
For computer vision tasks, mainstream pixel-based XAI methods explain DNN decisions by identifying important pixels, and emerging concept-based XAI explore forming explanations with concepts (e. g., a head in an image).
Adversarial Speaker Disentanglement Using Unannotated External Data for Self-supervised Representation Based Voice Conversion
Experimental results show that our proposed method achieves comparable similarity and higher naturalness than the supervised method, which needs a huge amount of annotated corpora for training and is applicable to improve similarity for VC methods with other SSL representations as input.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+4
Towards Generalizable Medical Image Segmentation with Pixel-wise Uncertainty Estimation
In contrast, the IID hypothesis is not universally guaranteed in numerous real-world applications, especially in medical image analysis.
Black-box Source-free Domain Adaptation via Two-stage Knowledge Distillation
In Stage \uppercase\expandafter{\romannumeral1}, we train the target model from scratch with soft pseudo-labels generated by the source model in a knowledge distillation manner.
"Oops, Did I Just Say That?" Testing and Repairing Unethical Suggestions of Large Language Models with Suggest-Critique-Reflect Process
Moreover, we propose a novel on-the-fly (OTF) repairing scheme that repairs unethical suggestions made by LLMs in real-time.
Meta-Reinforcement Learning Based on Self-Supervised Task Representation Learning
On MuJoCo and Meta-World benchmarks, MoSS outperforms prior works in terms of asymptotic performance, sample efficiency (3-50x faster), adaptation efficiency, and generalization robustness on broad and diverse task distributions.
DiffuseExpand: Expanding dataset for 2D medical image segmentation using diffusion models
Based on this insight, we propose an approach called DiffuseExpand for expanding datasets for 2D medical image segmentation using DPM, which first samples a variety of masks from Gaussian noise to ensure the diversity, and then synthesizes images to ensure the alignment of images and masks.
Towards Open-Vocabulary Video Instance Segmentation
Video Instance Segmentation (VIS) aims at segmenting and categorizing objects in videos from a closed set of training categories, lacking the generalization ability to handle novel categories in real-world videos.
Demonstration of InsightPilot: An LLM-Empowered Automated Data Exploration System
In brief, an IQuery is an abstraction and automation of data analysis operations, which mimics the approach of data analysts and simplifies the exploration process for users.
Simple Yet Effective Synthetic Dataset Construction for Unsupervised Opinion Summarization
Opinion summarization provides an important solution for summarizing opinions expressed among a large number of reviews.
Feature Alignment and Uniformity for Test Time Adaptation
Test time adaptation (TTA) aims to adapt deep neural networks when receiving out of distribution test domain samples.
Prototype Knowledge Distillation for Medical Segmentation with Missing Modality
Specifically, our ProtoKD can not only distillate the pixel-wise knowledge of multi-modality data to single-modality data but also transfer intra-class and inter-class feature variations, such that the student model could learn more robust feature representation from the teacher model and inference with only one single modality data.
Bootstrap The Original Latent: Learning a Private Model from a Black-box Model
In this paper, considering the balance of data/model privacy of model owners and user needs, we propose a new setting called Back-Propagated Black-Box Adaptation (BPBA) for users to better train their private models via the guidance of the back-propagated results of a Black-box foundation/source model.
Dynamic Benchmarking of Masked Language Models on Temporal Concept Drift with Multiple Views
Temporal concept drift refers to the problem of data changing over time.
Can ChatGPT Write a Good Boolean Query for Systematic Review Literature Search?
The ability of ChatGPT to follow complex instructions and generate queries with high precision makes it a valuable tool for researchers conducting systematic reviews, particularly for rapid reviews where time is a constraint and often trading-off higher precision for lower recall is acceptable.
Predictions of photophysical properties of phosphorescent platinum(II) complexes based on ensemble machine learning approach
Here, we develop a general protocol for accurate predictions of emission wavelength, radiative decay rate constant, and PL quantum yield for phosphorescent Pt(II) emitters based on the combination of first-principles quantum mechanical method, machine learning (ML) and experimental calibration.
SrTR: Self-reasoning Transformer with Visual-linguistic Knowledge for Scene Graph Generation
The execution efficiency of the one-stage scene graph generation approaches are quite high, which infer the effective relation between entity pairs using sparse proposal sets and a few queries.
Neural Rankers for Effective Screening Prioritisation in Medical Systematic Review Literature Search
An empirical analysis compares how effective neural methods compare to traditional methods for this task.
MeSH Suggester: A Library and System for MeSH Term Suggestion for Systematic Review Boolean Query Construction
One challenge to creating an effective systematic review Boolean query is the selection of effective MeSH Terms to include in the query.
Teaching What You Should Teach: A Data-Based Distillation Method
To be specific, we design a neural network-based data augmentation module with priori bias, which assists in finding what meets the teacher's strengths but the student's weaknesses, by learning magnitudes and probabilities to generate suitable data samples.
Beyond ADMM: A Unified Client-variance-reduced Adaptive Federated Learning Framework
In this paper, we firstly reveal the fact that the federated ADMM is essentially a client-variance-reduced algorithm.
Knowledge-Guided Exploration in Deep Reinforcement Learning
This paper proposes a new method to drastically speed up deep reinforcement learning (deep RL) training for problems that have the property of state-action permissibility (SAP).
Large-Scale Bandwidth and Power Optimization for Multi-Modal Edge Intelligence Autonomous Driving
However, wireless channels between the edge server and the autonomous vehicles are time-varying due to the high-mobility of vehicles.
Instruction Tuning for Few-Shot Aspect-Based Sentiment Analysis
Aspect-based Sentiment Analysis (ABSA) is a fine-grained sentiment analysis task which involves four elements from user-generated texts: aspect term, aspect category, opinion term, and sentiment polarity.
Aspect-Based Sentiment Analysis
Aspect-Based Sentiment Analysis (ABSA)
+3
Contrastive Training Improves Zero-Shot Classification of Semi-structured Documents
The standard classification setting where categories are fixed during both training and testing falls short in dynamic environments where new document categories could potentially emerge.
Decompiling x86 Deep Neural Network Executables
BTD takes DNN executables and outputs full model specifications, including types of DNN operators, network topology, dimensions, and parameters that are (nearly) identical to those of the input models.
Accelerated partial separable model using dimension-reduced optimization technique for ultra-fast cardiac MRI
The main objective of this study is to accelerate the PS model, shorten the time required for acquisition and reconstruction, and maintain good image quality simultaneously.
Automated MeSH Term Suggestion for Effective Query Formulation in Systematic Reviews Literature Search
However, identifying the correct MeSH terms to include in a query is difficult: information experts are often unfamiliar with the MeSH database and unsure about the appropriateness of MeSH terms for a query.
Language-aware Domain Generalization Network for Cross-Scene Hyperspectral Image Classification
Text information including extensive prior knowledge about land cover classes has been ignored in hyperspectral image classification (HSI) tasks.
Multi-Point Integrated Sensing and Communication: Fusion Model and Functionality Selection
Integrated sensing and communication (ISAC) represents a paradigm shift, where previously competing wireless transmissions are jointly designed to operate in harmony via the shared use of the hardware platform for improving the spectral and energy efficiencies.
Longitudinal Prediction of Postnatal Brain Magnetic Resonance Images via a Metamorphic Generative Adversarial Network
In this paper, we propose a deep learning framework to predict missing scans from acquired scans, catering to longitudinal infant studies.
XInsight: eXplainable Data Analysis Through The Lens of Causality
XInsight is a three-module, end-to-end pipeline designed to extract causal graphs, translate causal primitives into XDA semantics, and quantify the quantitative contribution of each explanation to a data fact.
Cross Vision-RF Gait Re-identification with Low-cost RGB-D Cameras and mmWave Radars
Human identification is a key requirement for many applications in everyday life, such as personalized services, automatic surveillance, continuous authentication, and contact tracing during pandemics, etc.
A Framework Based on Generational and Environmental Response Strategies for Dynamic Multi-objective Optimization
In this paper, a novel framework based on generational and environmental response strategies (FGERS) is proposed, in which response strategies are run both in the environmental change stage and the environmental static stage to obtain population evolution information of those both stages.
VIP-SLAM: An Efficient Tightly-Coupled RGB-D Visual Inertial Planar SLAM
Even if the plane parameters are involved in the optimization, we effectively simplify the back-end map by using planar structures.
Graph Information Aggregation Cross-Domain Few-Shot Learning for Hyperspectral Image Classification
The IDE-block is used to characterize and aggregate the intradomain nonlocal relationships and the interdomain feature and distribution similarities are captured in the CSA-block.
