Search Results for author:
Found 428 papers, 161 papers with code
AcTune: Uncertainty-Based Active Self-Training for Active Fine-Tuning of Pretrained Language Models
We develop AcTune, a new framework that improves the label efficiency of active PLM fine-tuning by unleashing the power of unlabeled data via self-training.
Prompt-Based Rule Discovery and Boosting for Interactive Weakly-Supervised Learning
Weakly-supervised learning (WSL) has shown promising results in addressing label scarcity on many NLP tasks, but manually designing a comprehensive, high-quality labeling rule set is tedious and difficult.
Transferring SLU Models in Novel Domains
When building models for novel natural language domains, a major challenge is the lack of data in the new domains, no matter whether the data is annotated or not.
GraphOmni: A Comprehensive and Extendable Benchmark Framework for Large Language Models on Graph-theoretic Tasks
In this paper, we presented GraphOmni, a comprehensive benchmark framework for systematically evaluating the graph reasoning capabilities of LLMs.
QualiSpeech: A Speech Quality Assessment Dataset with Natural Language Reasoning and Descriptions
To bridge this gap, we introduce QualiSpeech, a comprehensive low-level speech quality assessment dataset encompassing 11 key aspects and detailed natural language comments that include reasoning and contextual insights.
IAP: Improving Continual Learning of Vision-Language Models via Instance-Aware Prompting
Recent pre-trained vision-language models (PT-VLMs) often face a Multi-Domain Class-Incremental Learning (MCIL) scenario in practice, where several classes and domains of multi-modal tasks are incrementally arrived.
ACVUBench: Audio-Centric Video Understanding Benchmark
Audio often serves as an auxiliary modality in video understanding tasks of audio-visual large language models (LLMs), merely assisting in the comprehension of visual information.
Language Model Uncertainty Quantification with Attention Chain
To address this, we propose UQAC, an efficient method that narrows the reasoning space to a tractable size for marginalization.
Incomplete Multi-view Clustering via Diffusion Contrastive Generation
By performing contrastive learning on a limited set of paired multi-view samples, DCG can align the generated views with the real views, facilitating accurate recovery of views across arbitrary missing view scenarios.
YuE: Scaling Open Foundation Models for Long-Form Music Generation
We tackle the task of long-form music generation--particularly the challenging \textbf{lyrics-to-song} problem--by introducing YuE, a family of open foundation models based on the LLaMA2 architecture.
Every FLOP Counts: Scaling a 300B Mixture-of-Experts LING LLM without Premium GPUs
Ultimately, our experimental findings demonstrate that a 300B MoE LLM can be effectively trained on lower-performance devices while achieving comparable performance to models of a similar scale, including dense and MoE models.
Large-Scale AI in Telecom: Charting the Roadmap for Innovation, Scalability, and Enhanced Digital Experiences
This white paper discusses the role of large-scale AI in the telecommunications industry, with a specific focus on the potential of generative AI to revolutionize network functions and user experiences, especially in the context of 6G systems.
EAGLE-3: Scaling up Inference Acceleration of Large Language Models via Training-Time Test
We identify that this limitation arises from EAGLE's feature prediction constraints.
Low-Rank and Sparse Model Merging for Multi-Lingual Speech Recognition and Translation
Traditional multi-task training approaches aim to address this by jointly optimizing multiple speech recognition and translation tasks across various languages.
Class-Dependent Perturbation Effects in Evaluating Time Series Attributions
As machine learning models become increasingly prevalent in time series applications, Explainable Artificial Intelligence (XAI) methods are essential for understanding their predictions.
Explainable artificial intelligence
Explainable Artificial Intelligence (XAI)
+2
A Unified Modeling Framework for Automated Penetration Testing
The integration of artificial intelligence into automated penetration testing (AutoPT) has highlighted the necessity of simulation modeling for the training of intelligent agents, due to its cost-efficiency and swift feedback capabilities.
video-SALMONN-o1: Reasoning-enhanced Audio-visual Large Language Model
video-SALMONN-o1 achieves 3-8% accuracy improvements over the LLaVA-OneVision baseline across different video reasoning benchmarks.
Streamlining the Collaborative Chain of Models into A Single Forward Pass in Generation-Based Tasks
In Retrieval-Augmented Generation (RAG) and agent-based frameworks, the "Chain of Models" approach is widely used, where multiple specialized models work sequentially on distinct sub-tasks.
Hephaestus: Improving Fundamental Agent Capabilities of Large Language Models through Continual Pre-Training
Due to the scarcity of agent-oriented pre-training data, LLM-based autonomous agents typically rely on complex prompting or extensive fine-tuning, which often fails to introduce new capabilities while preserving strong generalizability.
MMGDreamer: Mixed-Modality Graph for Geometry-Controllable 3D Indoor Scene Generation
Controllable 3D scene generation has extensive applications in virtual reality and interior design, where the generated scenes should exhibit high levels of realism and controllability in terms of geometry.
Audio Large Language Models Can Be Descriptive Speech Quality Evaluators
Recent advances have enabled large language models (LLMs) to incorporate auditory systems for handling various speech-related tasks.
CE-SDWV: Effective and Efficient Concept Erasure for Text-to-Image Diffusion Models via a Semantic-Driven Word Vocabulary
Large-scale text-to-image (T2I) diffusion models have achieved remarkable generative performance about various concepts.
Hunyuan3D 2.0: Scaling Diffusion Models for High Resolution Textured 3D Assets Generation
This system includes two foundation components: a large-scale shape generation model -- Hunyuan3D-DiT, and a large-scale texture synthesis model -- Hunyuan3D-Paint.
The 1st SpeechWellness Challenge: Detecting Suicidal Risk Among Adolescents
The 1st SpeechWellness Challenge (SW1) aims to advance methods for detecting suicidal risk in adolescents using speech analysis techniques.
Detecting Defective Wafers Via Modular Networks
The growing availability of sensors within semiconductor manufacturing processes makes it feasible to detect defective wafers with data-driven models.
FedRSClip: Federated Learning for Remote Sensing Scene Classification Using Vision-Language Models
In this paper, we propose FedRSCLIP, the first federated learning framework designed for remote sensing image classification based on a VLM, specifically CLIP.
An Engorgio Prompt Makes Large Language Model Babble on
Auto-regressive large language models (LLMs) have yielded impressive performance in many real-world tasks.
Geometric Deep Learning for Realized Covariance Matrix Forecasting
Traditional methods employed in matrix volatility forecasting often overlook the inherent Riemannian manifold structure of symmetric positive definite matrices, treating them as elements of Euclidean space, which can lead to suboptimal predictive performance.
LatentSync: Audio Conditioned Latent Diffusion Models for Lip Sync
Since we did not change the overall training framework of SyncNet, our experience can also be applied to other lip sync and audio-driven portrait animation methods that utilize SyncNet.
Automated Dynamic Image Analysis for Particle Size and Shape Classification in Three Dimensions
Current state-of-the art instruments for dynamic image analysis are largely limited to two-dimensional imaging.
Implicit Priors Editing in Stable Diffusion via Targeted Token Adjustment
Given a "source" prompt (e. g., "rose") that elicits an implicit assumption (e. g., rose is red) and a "destination" prompt that specifies the desired attribute (e. g., "blue rose"), Embedit fine-tunes only the word token embedding (WTE) of the target object ("rose") to optimize the last hidden state of text encoder in Stable Diffusion, a SOTA text-to-image model.
SALMONN-omni: A Codec-free LLM for Full-duplex Speech Understanding and Generation
Unlike traditional modularised conversational AI systems, which separate speech recognition, understanding, and text-to-speech generation into distinct components, multimodal LLMs operate as single end-to-end models.
Self-Generated Critiques Boost Reward Modeling for Language Models
Reward modeling is crucial for aligning large language models (LLMs) with human preferences, especially in reinforcement learning from human feedback (RLHF).
ScalingNote: Scaling up Retrievers with Large Language Models for Real-World Dense Retrieval
However, scaling up retrieval models significantly increases online query latency.
Adversarial Attacks Using Differentiable Rendering: A Survey
Differentiable rendering methods have emerged as a promising means for generating photo-realistic and physically plausible adversarial attacks by manipulating 3D objects and scenes that can deceive deep neural networks (DNNs).
Fast Disentangled Slim Tensor Learning for Multi-view Clustering
To alleviate the negative influence of feature redundancy, inspired by robust PCA, DSTL disentangles the latent low-dimensional representation into a semantic-unrelated part and a semantic-related part for each view.
Matryoshka: Learning to Drive Black-Box LLMs with LLMs
To address this challenge, we introduce Matryoshika, a lightweight white-box LLM controller that guides a large-scale black-box LLM generator by decomposing complex tasks into a series of intermediate outputs.
BELM: Bidirectional Explicit Linear Multi-step Sampler for Exact Inversion in Diffusion Models
In this paper, we introduce a generic formulation, \emph{Bidirectional Explicit Linear Multi-step} (BELM) samplers, of the exact inversion samplers, which includes all previously proposed heuristic exact inversion samplers as special cases.
Enhancing Multimodal LLM for Detailed and Accurate Video Captioning using Multi-Round Preference Optimization
To address potential catastrophic forgetting of non-captioning abilities due to mrDPO, we propose rebirth tuning, which finetunes the pre-DPO LLM by using the captions generated by the mrDPO-trained model as supervised labels.
Editing Music with Melody and Text: Using ControlNet for Diffusion Transformer
For more precise and fine-grained melody control, we introduce a novel top-$k$ constant-Q Transform representation as the melody prompt, reducing ambiguity compared to previous representations (e. g., chroma), particularly for music with multiple tracks or a wide range of pitch values.
LoRC: Low-Rank Compression for LLMs KV Cache with a Progressive Compression Strategy
Existing approaches to mitigate this issue include: (1) efficient attention variants integrated in upcycling stages, which requires extensive parameter tuning thus unsuitable for pre-trained LLMs; (2) KV cache compression at test time, primarily through token eviction policies, which often overlook inter-layer dependencies and can be task-specific.
SWIM: Short-Window CNN Integrated with Mamba for EEG-Based Auditory Spatial Attention Decoding
By joint training SW$_\text{CNN}$ and Mamba, the proposed SWIM structure uses both short-term and long-term information and achieves an accuracy of 86. 2%, which reduces the classification errors by a relative 31. 0% compared to the previous state-of-the-art result.
LW2G: Learning Whether to Grow for Prompt-based Continual Learning
To promote cross-task knowledge facilitation and form an effective and efficient prompt sets pool, we propose a plug-in module in the former stage to \textbf{Learn Whether to Grow (LW2G)} based on the disparities between tasks.
Enabling Auditory Large Language Models for Automatic Speech Quality Evaluation
The results demonstrate that auditory LLMs achieve competitive performance compared to state-of-the-art task-specific small models in predicting MOS and SIM, while also delivering promising results in A/B testing and natural language descriptions.
MT2KD: Towards A General-Purpose Encoder for Speech, Speaker, and Audio Events
In this work, MT2KD, a novel two-stage multi-task learning framework is proposed to build a general-purpose speech and audio encoder that jointly performs three fundamental tasks: automatic speech recognition (ASR), audio tagging (AT) and speaker verification (SV).
Beyond Relevance: Improving User Engagement by Personalization for Short-Video Search
Personalized search has been extensively studied in various applications, including web search, e-commerce, social networks, etc.
Large Language Model Based Generative Error Correction: A Challenge and Baselines for Speech Recognition, Speaker Tagging, and Emotion Recognition
Given recent advances in generative AI technology, a key question is how large language models (LLMs) can enhance acoustic modeling tasks using text decoding results from a frozen, pretrained automatic speech recognition (ASR) model.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+5
Extract and Diffuse: Latent Integration for Improved Diffusion-based Speech and Vocal Enhancement
Diffusion-based generative models have recently achieved remarkable results in speech and vocal enhancement due to their ability to model complex speech data distributions.
RNR: Teaching Large Language Models to Follow Roles and Rules
However, existing models trained on open-source IFT datasets only have the ability to follow instructions from users, and often fail to follow complex role and rules specified by developers, a. k. a.
TextToucher: Fine-Grained Text-to-Touch Generation
We model these granularities of information through text descriptions and propose a fine-grained Text-to-Touch generation method (TextToucher) to generate high-quality tactile samples.
SA-MLP: A Low-Power Multiplication-Free Deep Network for 3D Point Cloud Classification in Resource-Constrained Environments
Point cloud classification plays a crucial role in the processing and analysis of data from 3D sensors such as LiDAR, which are commonly used in applications like autonomous vehicles, robotics, and environmental monitoring.
Efficient Point Cloud Classification via Offline Distillation Framework and Negative-Weight Self-Distillation Technique
To address these challenges, we introduce an innovative offline recording strategy that avoids the simultaneous loading of both teacher and student models, thereby reducing hardware demands.
PMT-MAE: Dual-Branch Self-Supervised Learning with Distillation for Efficient Point Cloud Classification
This paper introduces PMT-MAE (Point MLP-Transformer Masked Autoencoder), a novel self-supervised learning framework for point cloud classification.
Tangram: Benchmark for Evaluating Geometric Element Recognition in Large Multimodal Models
Significant advancements in Large Multimodal Models (LMMs) have enabled them to tackle complex problems involving visual-mathematical reasoning.
Enhancing Audio-Language Models through Self-Supervised Post-Training with Text-Audio Pairs
In this paper, we propose to equip the multi-modal ALMs with temporal understanding without loosing their inherent prior capabilities of audio-language tasks with a temporal instillation method TeminAL.
ViC: Virtual Compiler Is All You Need For Assembly Code Search
Assembly code search is vital for reducing the burden on reverse engineers, allowing them to quickly identify specific functions using natural language within vast binary programs.
PointMT: Efficient Point Cloud Analysis with Hybrid MLP-Transformer Architecture
In recent years, point cloud analysis methods based on the Transformer architecture have made significant progress, particularly in the context of multimedia applications such as 3D modeling, virtual reality, and autonomous systems.
Strategic Federated Learning: Application to Smart Meter Data Clustering
In this paper, we introduce a novel FL framework in which the FC uses an aggregate version of the MI to make decisions that affect the client's utility functions.
Confidence Estimation for Automatic Detection of Depression and Alzheimer's Disease Based on Clinical Interviews
Speech-based automatic detection of Alzheimer's disease (AD) and depression has attracted increased attention.
ChipExpert: The Open-Source Integrated-Circuit-Design-Specific Large Language Model
Finally, to mitigate the hallucinations of ChipExpert, we have developed a Retrieval-Augmented Generation (RAG) system, based on the IC design knowledge base.
DriveDiTFit: Fine-tuning Diffusion Transformers for Autonomous Driving
Such datasets are expected to cover various driving scenarios with adverse weather, lighting conditions and diverse moving objects.
Goal-Oriented State Information Compression for Linear Dynamical System Control
In this paper, we consider controlled linear dynamical systems in which the controller has only access to a compressed version of the system state.
Generative AI for RF Sensing in IoT systems
The development of wireless sensing technologies, using signals such as Wi-Fi, infrared, and RF to gather environmental data, has significantly advanced within Internet of Things (IoT) systems.
PECTP: Parameter-Efficient Cross-Task Prompts for Incremental Vision Transformer
Prompt-fixed methods only learn a single set of prompts on one of the incremental tasks and can not handle all the incremental tasks effectively.
RankRAG: Unifying Context Ranking with Retrieval-Augmented Generation in LLMs
Large language models (LLMs) typically utilize the top-k contexts from a retriever in retrieval-augmented generation (RAG).
Ranked #3 on
Question Answering
on PubMedQA
Learning Graph Structures and Uncertainty for Accurate and Calibrated Time-series Forecasting
Recent works model the relations between time-series as graphs and have shown that propagating information over the relation graph can improve time series forecasting.
SOT Triggered Neural Clustering for Speaker Attributed ASR
This is achieved by the use of ASR, trained using a serialised output training method, together with segment-level discriminative neural clustering (SDNC) to assign speaker labels.
Robust Zero-Shot Text-to-Speech Synthesis with Reverse Inference Optimization
By leveraging reverse inference as the standard to select exemplars used in RLHF from the speech samples generated by the TTS system itself, RIO steers the subsequent optimization towards a direction of enhancing the TTS robustness.
PointViG: A Lightweight GNN-based Model for Efficient Point Cloud Analysis
These results underscore the potential and efficiency of PointViG in point cloud analysis.
Large Language Models for Power Scheduling: A User-Centric Approach
While traditional optimization and scheduling schemes are designed to meet fixed, predefined system requirements, future systems are moving toward user-driven approaches and personalized services, aiming to achieve high quality-of-experience (QoE) and flexibility.
EAGLE-2: Faster Inference of Language Models with Dynamic Draft Trees
Inference with modern Large Language Models (LLMs) is expensive and time-consuming, and speculative sampling has proven to be an effective solution.
Efficient Evolutionary Search Over Chemical Space with Large Language Models
Molecular discovery, when formulated as an optimization problem, presents significant computational challenges because optimization objectives can be non-differentiable.
video-SALMONN: Speech-Enhanced Audio-Visual Large Language Models
To obtain fine-grained temporal information required by speech understanding, while keeping efficient for other video elements, this paper proposes a novel multi-resolution causal Q-Former (MRC Q-Former) structure to connect pre-trained audio-visual encoders and the backbone large language model.
Retrieve-Plan-Generation: An Iterative Planning and Answering Framework for Knowledge-Intensive LLM Generation
Despite the significant progress of large language models (LLMs) in various tasks, they often produce factual errors due to their limited internal knowledge.
LLMatDesign: Autonomous Materials Discovery with Large Language Models
Discovering new materials can have significant scientific and technological implications but remains a challenging problem today due to the enormity of the chemical space.
NTIRE 2024 Challenge on Night Photography Rendering
The top ranking participants' solutions effectively represent the state-of-the-art in nighttime photography rendering.
Time-Series Forecasting for Out-of-Distribution Generalization Using Invariant Learning
In this paper, we aim to alleviate the inherent OOD problem in TSF via invariant learning.
Time-MMD: Multi-Domain Multimodal Dataset for Time Series Analysis
This oversight is due to the untapped potential of textual series data and the absence of a comprehensive, high-quality multimodal dataset.
From a Social Cognitive Perspective: Context-aware Visual Social Relationship Recognition
People's social relationships are often manifested through their surroundings, with certain objects or interactions acting as symbols for specific relationships, e. g., wedding rings, roses, hugs, or holding hands.
An Improved Empirical Fisher Approximation for Natural Gradient Descent
Approximate Natural Gradient Descent (NGD) methods are an important family of optimisers for deep learning models, which use approximate Fisher information matrices to pre-condition gradients during training.
Aligning Large Language Models with Representation Editing: A Control Perspective
To achieve alignment for specific objectives, we introduce external control signals into the state space of this language dynamical system.
Spontaneous Speech-Based Suicide Risk Detection Using Whisper and Large Language Models
The early detection of suicide risk is important since it enables the intervention to prevent potential suicide attempts.
HYDRA: Model Factorization Framework for Black-Box LLM Personalization
To address these challenges, we propose HYDRA, a model factorization framework that captures both user-specific behavior patterns from historical data and shared general knowledge among all users to deliver personalized generation.
PLaD: Preference-based Large Language Model Distillation with Pseudo-Preference Pairs
Through extensive experiments on two sequence generation tasks and with various LLMs, we demonstrate the effectiveness of our proposed PLaD framework.
GeminiFusion: Efficient Pixel-wise Multimodal Fusion for Vision Transformer
Cross-modal transformers have demonstrated superiority in various vision tasks by effectively integrating different modalities.
Ranked #1 on
Semantic Segmentation
on SUN-RGBD
Enhancing Zero-shot Text-to-Speech Synthesis with Human Feedback
In recent years, text-to-speech (TTS) technology has witnessed impressive advancements, particularly with large-scale training datasets, showcasing human-level speech quality and impressive zero-shot capabilities on unseen speakers.
MSSC-BiMamba: Multimodal Sleep Stage Classification and Early Diagnosis of Sleep Disorders with Bidirectional Mamba
Our model, which can effectively handle diverse sleep conditions, is the first to apply BiMamba to sleep staging with multimodal PSG data, showing substantial gains in computational and memory efficiency over traditional Transformer-style models.
NoteLLM-2: Multimodal Large Representation Models for Recommendation
While leveraging existing Multimodal Large Language Models (MLLMs) for such tasks is promising, challenges arise due to their delayed release compared to corresponding LLMs and the inefficiency in representation tasks.
Incremental Pseudo-Labeling for Black-Box Unsupervised Domain Adaptation
Black-Box unsupervised domain adaptation (BBUDA) learns knowledge only with the prediction of target data from the source model without access to the source data and source model, which attempts to alleviate concerns about the privacy and security of data.
Bayesian WeakS-to-Strong from Text Classification to Generation
Advances in large language models raise the question of how alignment techniques will adapt as models become increasingly complex and humans will only be able to supervise them weakly.
Self-Taught Recognizer: Toward Unsupervised Adaptation for Speech Foundation Models
We propose an unsupervised adaptation framework, Self-TAught Recognizer (STAR), which leverages unlabeled data to enhance the robustness of automatic speech recognition (ASR) systems in diverse target domains, such as noise and accents.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+1
CrossCheckGPT: Universal Hallucination Ranking for Multimodal Foundation Models
Multimodal foundation models are prone to hallucination, generating outputs that either contradict the input or are not grounded by factual information.
Hunyuan-DiT: A Powerful Multi-Resolution Diffusion Transformer with Fine-Grained Chinese Understanding
For fine-grained language understanding, we train a Multimodal Large Language Model to refine the captions of the images.
Goal-oriented compression for $L_p$-norm-type goal functions: Application to power consumption scheduling
Motivated by applications such as the smart grid, this paper focuses on a goal function which is of $L_p$-norm-type.
BMRetriever: Tuning Large Language Models as Better Biomedical Text Retrievers
Developing effective biomedical retrieval models is important for excelling at knowledge-intensive biomedical tasks but still challenging due to the deficiency of sufficient publicly annotated biomedical data and computational resources.
Frequency-Guided Multi-Level Human Action Anomaly Detection with Normalizing Flows
We introduce the task of human action anomaly detection (HAAD), which aims to identify anomalous motions in an unsupervised manner given only the pre-determined normal category of training action samples.
Bayesian Example Selection Improves In-Context Learning for Speech, Text, and Visual Modalities
Large language models (LLMs) can adapt to new tasks through in-context learning (ICL) based on a few examples presented in dialogue history without any model parameter update.
Emerging Advancements in 6G NTN Radio Access Technologies: An Overview
The efforts on the development, standardization and improvements to communication systems towards 5G Advanced and 6G are on track to provide benefits such as an unprecedented level of connectivity and performance, enabling a diverse range of vertical services.
Cepstral Analysis Based Artifact Detection, Recognition and Removal for Prefrontal EEG
The proposed method achieves an accuracy of 99. 62% on the artifact detection task and a 82. 79% accuracy on the 6-category eye movement classification task.
Empowering Image Recovery_ A Multi-Attention Approach
Integration of attention mechanisms across feature and positional dimensions further enhances the recovery of fine details.
Semantic Map-based Generation of Navigation Instructions
In this paper, we propose a new approach to navigation instruction generation by framing the problem as an image captioning task using semantic maps as visual input.
Leveraging Large Language Model to Generate a Novel Metaheuristic Algorithm with CRISPE Framework
In this paper, we borrow the large language model (LLM) ChatGPT-3. 5 to automatically and quickly design a new metaheuristic algorithm (MA) with only a small amount of input.
M$^3$AV: A Multimodal, Multigenre, and Multipurpose Audio-Visual Academic Lecture Dataset
Although multiple academic video datasets have been constructed and released, few of them support both multimodal content recognition and understanding tasks, which is partially due to the lack of high-quality human annotations.
ProgGen: Generating Named Entity Recognition Datasets Step-by-step with Self-Reflexive Large Language Models
Although Large Language Models (LLMs) exhibit remarkable adaptability across domains, these models often fall short in structured knowledge extraction tasks such as named entity recognition (NER).
Efficient Multiplayer Battle Game Optimizer for Adversarial Robust Neural Architecture Search
This paper introduces a novel metaheuristic algorithm, known as the efficient multiplayer battle game optimizer (EMBGO), specifically designed for addressing complex numerical optimization tasks.
DiaLoc: An Iterative Approach to Embodied Dialog Localization
However, most existing works in embodied dialog research focus on navigation and leave the localization task understudied.
NoteLLM: A Retrievable Large Language Model for Note Recommendation
Indeed, learning to generate hashtags/categories can potentially enhance note embeddings, both of which compress key note information into limited content.
APISR: Anime Production Inspired Real-World Anime Super-Resolution
In addition, we identify two anime-specific challenges of distorted and faint hand-drawn lines and unwanted color artifacts.
Accelerating materials discovery for polymer solar cells: Data-driven insights enabled by natural language processing
We present a simulation of various active learning strategies for the discovery of polymer solar cell donor/acceptor pairs using data extracted from the literature spanning $\sim$20 years by a natural language processing pipeline.
Diffusion Models as Constrained Samplers for Optimization with Unknown Constraints
To constrain the optimization process to the data manifold, we reformulate the original optimization problem as a sampling problem from the product of the Boltzmann distribution defined by the objective function and the data distribution learned by the diffusion model.
CLAP: Learning Transferable Binary Code Representations with Natural Language Supervision
At the core, our approach boosts superior transfer learning capabilities by effectively aligning binary code with their semantics explanations (in natural language), resulting a model able to generate better embeddings for binary code.
A Simple but Effective Approach to Improve Structured Language Model Output for Information Extraction
It breaks the generation into a two-step pipeline: initially, LLMs generate answers in natural language as intermediate responses.
Handling Ambiguity in Emotion: From Out-of-Domain Detection to Distribution Estimation
The evidential uncertainty measure is extended to quantify the uncertainty in emotion distribution estimation.
BBox-Adapter: Lightweight Adapting for Black-Box Large Language Models
BBox-Adapter distinguishes target and source domain data by treating target data as positive and source data as negative.
EAGLE: Speculative Sampling Requires Rethinking Feature Uncertainty
In this paper, we reconsider speculative sampling and derive two key observations.
Neural Sinkhorn Gradient Flow
We utilize the velocity field matching training scheme in NSGF, which only requires samples from the source and target distribution to compute an empirical velocity field approximation.
LightSleepNet: Design of a Personalized Portable Sleep Staging System Based on Single-Channel EEG
This paper proposed LightSleepNet - a light-weight, 1-d Convolutional Neural Network (CNN) based personalized architecture for real-time sleep staging, which can be implemented on various mobile platforms with limited hardware resources.
TPD: Enhancing Student Language Model Reasoning via Principle Discovery and Guidance
The teacher LLM generates problem-solving instructions and corrective principles based on the student LLM's errors.
HiGen: Hierarchy-Aware Sequence Generation for Hierarchical Text Classification
The best-performing models aim to learn a static representation by combining document and hierarchical label information.
Large Language Models are Efficient Learners of Noise-Robust Speech Recognition
To this end, we propose to extract a language-space noise embedding from the N-best list to represent the noise conditions of source speech, which can promote the denoising process in GER.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+6
FinSQL: Model-Agnostic LLMs-based Text-to-SQL Framework for Financial Analysis
Text-to-SQL, which provides zero-code interface for operating relational databases, has gained much attention in financial analysis; because, financial professionals may not well-skilled in SQL programming.
Misconfidence-based Demonstration Selection for LLM In-Context Learning
In each step, it analyzes a pool of candidate examples and identifies the ones most likely to challenge the LLM's current understanding, measured by a new metric called misconfidence.
Multi-Channel Multi-Domain based Knowledge Distillation Algorithm for Sleep Staging with Single-Channel EEG
The proposed algorithm achieves a state-of-the-art single-channel sleep staging accuracy of 86. 5%, with only 0. 6% deterioration from the state-of-the-art multi-channel model.
A Closed-loop Brain-Machine Interface SoC Featuring a 0.2$μ$J/class Multiplexer Based Neural Network
This work presents the first fabricated electrophysiology-optogenetic closed-loop bidirectional brain-machine interface (CL-BBMI) system-on-chip (SoC) with electrical neural signal recording, on-chip sleep staging and optogenetic stimulation.
3DMIT: 3D Multi-modal Instruction Tuning for Scene Understanding
The remarkable potential of multi-modal large language models (MLLMs) in comprehending both vision and language information has been widely acknowledged.
Towards Modeling Uncertainties of Self-explaining Neural Networks via Conformal Prediction
To tackle the aforementioned challenges, in this paper, we design a novel uncertainty modeling framework for self-explaining networks, which not only demonstrates strong distribution-free uncertainty modeling performance for the generated explanations in the interpretation layer but also excels in producing efficient and effective prediction sets for the final predictions based on the informative high-level basis explanations.
Multiplayer Battle Game-Inspired Optimizer for Complex Optimization Problems
Various popular multiplayer battle royale games share a lot of common elements.
Two-Step Offline Preference-Based Reinforcement Learning with Constrained Actions
To overcome the challenge, our insight is that both challenges come from the state-actions not supported in the dataset.
Large Language Models for Generative Information Extraction: A Survey
Information extraction (IE) aims to extract structural knowledge from plain natural language texts.
GAD-PVI: A General Accelerated Dynamic-Weight Particle-Based Variational Inference Framework
Particle-based Variational Inference (ParVI) methods approximate the target distribution by iteratively evolving finite weighted particle systems.
A Joint Multi-Gradient Algorithm for Demosaicing Bayer Images
Experiments show that the algorithm in this paper has better recovery in image edges as well as texture complex regions with higher PSNR and SSIM values and better subjective visual perception compared to the traditional gradient algorithms such as BI, Cok, Hibbard, Laroche, Hamiton, while the algorithm involves only the add-subtract and shift operations, which is suitable to be implemented on the hardware platform.
Multilevel Saliency-Guided Self-Supervised Learning for Image Anomaly Detection
To fully exploit saliency guidance, on each map, we select a pixel pair from the cluster with the highest centroid saliency to form a patch pair.
LanGWM: Language Grounded World Model
Furthermore, our proposed technique of explicit language-grounded visual representation learning has the potential to improve models for human-robot interaction because our extracted visual features are language grounded.
How Far Have We Gone in Vulnerability Detection Using Large Language Models
Given the significant successes of large language models (LLMs) in various tasks, there is growing anticipation of their efficacy in vulnerability detection.
Data Diversity Matters for Robust Instruction Tuning
QDIT provides a simple method to simultaneously control dataset diversity and quality, allowing us to conduct an in-depth study on the effect of diversity and quality on instruction tuning performance.
Speech-based Slot Filling using Large Language Models
Recently, advancements in large language models (LLMs) have shown an unprecedented ability across various language tasks.
Explanation-aware Soft Ensemble Empowers Large Language Model In-context Learning
To fully unleash the power of explanations, we propose EASE, an Explanation-Aware Soft Ensemble framework to empower in-context learning with LLMs.
Assessing Logical Puzzle Solving in Large Language Models: Insights from a Minesweeper Case Study
Large Language Models (LLMs) have shown remarkable proficiency in language understanding and have been successfully applied to a variety of real-world tasks through task-specific fine-tuning or prompt engineering.
PolyIE: A Dataset of Information Extraction from Polymer Material Scientific Literature
Scientific information extraction (SciIE), which aims to automatically extract information from scientific literature, is becoming more important than ever.
Image-Pointcloud Fusion based Anomaly Detection using PD-REAL Dataset
We present PD-REAL, a novel large-scale dataset for unsupervised anomaly detection (AD) in the 3D domain.
Improving MIMO channel estimation via receive power feedback
Estimating the channel state is known to be an important problem in wireless networks.
SMURF-THP: Score Matching-based UnceRtainty quantiFication for Transformer Hawkes Process
We conduct extensive experiments in both event type prediction and uncertainty quantification of arrival time.
Orientation-Aware Leg Movement Learning for Action-Driven Human Motion Prediction
Specifically, we follow a two-stage forecasting strategy by first employing the motion diffusion model to generate the target motion with a specified future action, and then producing the in-betweening to smoothly connect the observation and prediction to eventually address motion prediction.
ToolChain*: Efficient Action Space Navigation in Large Language Models with A* Search
It formulates the entire action space as a decision tree, where each node represents a possible API function call involved in a solution plan.
SALMONN: Towards Generic Hearing Abilities for Large Language Models
Hearing is arguably an essential ability of artificial intelligence (AI) agents in the physical world, which refers to the perception and understanding of general auditory information consisting of at least three types of sounds: speech, audio events, and music.
When Rigidity Hurts: Soft Consistency Regularization for Probabilistic Hierarchical Time Series Forecasting
We close both these gap and propose PROFHiT, which is a fully probabilistic hierarchical forecasting model that jointly models forecast distribution of entire hierarchy.
Query-dominant User Interest Network for Large-Scale Search Ranking
The overall historical behaviors are various but noisy while search behaviors are always sparse.
Fine-grained Audio-Visual Joint Representations for Multimodal Large Language Models
Audio-visual large language models (LLM) have drawn significant attention, yet the fine-grained combination of both input streams is rather under-explored, which is challenging but necessary for LLMs to understand general video inputs.
Conditional Diffusion Model for Target Speaker Extraction
For the reverse-time process, a parametrised score function is conditioned on a target speaker embedding to extract the target speaker from the mixture of sources.
Transferring speech-generic and depression-specific knowledge for Alzheimer's disease detection
Apart from the knowledge from speech-generic representations, this paper also proposes to simultaneously transfer the knowledge from a speech depression detection task based on the high comorbidity rates of depression and AD.
Joint Projection Learning and Tensor Decomposition Based Incomplete Multi-view Clustering
We further consider the graph noise of projected data caused by missing samples and use a tensor-decomposition based graph filter for robust clustering. JPLTD decomposes the original tensor into an intrinsic tensor and a sparse tensor.
Multi-Dimension-Embedding-Aware Modality Fusion Transformer for Psychiatric Disorder Clasification
In this work, we proposed a multi-dimension-embedding-aware modality fusion transformer (MFFormer) for schizophrenia and bipolar disorder classification using rs-fMRI and T1 weighted structural MRI (T1w sMRI).
Adapting LLM Agents with Universal Feedback in Communication
To optimize agent interactions for task-specific learning with our universal buffer and pipeline, we introduce diverse communication patterns tailored for both single-agent and multi-agent environments.
It HAS to be Subjective: Human Annotator Simulation via Zero-shot Density Estimation
Human annotator simulation (HAS) serves as a cost-effective substitute for human evaluation such as data annotation and system assessment.
Subspace-Guided Feature Reconstruction for Unsupervised Anomaly Localization
In this paper, we propose a novel subspace-guided feature reconstruction framework to pursue adaptive feature approximation for anomaly localization.
Connecting Speech Encoder and Large Language Model for ASR
Q-Former-based LLMs can generalise well to out-of-domain datasets, where 12% relative WER reductions over the Whisper baseline ASR model were achieved on the Eval2000 test set without using any in-domain training data from Switchboard.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+4
Affect Recognition in Conversations Using Large Language Models
Affect recognition, encompassing emotions, moods, and feelings, plays a pivotal role in human communication.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+2
Enhancing Quantised End-to-End ASR Models via Personalisation
Recent end-to-end automatic speech recognition (ASR) models have become increasingly larger, making them particularly challenging to be deployed on resource-constrained devices.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+2
A Multi-In and Multi-Out Dendritic Neuron Model and its Optimization
Recently, a novel mathematical ANN model, known as the dendritic neuron model (DNM), has been proposed to address nonlinear problems by more accurately reflecting the structure of real neurons.
Can Whisper perform speech-based in-context learning?
Language-level adaptation experiments using Chinese dialects showed that when applying SICL to isolated word ASR, consistent and considerable relative WER reductions can be achieved using Whisper models of any size on two dialects, which is on average 32. 3%.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+3
RAIN: Your Language Models Can Align Themselves without Finetuning
We discover that by integrating self-evaluation and rewind mechanisms, unaligned LLMs can directly produce responses consistent with human preferences via self-boosting.
AGMDT: Virtual Staining of Renal Histology Images with Adjacency-Guided Multi-Domain Transfer
Renal pathology, as the gold standard of kidney disease diagnosis, requires doctors to analyze a series of tissue slices stained by H&E staining and special staining like Masson, PASM, and PAS, respectively.
Cross-Utterance Conditioned VAE for Speech Generation
Experimental results on the LibriTTS datasets demonstrate that our proposed models significantly enhance speech synthesis and editing, producing more natural and expressive speech.
PolyGET: Accelerating Polymer Simulations by Accurate and Generalizable Forcefield with Equivariant Transformer
Machine learning (ML) forcefields have been developed to achieve both the accuracy of ab initio methods and the efficiency of empirical force fields.
Situated Natural Language Explanations
Situated NLE provides a perspective and facilitates further research on the generation and evaluation of explanations.
kTrans: Knowledge-Aware Transformer for Binary Code Embedding
By feeding explicit knowledge as additional inputs to the Transformer, and fusing implicit knowledge with a novel pre-training task, kTrans provides a new perspective to incorporating domain knowledge into a Transformer framework.
Integrating Emotion Recognition with Speech Recognition and Speaker Diarisation for Conversations
Two metrics are proposed to evaluate AER performance with automatic segmentation based on time-weighted emotion and speaker classification errors.
One-bit Flip is All You Need: When Bit-flip Attack Meets Model Training
We propose a training-assisted bit flip attack, in which the adversary is involved in the training stage to build a high-risk model to release.
DF2: Distribution-Free Decision-Focused Learning
However, existing end-to-end DFL methods are hindered by three significant bottlenecks: model mismatch error, sample average approximation error, and gradient approximation error.
Alexa, play with robot: Introducing the First Alexa Prize SimBot Challenge on Embodied AI
The Alexa Prize program has empowered numerous university students to explore, experiment, and showcase their talents in building conversational agents through challenges like the SocialBot Grand Challenge and the TaskBot Challenge.
Revisiting DETR Pre-training for Object Detection
Motivated by the remarkable achievements of DETR-based approaches on COCO object detection and segmentation benchmarks, recent endeavors have been directed towards elevating their performance through self-supervised pre-training of Transformers while preserving a frozen backbone.
Graph Neural Networks for Forecasting Multivariate Realized Volatility with Spillover Effects
We present a novel methodology for modeling and forecasting multivariate realized volatilities using customized graph neural networks to incorporate spillover effects across stocks.
Understanding Deep Neural Networks via Linear Separability of Hidden Layers
In this paper, we measure the linear separability of hidden layer outputs to study the characteristics of deep neural networks.
Autoregressive Diffusion Model for Graph Generation
However, existing diffusion-based graph generative models are mostly one-shot generative models that apply Gaussian diffusion in the dequantized adjacency matrix space.
C3: Zero-shot Text-to-SQL with ChatGPT
This paper proposes a ChatGPT-based zero-shot Text-to-SQL method, dubbed C3, which achieves 82. 3\% in terms of execution accuracy on the holdout test set of Spider and becomes the state-of-the-art zero-shot Text-to-SQL method on the Spider Challenge.
Ranked #7 on
Text-To-SQL
on spider
Graph Contrastive Learning with Multi-Objective for Personalized Product Retrieval in Taobao Search
Graph-based CF methods improve personalization by modeling collaborative signal within the user click graph.
Knowledge-Aware Audio-Grounded Generative Slot Filling for Limited Annotated Data
In this work, we propose a Knowledge-Aware Audio-Grounded generative slot-filling framework, termed KA2G, that focuses on few-shot and zero-shot slot filling for ToD with speech input.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+6
Towards Optimal Randomized Strategies in Adversarial Example Game
The vulnerability of deep neural network models to adversarial example attacks is a practical challenge in many artificial intelligence applications.
A Solution to CVPR'2023 AQTC Challenge: Video Alignment for Multi-Step Inference
In this paper, we present a solution for enhancing video alignment to improve multi-step inference.
G-STO: Sequential Main Shopping Intention Detection via Graph-Regularized Stochastic Transformer
The detection of the underlying shopping intentions of users based on their historical interactions is a crucial aspect for e-commerce platforms, such as Amazon, to enhance the convenience and efficiency of their customers' shopping experiences.
ToolQA: A Dataset for LLM Question Answering with External Tools
To address this issue, we introduce a new dataset called ToolQA, which is designed to faithfully evaluate LLMs' ability to use external tools for question answering.
Pushing the Limits of Unsupervised Unit Discovery for SSL Speech Representation
Our models outperform other SSL models significantly on the LibriSpeech benchmark without the need for iterative re-clustering and re-training.
MUBen: Benchmarking the Uncertainty of Molecular Representation Models
While some studies have included UQ to improve molecular pre-trained models, the process of selecting suitable backbone and UQ methods for reliable molecular uncertainty estimation remains underexplored.
PACER: A Fully Push-forward-based Distributional Reinforcement Learning Algorithm
As a result, PACER fully utilizes the modeling capability of the push-forward operator and is able to explore a broader class of the policy space, compared with limited policy classes used in existing distributional actor critic algorithms (i. e. Gaussians).
Estimating the Uncertainty in Emotion Attributes using Deep Evidential Regression
In automatic emotion recognition (AER), labels assigned by different human annotators to the same utterance are often inconsistent due to the inherent complexity of emotion and the subjectivity of perception.
TrajectoryFormer: 3D Object Tracking Transformer with Predictive Trajectory Hypotheses
3D multi-object tracking (MOT) is vital for many applications including autonomous driving vehicles and service robots.
FlowFormer: A Transformer Architecture and Its Masked Cost Volume Autoencoding for Optical Flow
This paper introduces a novel transformer-based network architecture, FlowFormer, along with the Masked Cost Volume AutoEncoding (MCVA) for pretraining it to tackle the problem of optical flow estimation.
Local Boosting for Weakly-Supervised Learning
In this work, we show that the standard implementation of the convex combination of base learners can hardly work due to the presence of noisy labels.
Can Contextual Biasing Remain Effective with Whisper and GPT-2?
End-to-end automatic speech recognition (ASR) and large language models, such as Whisper and GPT-2, have recently been scaled to use vast amounts of training data.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+1
DyGen: Learning from Noisy Labels via Dynamics-Enhanced Generative Modeling
Most existing methods for learning from noisy labels use static input features for denoising, but these methods are limited by the information they can provide on true label distributions and can result in biased or incorrect predictions.
Graph Neural Networks for Contextual ASR with the Tree-Constrained Pointer Generator
The incorporation of biasing words obtained through contextual knowledge is of paramount importance in automatic speech recognition (ASR) applications.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+2
Graph Reasoning for Question Answering with Triplet Retrieval
State-of-the-art methods often utilize entities in questions to retrieve local subgraphs, which are then fed into KG encoder, e. g. graph neural networks (GNNs), to model their local structures and integrated into language models for question answering.
AdaPlanner: Adaptive Planning from Feedback with Language Models
We propose a closed-loop approach, AdaPlanner, which allows the LLM agent to refine its self-generated plan adaptively in response to environmental feedback.
Extracting Shopping Interest-Related Product Types from the Web
To establish such connections, we propose to extract PTs from the Web pages containing hand-crafted PT recommendations for SIs.
Self-supervised representations in speech-based depression detection
This paper proposes handling training data sparsity in speech-based automatic depression detection (SDD) using foundation models pre-trained with self-supervised learning (SSL).
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+5
CCGen: Explainable Complementary Concept Generation in E-Commerce
We propose and study Complementary Concept Generation (CCGen): given a concept of interest, e. g., "Digital Cameras", generating a list of complementary concepts, e. g., 1) Camera Lenses 2) Batteries 3) Camera Cases 4) Memory Cards 5) Battery Chargers.
ReGen: Zero-Shot Text Classification via Training Data Generation with Progressive Dense Retrieval
With the development of large language models (LLMs), zero-shot learning has attracted much attention for various NLP tasks.
Ranked #1 on
Zero-Shot Text Classification
on AG News
