Search Results for author:
Found 577 papers, 246 papers with code
SemanticAdv: Generating Adversarial Examples via Attribute-conditioned Image Editing
Deep neural networks (DNNs) have achieved great successes in various vision applications due to their strong expressive power.
Relative Pose Estimation of Calibrated Cameras with Known SE(3) Invariants
In this paper, we present a complete comprehensive study of the relative pose estimation problem for a calibrated camera constrained by known $\mathrm{SE}(3)$ invariant, which involves 5 minimal problems in total.
Profanity-Avoiding Training Framework for Seq2seq Models with Certified Robustness
To address this problem, we propose a training framework with certified robustness to eliminate the causes that trigger the generation of profanity.
Alibaba Speech Translation Systems for IWSLT 2018
This work describes the En→De Alibaba speech translation system developed for the evaluation campaign of the International Workshop on Spoken Language Translation (IWSLT) 2018.
Introducing v0.5 of the AI Safety Benchmark from MLCommons
We created a new taxonomy of 13 hazard categories, of which 7 have tests in the v0. 5 benchmark.
SambaLingo: Teaching Large Language Models New Languages
In this paper, we present a comprehensive investigation into the adaptation of LLMs to new languages.
ALERT: A Comprehensive Benchmark for Assessing Large Language Models' Safety through Red Teaming
When building Large Language Models (LLMs), it is paramount to bear safety in mind and protect them with guardrails.
KnowHalu: Hallucination Detection via Multi-Form Knowledge Based Factual Checking
This paper introduces KnowHalu, a novel approach for detecting hallucinations in text generated by large language models (LLMs), utilizing step-wise reasoning, multi-formulation query, multi-form knowledge for factual checking, and fusion-based detection mechanism.
QuaRot: Outlier-Free 4-Bit Inference in Rotated LLMs
We introduce QuaRot, a new Quantization scheme based on Rotations, which is able to quantize LLMs end-to-end, including all weights, activations, and KV cache in 4 bits.
Checkpoint Merging via Bayesian Optimization in LLM Pretraining
The rapid proliferation of large language models (LLMs) such as GPT-4 and Gemini underscores the intense demand for resources during their training processes, posing significant challenges due to substantial computational and environmental costs.
Multi-Task Dense Prediction via Mixture of Low-Rank Experts
Furthermore, to control the parameters and computational cost brought by the increase in the number of experts, we take inspiration from LoRA and propose to leverage the low-rank format of a vanilla convolution in the expert network.
TablePuppet: A Generic Framework for Relational Federated Learning
In this paper, we formalize this problem as relational federated learning (RFL).
PPA-Game: Characterizing and Learning Competitive Dynamics Among Online Content Creators
We introduce the Proportional Payoff Allocation Game (PPA-Game) to model how agents, akin to content creators on platforms like YouTube and TikTok, compete for divisible resources and consumers' attention.
Empowering Segmentation Ability to Multi-modal Large Language Models
Multi-modal large language models (MLLMs) can understand image-language prompts and demonstrate impressive reasoning ability.
Contrastive Balancing Representation Learning for Heterogeneous Dose-Response Curves Estimation
Estimating the individuals' potential response to varying treatment doses is crucial for decision-making in areas such as precision medicine and management science.
Few-shot Object Localization
This task achieves generalized object localization by leveraging a small number of labeled support samples to query the positional information of objects within corresponding images.
RigorLLM: Resilient Guardrails for Large Language Models against Undesired Content
The innovative use of constrained optimization and a fusion-based guardrail approach represents a significant step forward in developing more secure and reliable LLMs, setting a new standard for content moderation frameworks in the face of evolving digital threats.
Decoding Compressed Trust: Scrutinizing the Trustworthiness of Efficient LLMs Under Compression
While state-of-the-art (SoTA) compression methods boast impressive advancements in preserving benign task performance, the potential risks of compression in terms of safety and trustworthiness have been largely neglected.
COLEP: Certifiably Robust Learning-Reasoning Conformal Prediction via Probabilistic Circuits
In this work, we propose a certifiably robust learning-reasoning conformal prediction framework (COLEP) via probabilistic circuits, which comprise a data-driven learning component that trains statistical models to learn different semantic concepts, and a reasoning component that encodes knowledge and characterizes the relationships among the trained models for logic reasoning.
Shake to Leak: Fine-tuning Diffusion Models Can Amplify the Generative Privacy Risk
While diffusion models have recently demonstrated remarkable progress in generating realistic images, privacy risks also arise: published models or APIs could generate training images and thus leak privacy-sensitive training information.
2023 Low-Power Computer Vision Challenge (LPCVC) Summary
This article describes the 2023 IEEE Low-Power Computer Vision Challenge (LPCVC).
3D-aware Image Generation and Editing with Multi-modal Conditions
3D-consistent image generation from a single 2D semantic label is an important and challenging research topic in computer graphics and computer vision.
Gemini 1.5: Unlocking multimodal understanding across millions of tokens of context
In this report, we present the latest model of the Gemini family, Gemini 1. 5 Pro, a highly compute-efficient multimodal mixture-of-experts model capable of recalling and reasoning over fine-grained information from millions of tokens of context, including multiple long documents and hours of video and audio.
Ranked #20 on
Code Generation
on HumanEval
BjTT: A Large-scale Multimodal Dataset for Traffic Prediction
The key challenge of the TTG task is how to associate text with the spatial structure of the road network and traffic data for generating traffic situations.
Testing Business Cycle Theories: Evidence from the Great Recession
Empirical business cycle studies using cross-country data usually cannot achieve causal relationships while within-country studies mostly focus on the bust period.
COMMIT: Certifying Robustness of Multi-Sensor Fusion Systems against Semantic Attacks
In this work, we propose the first robustness certification framework COMMIT certify robustness of multi-sensor fusion systems against semantic attacks.
Perceptive self-supervised learning network for noisy image watermark removal
Specifically, mentioned paired watermark images are obtained in a self supervised way, and paired noisy images (i. e., noisy and reference images) are obtained in a supervised way.
Improving Adversarial Energy-Based Model via Diffusion Process
Generative models have shown strong generation ability while efficient likelihood estimation is less explored.
Differentially Private Synthetic Data via Foundation Model APIs 2: Text
Lin et al. (2024) recently introduced the Private Evolution (PE) algorithm to generate DP synthetic images with only API access to diffusion models.
KeNet:Knowledge-enhanced Doc-Label Attention Network for Multi-label text classification
It is imperative to additionally acknowledge that the significance of knowledge is substantiated in the realm of MLTC.
Boosting Box-supervised Instance Segmentation with Pseudo Depth
This innovative approach empowers the network to simultaneously predict masks and depth, enhancing its ability to capture nuanced depth-related information during the instance segmentation process.
ROME: Memorization Insights from Text, Probability and Hidden State in Large Language Models
Probing the memorization of large language models holds significant importance.
HALC: Object Hallucination Reduction via Adaptive Focal-Contrast Decoding
While large vision-language models (LVLMs) have demonstrated impressive capabilities in interpreting multi-modal contexts, they invariably suffer from object hallucinations (OH).
Tree-Regularized Tabular Embeddings
Tabular neural network (NN) has attracted remarkable attentions and its recent advances have gradually narrowed the performance gap with respect to tree-based models on many public datasets.
A Novel Approach to Industrial Defect Generation through Blended Latent Diffusion Model with Online Adaptation
Effectively addressing the challenge of industrial Anomaly Detection (AD) necessitates an ample supply of defective samples, a constraint often hindered by their scarcity in industrial contexts.
DART: Depth-Enhanced Accurate and Real-Time Background Matting
In this paper, we leverage the rich depth information provided by the RGB-Depth (RGB-D) cameras to enhance background matting performance in real-time, dubbed DART.
Mitigating Fine-tuning Jailbreak Attack with Backdoor Enhanced Alignment
Despite the general capabilities of Large Language Models (LLMs) like GPT-4 and Llama-2, these models still request fine-tuning or adaptation with customized data when it comes to meeting the specific business demands and intricacies of tailored use cases.
Handling Ambiguity in Emotion: From Out-of-Domain Detection to Distribution Estimation
The evidential uncertainty measure is extended to quantify the uncertainty in emotion distribution estimation.
ArtPrompt: ASCII Art-based Jailbreak Attacks against Aligned LLMs
In this paper, we propose a novel ASCII art-based jailbreak attack and introduce a comprehensive benchmark Vision-in-Text Challenge (ViTC) to evaluate the capabilities of LLMs in recognizing prompts that cannot be solely interpreted by semantics.
Beyond Quantities: Machine Learning-based Characterization of Inequality in Infrastructure Quality Provision in Cities
While a growing of body of literature has recognized the importance of characterizing infrastructure inequality in cities and provided quantified metrics to inform urban development plans, the majority of the existing approaches focus primarily on measuring the quantity of infrastructure, assuming that more infrastructure is better.
Game of Trojans: Adaptive Adversaries Against Output-based Trojaned-Model Detectors
We propose and analyze an adaptive adversary that can retrain a Trojaned DNN and is also aware of SOTA output-based Trojaned model detectors.
HarmBench: A Standardized Evaluation Framework for Automated Red Teaming and Robust Refusal
Automated red teaming holds substantial promise for uncovering and mitigating the risks associated with the malicious use of large language models (LLMs), yet the field lacks a standardized evaluation framework to rigorously assess new methods.
C-RAG: Certified Generation Risks for Retrieval-Augmented Language Models
Specifically, we provide conformal risk analysis for RAG models and certify an upper confidence bound of generation risks, which we refer to as conformal generation risk.
Robust Prompt Optimization for Defending Language Models Against Jailbreaking Attacks
Despite advances in AI alignment, language models (LM) remain vulnerable to adversarial attacks or jailbreaking, in which adversaries modify input prompts to induce harmful behavior.
Validating Climate Models with Spherical Convolutional Wasserstein Distance
The validation of global climate models is crucial to ensure the accuracy and efficacy of model output.
GRATH: Gradual Self-Truthifying for Large Language Models
GRATH iteratively refines truthfulness data and updates the model, leading to a gradual improvement in model truthfulness in a self-supervised manner.
Benchmarking Large Multimodal Models against Common Corruptions
This technical report aims to fill a deficiency in the assessment of large multimodal models (LMMs) by specifically examining the self-consistency of their outputs when subjected to common corruptions.
BadChain: Backdoor Chain-of-Thought Prompting for Large Language Models
Moreover, we show that LLMs endowed with stronger reasoning capabilities exhibit higher susceptibility to BadChain, exemplified by a high average attack success rate of 97. 0% across the six benchmark tasks on GPT-4.
Efficient Adapter Finetuning for Tail Languages in Streaming Multilingual ASR
Meanwhile, the heterogeneous nature and imbalanced data abundance of different languages may cause performance degradation, leading to asynchronous peak performance for different languages during training, especially on tail ones.
Crafter: Facial Feature Crafting against Inversion-based Identity Theft on Deep Models
However, such methods can not defend against adaptive attacks, in which an attacker takes a countermove against a known defence strategy.
Convolutional Neural Network Ensemble Learning for Hyperspectral Imaging-based Blackberry Fruit Ripeness Detection in Uncontrolled Farm Environment
To address this engineering application challenge, this paper proposes a novel multi-input convolutional neural network (CNN) ensemble classifier for detecting subtle traits of ripeness in blackberry fruits.
CaMML: Context-Aware Multimodal Learner for Large Models
In this work, we introduce Context-Aware MultiModal Learner (CaMML), for tuning large multimodal models (LMMs).
Ranked #50 on
Visual Question Answering
on MM-Vet
VOT: Revolutionizing Speaker Verification with Memory and Attention Mechanisms
Speaker verification is to judge the similarity of two unknown voices in an open set, where the ideal speaker embedding should be able to condense discriminant information into a compact utterance-level representation that has small intra-speaker distances and large inter-speaker distances. We propose a novel model named Voice Transformer(VOT) for speaker verification.
WordScape: a Pipeline to extract multilingual, visually rich Documents with Layout Annotations from Web Crawl Data
Together with the pipeline, we will additionally release 9. 5M urls to word documents which can be processed using WordScape to create a dataset of over 40M pages.
CF-NeRF: Camera Parameter Free Neural Radiance Fields with Incremental Learning
Neural Radiance Fields (NeRF) have demonstrated impressive performance in novel view synthesis.
Labels Need Prompts Too: Mask Matching for Natural Language Understanding Tasks
Textual label names (descriptions) are typically semantically rich in many natural language understanding (NLU) tasks.
USM-Lite: Quantization and Sparsity Aware Fine-tuning for Speech Recognition with Universal Speech Models
We conducted extensive experiments with a 2-billion parameter USM on a large-scale voice search dataset to evaluate our proposed method.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+3
Decoupling Degradation and Content Processing for Adverse Weather Image Restoration
The former facilitates channel-dependent degradation removal operation, allowing the network to tailor responses to various adverse weather types; the latter, by integrating Fourier's global properties into channel-independent content features, enhances network capacity for consistent global content reconstruction.
An explanation for the distribution characteristics of stock returns
In this work, we assume that the effects of events or information on prices obey normal distribution, while financial markets often overreact or underreact to events or information, resulting in non normal distributions of stock returns.
Efficient Incremental Potential Contact for Actuated Face Simulation
We present a quasi-static finite element simulator for human face animation.
Revisiting Single Image Reflection Removal In the Wild
This research focuses on the issue of single-image reflection removal (SIRR) in real-world conditions, examining it from two angles: the collection pipeline of real reflection pairs and the perception of real reflection locations.
Panoptic Video Scene Graph Generation
PVSG relates to the existing video scene graph generation (VidSGG) problem, which focuses on temporal interactions between humans and objects grounded with bounding boxes in videos.
ChatTraffic: Text-to-Traffic Generation via Diffusion Model
The key challenge of the TTG task is how to associate text with the spatial structure of the road network and traffic data for generating traffic situations.
DP-OPT: Make Large Language Model Your Privacy-Preserving Prompt Engineer
To ensure that the prompts do not leak private information, we introduce the first private prompt generation mechanism, by a differentially-private (DP) ensemble of in-context learning with private demonstrations.
TextGuard: Provable Defense against Backdoor Attacks on Text Classification
In this work, we propose TextGuard, the first provable defense against backdoor attacks on text classification.
Generalization and Hallucination of Large Vision-Language Models through a Camouflaged Lens
Large Vision-Language Model (LVLM) has seen burgeoning development and increasing attention recently.
SparseSpikformer: A Co-Design Framework for Token and Weight Pruning in Spiking Transformer
As the third-generation neural network, the Spiking Neural Network (SNN) has the advantages of low power consumption and high energy efficiency, making it suitable for implementation on edge devices.
Geometry-Calibrated DRO: Combating Over-Pessimism with Free Energy Implications
Machine learning algorithms minimizing average risk are susceptible to distributional shifts.
OtterHD: A High-Resolution Multi-modality Model
In this paper, we present OtterHD-8B, an innovative multimodal model evolved from Fuyu-8B, specifically engineered to interpret high-resolution visual inputs with granular precision.
Ranked #85 on
Visual Question Answering
on MM-Vet
Identifying and Mitigating Vulnerabilities in LLM-Integrated Applications
Successful exploits of the identified vulnerabilities result in the users receiving responses tailored to the intent of a threat initiator.
Invariant-Feature Subspace Recovery: A New Class of Provable Domain Generalization Algorithms
First, in the binary classification setup of Rosenfeld et al. (2021), we show that our first algorithm, ISR-Mean, can identify the subspace spanned by invariant features from the first-order moments of the class-conditional distributions, and achieve provable domain generalization with $d_s+1$ training environments.
IMPRESS: Evaluating the Resilience of Imperceptible Perturbations Against Unauthorized Data Usage in Diffusion-Based Generative AI
IMPRESS is based on the key observation that imperceptible perturbations could lead to a perceptible inconsistency between the original image and the diffusion-reconstructed image, which can be used to devise a new optimization strategy for purifying the image, which may weaken the protection of the original image from unauthorized data usage (e. g., style mimicking, malicious editing).
Bipartite Graph Pre-training for Unsupervised Extractive Summarization with Graph Convolutional Auto-Encoders
Pre-trained sentence representations are crucial for identifying significant sentences in unsupervised document extractive summarization.
DiffAttack: Evasion Attacks Against Diffusion-Based Adversarial Purification
In particular, we propose a deviated-reconstruction loss at intermediate diffusion steps to induce inaccurate density gradient estimation to tackle the problem of vanishing/exploding gradients.
CBD: A Certified Backdoor Detector Based on Local Dominant Probability
Notably, for backdoor attacks with random perturbation triggers bounded by $\ell_2\leq0. 75$ which achieves more than 90\% attack success rate, CBD achieves 100\% (98\%), 100\% (84\%), 98\% (98\%), and 72\% (40\%) empirical (certified) detection true positive rates on the four benchmark datasets GTSRB, SVHN, CIFAR-10, and TinyImageNet, respectively, with low false positive rates.
Gradual Domain Adaptation: Theory and Algorithms
Unsupervised domain adaptation (UDA) adapts a model from a labeled source domain to an unlabeled target domain in a one-off way.
Effective and Efficient Federated Tree Learning on Hybrid Data
To address this, we propose HybridTree, a novel federated learning approach that enables federated tree learning on hybrid data.
Exploring Decision-based Black-box Attacks on Face Forgery Detection
Further, the fake faces by our method can pass face forgery detection and face recognition, which exposes the security problems of face forgery detectors.
RGM: A Robust Generalizable Matching Model
Finding corresponding pixels within a pair of images is a fundamental computer vision task with various applications.
Towards Training-free Open-world Segmentation via Image Prompt Foundation Models
The realm of computer vision has witnessed a paradigm shift with the advent of foundational models, mirroring the transformative influence of large language models in the domain of natural language processing.
Ring-A-Bell! How Reliable are Concept Removal Methods for Diffusion Models?
While efforts have been made to mitigate such problems, either by implementing a safety filter at the evaluation stage or by fine-tuning models to eliminate undesirable concepts or styles, the effectiveness of these safety measures in dealing with a wide range of prompts remains largely unexplored.
Unraveling Fundamental Properties of Power System Resilience Curves using Unsupervised Machine Learning
Trapezoidal archetypes explain resilience curves based on 1. duration of sustained function loss and 2. constant recovery rate.
LRRU: Long-short Range Recurrent Updating Networks for Depth Completion
Existing deep learning-based depth completion methods generally employ massive stacked layers to predict the dense depth map from sparse input data.
Octopus: Embodied Vision-Language Programmer from Environmental Feedback
Large vision-language models (VLMs) have achieved substantial progress in multimodal perception and reasoning.
InstructRetro: Instruction Tuning post Retrieval-Augmented Pretraining
After instruction tuning on Retro, InstructRetro demonstrates significant improvement over the instruction tuned GPT on a wide range of zero-shot tasks.
PST: Improving Quantitative Trading via Program Sketch-based Tuning
Particularly, PST first proposes using a novel symbolic program sketch to embed the abstract human expert knowledge of market trends.
AI-based association analysis for medical imaging using latent-space geometric confounder correction
AI has greatly enhanced medical image analysis, yet its use in epidemiological population imaging studies remains limited due to visualization challenges in non-linear models and lack of confounder control.
RLLTE: Long-Term Evolution Project of Reinforcement Learning
We present RLLTE: a long-term evolution, extremely modular, and open-source framework for reinforcement learning (RL) research and application.
Massive End-to-end Models for Short Search Queries
In this work, we investigate two popular end-to-end automatic speech recognition (ASR) models, namely Connectionist Temporal Classification (CTC) and RNN-Transducer (RNN-T), for offline recognition of voice search queries, with up to 2B model parameters.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+1
Visible and NIR Image Fusion Algorithm Based on Information Complementarity
Second, to generate the initial visible-NIR complementarity weight map, the difference maps of visible and NIR are filtered by the extend-DoG filter.
Rethinking Imitation-based Planner for Autonomous Driving
In recent years, imitation-based driving planners have reported considerable success.
Self-supervised Multi-view Clustering in Computer Vision: A Survey
However, as contrastive learning continues to evolve within the field of computer vision, self-supervised learning has also made substantial research progress and is progressively becoming dominant in MVC methods.
Zero-Shot Co-salient Object Detection Framework
Despite recent advancements in deep learning models, these models still rely on training with well-annotated CoSOD datasets.
DiffSmooth: Certifiably Robust Learning via Diffusion Models and Local Smoothing
Diffusion models have been leveraged to perform adversarial purification and thus provide both empirical and certified robustness for a standard model.
Classification Committee for Active Deep Object Detection
The role of the classification committee is to select the most informative images according to their uncertainty values from the view of classification, which is expected to focus more on the discrepancy and representative of instances.
An Interpretable Machine Learning Model with Deep Learning-based Imaging Biomarkers for Diagnosis of Alzheimer's Disease
However, some machine learning methods based on imaging data have poor interpretability because it is usually unclear how they make their decisions.
Target before Shooting: Accurate Anomaly Detection and Localization under One Millisecond via Cascade Patch Retrieval
In this framework, the anomaly detection problem is solved via a cascade patch retrieval procedure that retrieves the nearest neighbors for each test image patch in a coarse-to-fine fashion.
Ranked #1 on
Supervised Anomaly Detection
on BTAD
LoRA-FA: Memory-efficient Low-rank Adaptation for Large Language Models Fine-tuning
The low-rank adaptation (LoRA) method can largely reduce the amount of trainable parameters for fine-tuning large language models (LLMs), however, it still requires expensive activation memory to update low-rank weights.
Eva: A General Vectorized Approximation Framework for Second-order Optimization
Second-order optimization algorithms exhibit excellent convergence properties for training deep learning models, but often incur significant computation and memory overheads.
Benchmarking and Analyzing Generative Data for Visual Recognition
Advancements in large pre-trained generative models have expanded their potential as effective data generators in visual recognition.
MMBench: Is Your Multi-modal Model an All-around Player?
In response to these challenges, we propose MMBench, a novel multi-modality benchmark.
Ranked #1 on
Visual Question Answering
on MMBench
SoK: Privacy-Preserving Data Synthesis
As the prevalence of data analysis grows, safeguarding data privacy has become a paramount concern.
Structured Network Pruning by Measuring Filter-wise Interactions
Utilizing this new redundancy criterion, we propose a structured network pruning approach SNPFI (Structured Network Pruning by measuring Filter-wise Interaction).
Query-Efficient Decision-based Black-Box Patch Attack
In this work, we first explore the decision-based patch attack.
Learning to Pan-sharpening with Memories of Spatial Details
To address this issue, in this paper we observe that the spatial details from PAN images are mainly high-frequency cues, i. e., the edges reflect the contour of input PAN images.
FunQA: Towards Surprising Video Comprehension
Surprising videos, such as funny clips, creative performances, or visual illusions, attract significant attention.
Synthetic data shuffling accelerates the convergence of federated learning under data heterogeneity
In this paper, we establish a precise and quantifiable correspondence between data heterogeneity and parameters in the convergence rate when a fraction of data is shuffled across clients.
DecodingTrust: A Comprehensive Assessment of Trustworthiness in GPT Models
Yet, while the literature on the trustworthiness of GPT models remains limited, practitioners have proposed employing capable GPT models for sensitive applications such as healthcare and finance -- where mistakes can be costly.
Deep learning-based group-wise registration for longitudinal MRI analysis in glioma
The proposed methods may serve as an alternative to classical toolboxes, to provide further insight into glioma growth.
Prior-knowledge-informed deep learning for lacune detection and quantification using multi-site brain MRI
Lacunes of presumed vascular origin, also referred to as lacunar infarcts, are important to assess cerebral small vessel disease and cognitive diseases such as dementia.
Evaluation and Optimization of Gradient Compression for Distributed Deep Learning
To accelerate distributed training, many gradient compression methods have been proposed to alleviate the communication bottleneck in synchronous stochastic gradient descent (S-SGD), but their efficacy in real-world applications still remains unclear.
MIMIC-IT: Multi-Modal In-Context Instruction Tuning
We release the MIMIC-IT dataset, instruction-response collection pipeline, benchmarks, and the Otter model.
Ranked #87 on
Visual Question Answering
on MM-Vet
MMSum: A Dataset for Multimodal Summarization and Thumbnail Generation of Videos
To address these challenges and provide a comprehensive dataset for this new direction, we have meticulously curated the \textbf{MMSum} dataset.
How to Estimate Model Transferability of Pre-Trained Speech Models?
In this work, we introduce a "score-based assessment" framework for estimating the transferability of pre-trained speech models (PSMs) for fine-tuning target tasks.
Competing for Shareable Arms in Multi-Player Multi-Armed Bandits
In reality, agents often have to learn and maximize the rewards of the resources at the same time.
UMD: Unsupervised Model Detection for X2X Backdoor Attacks
Backdoor (Trojan) attack is a common threat to deep neural networks, where samples from one or more source classes embedded with a backdoor trigger will be misclassified to adversarial target classes.
On the Tool Manipulation Capability of Open-source Large Language Models
In this paper, we ask can we enhance open-source LLMs to be competitive to leading closed LLM APIs in tool manipulation, with practical amount of human supervision.
GrowSP: Unsupervised Semantic Segmentation of 3D Point Clouds
Our method consists of three major components, 1) the feature extractor to learn per-point features from input point clouds, 2) the superpoint constructor to progressively grow the sizes of superpoints, and 3) the semantic primitive clustering module to group superpoints into semantic elements for the final semantic segmentation.
Mixture-of-Expert Conformer for Streaming Multilingual ASR
We evaluate the proposed model on a set of 12 languages, and achieve an average 11. 9% relative improvement in WER over the baseline.
Reconstructive Neuron Pruning for Backdoor Defense
Specifically, RNP first unlearns the neurons by maximizing the model's error on a small subset of clean samples and then recovers the neurons by minimizing the model's error on the same data.
Modular Domain Adaptation for Conformer-Based Streaming ASR
Speech data from different domains has distinct acoustic and linguistic characteristics.
Can Public Large Language Models Help Private Cross-device Federated Learning?
We study (differentially) private federated learning (FL) of language models.
Re-thinking Data Availablity Attacks Against Deep Neural Networks
The unauthorized use of personal data for commercial purposes and the clandestine acquisition of private data for training machine learning models continue to raise concerns.
Towards Generalizable Data Protection With Transferable Unlearnable Examples
One of the crucial factors contributing to this success has been the access to an abundance of high-quality data for constructing machine learning models.
Otter: A Multi-Modal Model with In-Context Instruction Tuning
Large language models (LLMs) have demonstrated significant universal capabilities as few/zero-shot learners in various tasks due to their pre-training on vast amounts of text data, as exemplified by GPT-3, which boosted to InstrctGPT and ChatGPT, effectively following natural language instructions to accomplish real-world tasks.
Ranked #8 on
Visual Question Answering
on BenchLMM
Evaluating ChatGPT's Information Extraction Capabilities: An Assessment of Performance, Explainability, Calibration, and Faithfulness
The capability of Large Language Models (LLMs) like ChatGPT to comprehend user intent and provide reasonable responses has made them extremely popular lately.
The Second Monocular Depth Estimation Challenge
This paper discusses the results for the second edition of the Monocular Depth Estimation Challenge (MDEC).
Shall We Pretrain Autoregressive Language Models with Retrieval? A Comprehensive Study
Thus, it is still an open question: shall we pretrain large autoregressive LMs with retrieval?
Fast vehicle detection algorithm based on lightweight YOLO7-tiny
The swift and precise detection of vehicles plays a significant role in intelligent transportation systems.
Can SAM Segment Anything? When SAM Meets Camouflaged Object Detection
In this study, we try to ask if SAM can address the COD task and evaluate the performance of SAM on the COD benchmark by employing maximum segmentation evaluation and camouflage location evaluation.
Predictive Heterogeneity: Measures and Applications
As an intrinsic and fundamental property of big data, data heterogeneity exists in a variety of real-world applications, such as precision medicine, autonomous driving, financial applications, etc.
Invertible Convolution with Symmetric Paddings
We show that symmetrically padded convolution can be analytically inverted via DFT.
Efficient Decision-based Black-box Patch Attacks on Video Recognition
First, STDE introduces target videos as patch textures and only adds patches on keyframes that are adaptively selected by temporal difference.
Graph Transformer GANs for Graph-Constrained House Generation
We present a novel graph Transformer generative adversarial network (GTGAN) to learn effective graph node relations in an end-to-end fashion for the challenging graph-constrained house generation task.
TrojDiff: Trojan Attacks on Diffusion Models with Diverse Targets
To answer these questions, we propose an effective Trojan attack against diffusion models, TrojDiff, which optimizes the Trojan diffusion and generative processes during training.
Google USM: Scaling Automatic Speech Recognition Beyond 100 Languages
We introduce the Universal Speech Model (USM), a single large model that performs automatic speech recognition (ASR) across 100+ languages.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+3
DeAR: Accelerating Distributed Deep Learning with Fine-Grained All-Reduce Pipelining
Communication scheduling has been shown to be effective in accelerating distributed training, which enables all-reduce communications to be overlapped with backpropagation computations.
Pose-Controllable 3D Facial Animation Synthesis using Hierarchical Audio-Vertex Attention
In this paper, a novel pose-controllable 3D facial animation synthesis method is proposed by utilizing hierarchical audio-vertex attention.
UML: A Universal Monolingual Output Layer for Multilingual ASR
Consequently, the UML enables to switch in the interpretation of each output node depending on the language of the input speech.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+1
Delving into the Adversarial Robustness of Federated Learning
In this work, we propose a novel algorithm called Decision Boundary based Federated Adversarial Training (DBFAT), which consists of two components (local re-weighting and global regularization) to improve both accuracy and robustness of FL systems.
Massively Multilingual Shallow Fusion with Large Language Models
In this work, we propose to train a single multilingual language model (LM) for shallow fusion in multiple languages.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+2
JEIT: Joint End-to-End Model and Internal Language Model Training for Speech Recognition
We propose JEIT, a joint end-to-end (E2E) model and internal language model (ILM) training method to inject large-scale unpaired text into ILM during E2E training which improves rare-word speech recognition.
PerAda: Parameter-Efficient Federated Learning Personalization with Generalization Guarantees
In this paper, we propose PerAda, a parameter-efficient pFL framework that reduces communication and computational costs and exhibits superior generalization performance, especially under test-time distribution shifts.
3D Colored Shape Reconstruction from a Single RGB Image through Diffusion
In shape prediction module, the reference RGB image is first encoded into a high-level shape feature and then the shape feature is utilized as a condition to predict the reverse geometric noise in diffusion model.
Re-ViLM: Retrieval-Augmented Visual Language Model for Zero and Few-Shot Image Captioning
Augmenting pretrained language models (LMs) with a vision encoder (e. g., Flamingo) has obtained the state-of-the-art results in image-to-text generation.
Interpolation for Robust Learning: Data Augmentation on Wasserstein Geodesics
Our work provides a new perspective of model robustness through the lens of Wasserstein geodesic-based interpolation with a practical off-the-shelf strategy that can be combined with existing robust training methods.
Efficient Domain Adaptation for Speech Foundation Models
The FM encoder adapter and decoder are then finetuned to the target domain with a small amount of supervised in-domain data.
Defensive ML: Defending Architectural Side-channels with Adversarial Obfuscation
To address this problem, this paper explores using Adversarial Machine Learning (AML) methods as a defense at the computer architecture layer to obfuscate side channels.
Automatic Intrinsic Reward Shaping for Exploration in Deep Reinforcement Learning
We present AIRS: Automatic Intrinsic Reward Shaping that intelligently and adaptively provides high-quality intrinsic rewards to enhance exploration in reinforcement learning (RL).
From English to More Languages: Parameter-Efficient Model Reprogramming for Cross-Lingual Speech Recognition
In this work, we propose a new parameter-efficient learning framework based on neural model reprogramming for cross-lingual speech recognition, which can \textbf{re-purpose} well-trained English automatic speech recognition (ASR) models to recognize the other languages.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+1
Proportional Fairness in Obnoxious Facility Location
On the other hand, in the randomized setting, we identify proportionally fair and strategyproof mechanisms that give an expected welfare within a constant factor of the optimal welfare.
A Bertrand duopoly game with differentiated products reconsidered
In this paper, we explore a dynamic Bertrand duopoly game with differentiated products, where firms are boundedly rational and consumers are assumed to possess an underlying CES utility function.
PHA: Patch-Wise High-Frequency Augmentation for Transformer-Based Person Re-Identification
Although recent studies empirically show that injecting Convolutional Neural Networks (CNNs) into Vision Transformers (ViTs) can improve the performance of person re-identification, the rationale behind it remains elusive.
Dual Alignment Unsupervised Domain Adaptation for Video-Text Retrieval
To tackle this, we propose a novel method named Dual Alignment Domain Adaptation (DADA).
Rethinking the Learning Paradigm for Dynamic Facial Expression Recognition
Dynamic Facial Expression Recognition (DFER) is a rapidly developing field that focuses on recognizing facial expressions in video format.
Ranked #7 on
Dynamic Facial Expression Recognition
on FERV39k
Dynamic Facial Expression Recognition
Facial Expression Recognition
AREA: Adaptive Reweighting via Effective Area for Long-Tailed Classification
Consequently, we estimate the size of the spanned space for each category, namely effective area, by detailedly analyzing its samples' distribution.
Reviewing Labels: Label Graph Network with Top-k Prediction Set for Relation Extraction
Specifically, for a given sample, we build a label graph to review candidate labels in the Top-k prediction set and learn the connections between them.
Ranked #1 on
Relation Extraction
on TACRED-Revisited
Sequence Generation with Label Augmentation for Relation Extraction
Sequence generation demonstrates promising performance in recent information extraction efforts, by incorporating large-scale pre-trained Seq2Seq models.
Ranked #1 on
Relation Extraction
on sciERC-sent
EDoG: Adversarial Edge Detection For Graph Neural Networks
In this paper, we propose a general adversarial edge detection pipeline EDoG without requiring knowledge of the attack strategies based on graph generation.
Forecasting West Nile Virus with Graph Neural Networks: Harnessing Spatial Dependence in Irregularly Sampled Geospatial Data
Machine learning methods have seen increased application to geospatial environmental problems, such as precipitation nowcasting, haze forecasting, and crop yield prediction.
Benchmarking Robustness of Multimodal Image-Text Models under Distribution Shift
Multimodal image-text models have shown remarkable performance in the past few years.
On the effectiveness of partial variance reduction in federated learning with heterogeneous data
In this paper, we first revisit the widely used FedAvg algorithm in a deep neural network to understand how data heterogeneity influences the gradient updates across the neural network layers.
Rethinking Disparity: A Depth Range Free Multi-View Stereo Based on Disparity
Existing learning-based multi-view stereo (MVS) methods rely on the depth range to build the 3D cost volume and may fail when the range is too large or unreliable.
Logic and Commonsense-Guided Temporal Knowledge Graph Completion
To address these challenges, we propose a Logic and Commonsense-Guided Embedding model (LCGE) to jointly learn the time-sensitive representation involving timeliness and causality of events, together with the time-independent representation of events from the perspective of commonsense.
Tackling Visual Control via Multi-View Exploration Maximization
We present MEM: Multi-view Exploration Maximization for tackling complex visual control tasks.
Confounder Balancing for Instrumental Variable Regression with Latent Variable
This paper studies the confounding effects from the unmeasured confounders and the imbalance of observed confounders in IV regression and aims at unbiased causal effect estimation.
AnimeRun: 2D Animation Visual Correspondence from Open Source 3D Movies
Existing correspondence datasets for two-dimensional (2D) cartoon suffer from simple frame composition and monotonic movements, making them insufficient to simulate real animations.
HFedMS: Heterogeneous Federated Learning with Memorable Data Semantics in Industrial Metaverse
Federated Learning (FL), as a rapidly evolving privacy-preserving collaborative machine learning paradigm, is a promising approach to enable edge intelligence in the emerging Industrial Metaverse.
Resource-Efficient Transfer Learning From Speech Foundation Model Using Hierarchical Feature Fusion
Experimental results show that the proposed method can achieve better performance on speech recognition task than existing algorithms with fewer number of trainable parameters, less computational memory cost and faster training speed.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+2
Fairness in Federated Learning via Core-Stability
Nonetheless, the heterogeneity nature of distributed data makes it challenging to define and ensure fairness among local agents.
A Quantum Kernel Learning Approach to Acoustic Modeling for Spoken Command Recognition
We propose a quantum kernel learning (QKL) framework to address the inherent data sparsity issues often encountered in training large-scare acoustic models in low-resource scenarios.
Unified End-to-End Speech Recognition and Endpointing for Fast and Efficient Speech Systems
In this work, we propose a method to jointly train the ASR and EP tasks in a single end-to-end (E2E) multitask model, improving EP quality by optionally leveraging information from the ASR audio encoder.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+1
DensePure: Understanding Diffusion Models towards Adversarial Robustness
By using the highest density point in the conditional distribution as the reversed sample, we identify the robust region of a given instance under the diffusion model's reverse process.
Shape Matters: Deformable Patch Attack
Though deep neural networks (DNNs) have demonstrated excellent performance in computer vision, they are susceptible and vulnerable to carefully crafted adversarial examples which can mislead DNNs to incorrect outputs.
CU-Net: LiDAR Depth-Only Completion With Coupled U-Net
We find that existing depth-only methods can obtain satisfactory results in the areas where the measurement points are almost accurate and evenly distributed (denoted as normal areas), while the performance is limited in the areas where the foreground and background points are overlapped due to occlusion (denoted as overlap areas) and the areas where there are no measurement points around (denoted as blank areas) since the methods have no reliable input information in these areas.
Group Distributionally Robust Reinforcement Learning with Hierarchical Latent Variables
One key challenge for multi-task Reinforcement learning (RL) in practice is the absence of task indicators.
LOT: Layer-wise Orthogonal Training on Improving $\ell_2$ Certified Robustness
On the other hand, as existing works show that semi-supervised training helps improve empirical robustness, we aim to bridge the gap and prove that semi-supervised learning also improves the certified robustness of Lipschitz-bounded models.
Handling Label Uncertainty for Camera Incremental Person Re-Identification
Specifically, new data collected from new cameras may probably contain an unknown proportion of identities seen before.
Product Ranking for Revenue Maximization with Multiple Purchases
In this paper, we assume that each consumer can purchase multiple products at will.
OpenOOD: Benchmarking Generalized Out-of-Distribution Detection
Out-of-distribution (OOD) detection is vital to safety-critical machine learning applications and has thus been extensively studied, with a plethora of methods developed in the literature.
JOIST: A Joint Speech and Text Streaming Model For ASR
In addition, we explore JOIST using a streaming E2E model with an order of magnitude more data, which are also novelties compared to previous works.
Feature Reconstruction Attacks and Countermeasures of DNN training in Vertical Federated Learning
To address this problem, we develop a novel feature protection scheme against the reconstruction attack that effectively misleads the search to some pre-specified random values.
The Equalization Losses: Gradient-Driven Training for Long-tailed Object Recognition
Long-tail distribution is widely spread in real-world applications.
Improving Long-tailed Object Detection with Image-Level Supervision by Multi-Task Collaborative Learning
Specifically, there are an object detection task (consisting of an instance-classification task and a localization task) and an image-classification task in our framework, responsible for utilizing the two types of supervision.
Scaling Up Deliberation for Multilingual ASR
In this work, we investigate second-pass deliberation for multilingual speech recognition.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+1
Semantics-Consistent Cross-domain Summarization via Optimal Transport Alignment
Multimedia summarization with multimodal output (MSMO) is a recently explored application in language grounding.
Untargeted Backdoor Watermark: Towards Harmless and Stealthy Dataset Copyright Protection
In this paper, we revisit dataset ownership verification.
Towards Stable Co-saliency Detection and Object Co-segmentation
In this paper, we present a novel model for simultaneous stable co-saliency detection (CoSOD) and object co-segmentation (CoSEG).
Attributed Network Embedding Model for Exposing COVID-19 Spread Trajectory Archetypes
To this end, this study creates a network embedding model capturing cross-county visitation networks, as well as heterogeneous features to uncover clusters of counties in the United States based on their pandemic spread transmission trajectories.
Rewarding Episodic Visitation Discrepancy for Exploration in Reinforcement Learning
Exploration is critical for deep reinforcement learning in complex environments with high-dimensional observations and sparse rewards.
Trustworthy Reinforcement Learning Against Intrinsic Vulnerabilities: Robustness, Safety, and Generalizability
A trustworthy reinforcement learning algorithm should be competent in solving challenging real-world problems, including {robustly} handling uncertainties, satisfying {safety} constraints to avoid catastrophic failures, and {generalizing} to unseen scenarios during deployments.
Graph Contrastive Learning with Personalized Augmentation
Thus, blindly augmenting all graphs without considering their individual characteristics may undermine the performance of GCL arts. To deal with this, we propose the first principled framework, termed as \textit{G}raph contrastive learning with \textit{P}ersonalized \textit{A}ugmentation (GPA), to advance conventional GCL by allowing each graph to choose its own suitable augmentation operations. In essence, GPA infers tailored augmentation strategies for each graph based on its topology and node attributes via a learnable augmentation selector, which is a plug-and-play module and can be effectively trained with downstream GCL models end-to-end.
Streaming End-to-End Multilingual Speech Recognition with Joint Language Identification
Language identification is critical for many downstream tasks in automatic speech recognition (ASR), and is beneficial to integrate into multilingual end-to-end ASR as an additional task.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+2
CARE: Certifiably Robust Learning with Reasoning via Variational Inference
In particular, we propose a certifiably robust learning with reasoning pipeline (CARE), which consists of a learning component and a reasoning component.
Unraveling the Connections between Privacy and Certified Robustness in Federated Learning Against Poisoning Attacks
We then provide two robustness certification criteria: certified prediction and certified attack inefficacy for DPFL on both user and instance levels.
Privacy of Autonomous Vehicles: Risks, Protection Methods, and Future Directions
However, training AVs usually requires a large amount of training data collected from different driving environments (e. g., cities) as well as different types of personal information (e. g., working hours and routes).
Synergistic Redundancy: Towards Verifiable Safety for Autonomous Vehicles
The synergistic safety layer uses only verifiable and logically analyzable software to fulfill its tasks.
Federated Learning with Label Distribution Skew via Logits Calibration
Extensive experiments on federated datasets and real-world datasets demonstrate that FedLC leads to a more accurate global model and much improved performance.
Verifiable Obstacle Detection
Perception of obstacles remains a critical safety concern for autonomous vehicles.
Turn-Taking Prediction for Natural Conversational Speech
This makes doing speech recognition with conversational speech, including one with multiple queries, a challenging task.
