Search Results for author:
Found 575 papers, 244 papers with code
Exploring the Limits of Domain-Adaptive Training for Detoxifying Large-Scale Language Models
In this work, we systematically explore domain-adaptive training to reduce the toxicity of language models.
Shall We Pretrain Autoregressive Language Models with Retrieval? A Comprehensive Study
Thus, it is still an open question: shall we pretrain large autoregressive LMs with retrieval?
InstructRetro: Instruction Tuning post Retrieval-Augmented Pretraining
After instruction tuning on Retro, InstructRetro demonstrates significant improvement over the instruction tuned GPT on a wide range of zero-shot tasks.
A Review of Single-Source Deep Unsupervised Visual Domain Adaptation
To cope with limited labeled training data, many have attempted to directly apply models trained on a large-scale labeled source domain to another sparsely labeled or unlabeled target domain.
SiamRPN++: Evolution of Siamese Visual Tracking with Very Deep Networks
Moreover, we propose a new model architecture to perform depth-wise and layer-wise aggregations, which not only further improves the accuracy but also reduces the model size.
Ranked #5 on
Visual Object Tracking
on VOT2017/18
Towards Interpretable R-CNN by Unfolding Latent Structures
We focus on weakly-supervised extractive rationale generation, that is learning to unfold latent discriminative part configurations of object instances automatically and simultaneously in detection without using any supervision for part configurations.
Otter: A Multi-Modal Model with In-Context Instruction Tuning
Large language models (LLMs) have demonstrated significant universal capabilities as few/zero-shot learners in various tasks due to their pre-training on vast amounts of text data, as exemplified by GPT-3, which boosted to InstrctGPT and ChatGPT, effectively following natural language instructions to accomplish real-world tasks.
Ranked #8 on
Visual Question Answering
on BenchLMM
MIMIC-IT: Multi-Modal In-Context Instruction Tuning
We release the MIMIC-IT dataset, instruction-response collection pipeline, benchmarks, and the Otter model.
Ranked #81 on
Visual Question Answering
on MM-Vet
OtterHD: A High-Resolution Multi-modality Model
In this paper, we present OtterHD-8B, an innovative multimodal model evolved from Fuyu-8B, specifically engineered to interpret high-resolution visual inputs with granular precision.
Ranked #79 on
Visual Question Answering
on MM-Vet
Lingvo: a Modular and Scalable Framework for Sequence-to-Sequence Modeling
Lingvo is a Tensorflow framework offering a complete solution for collaborative deep learning research, with a particular focus towards sequence-to-sequence models.
MMBench: Is Your Multi-modal Model an All-around Player?
In response to these challenges, we propose MMBench, a novel multi-modality benchmark.
Ranked #1 on
Visual Question Answering
on MMBench
SECOND: Sparsely Embedded Convolutional Detection
LiDAR-based or RGB-D-based object detection is used in numerous applications, ranging from autonomous driving to robot vision.
Global Convergence and Generalization Bound of Gradient-Based Meta-Learning with Deep Neural Nets
Gradient-based meta-learning (GBML) with deep neural nets (DNNs) has become a popular approach for few-shot learning.
Distractor-aware Siamese Networks for Visual Object Tracking
During the off-line training phase, an effective sampling strategy is introduced to control this distribution and make the model focus on the semantic distractors.
Ranked #11 on
Visual Object Tracking
on VOT2017/18
High Performance Visual Tracking With Siamese Region Proposal Network
Visual object tracking has been a fundamental topic in recent years and many deep learning based trackers have achieved state-of-the-art performance on multiple benchmarks.
Ranked #7 on
Visual Object Tracking
on VOT2017/18
Automatic Intrinsic Reward Shaping for Exploration in Deep Reinforcement Learning
We present AIRS: Automatic Intrinsic Reward Shaping that intelligently and adaptively provides high-quality intrinsic rewards to enhance exploration in reinforcement learning (RL).
Streaming End-to-end Speech Recognition For Mobile Devices
End-to-end (E2E) models, which directly predict output character sequences given input speech, are good candidates for on-device speech recognition.
Adversarial Attack and Defense on Graph Data: A Survey
Therefore, this review is intended to provide an overall landscape of more than 100 papers on adversarial attack and defense strategies for graph data, and establish a unified formulation encompassing most graph adversarial learning models.
OpenOOD: Benchmarking Generalized Out-of-Distribution Detection
Out-of-distribution (OOD) detection is vital to safety-critical machine learning applications and has thus been extensively studied, with a plethora of methods developed in the literature.
PSGAN++: Robust Detail-Preserving Makeup Transfer and Removal
In this paper, we address the makeup transfer and removal tasks simultaneously, which aim to transfer the makeup from a reference image to a source image and remove the makeup from the with-makeup image respectively.
Generating Adversarial Examples with Adversarial Networks
A challenge to explore adversarial robustness of neural networks on MNIST.
State-of-the-art Speech Recognition With Sequence-to-Sequence Models
Attention-based encoder-decoder architectures such as Listen, Attend, and Spell (LAS), subsume the acoustic, pronunciation and language model components of a traditional automatic speech recognition (ASR) system into a single neural network.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+2
Multi-View 3D Object Detection Network for Autonomous Driving
We encode the sparse 3D point cloud with a compact multi-view representation.
RLLTE: Long-Term Evolution Project of Reinforcement Learning
We present RLLTE: a long-term evolution, extremely modular, and open-source framework for reinforcement learning (RL) research and application.
Equalized Focal Loss for Dense Long-Tailed Object Detection
The conventional focal loss balances the training process with the same modulating factor for all categories, thus failing to handle the long-tailed problem.
The Equalization Losses: Gradient-Driven Training for Long-tailed Object Recognition
Long-tail distribution is widely spread in real-world applications.
InfoBERT: Improving Robustness of Language Models from An Information Theoretic Perspective
Large-scale language models such as BERT have achieved state-of-the-art performance across a wide range of NLP tasks.
Ranked #3 on
Natural Language Inference
on ANLI test
(using extra training data)
Federated Learning with Label Distribution Skew via Logits Calibration
Extensive experiments on federated datasets and real-world datasets demonstrate that FedLC leads to a more accurate global model and much improved performance.
Targeted Backdoor Attacks on Deep Learning Systems Using Data Poisoning
In this work, we consider a new type of attacks, called backdoor attacks, where the attacker's goal is to create a backdoor into a learning-based authentication system, so that he can easily circumvent the system by leveraging the backdoor.
3D Fully Convolutional Network for Vehicle Detection in Point Cloud
2D fully convolutional network has been recently successfully applied to object detection from images.
Sparse Mixture-of-Experts are Domain Generalizable Learners
It is motivated by an empirical finding that transformer-based models trained with empirical risk minimization (ERM) outperform CNN-based models employing state-of-the-art (SOTA) DG algorithms on multiple DG datasets.
Ranked #11 on
Domain Generalization
on DomainNet
(using extra training data)
Octopus: Embodied Vision-Language Programmer from Environmental Feedback
Large vision-language models (VLMs) have achieved substantial progress in multimodal perception and reasoning.
General Cutting Planes for Bound-Propagation-Based Neural Network Verification
Our generalized bound propagation method, GCP-CROWN, opens up the opportunity to apply general cutting plane methods for neural network verification while benefiting from the efficiency and GPU acceleration of bound propagation methods.
BBDM: Image-to-image Translation with Brownian Bridge Diffusion Models
In this paper, a novel image-to-image translation method based on the Brownian Bridge Diffusion Model (BBDM) is proposed, which models image-to-image translation as a stochastic Brownian bridge process, and learns the translation between two domains directly through the bidirectional diffusion process rather than a conditional generation process.
SAFA: Structure Aware Face Animation
On the other hand, some area of the generated image might be occluded in the source image, which makes it difficult for GAN to generate realistic appearance.
Multi-source Domain Adaptation for Semantic Segmentation
In this paper, we propose to investigate multi-source domain adaptation for semantic segmentation.
Ranked #2 on
Domain Adaptation
on GTA5+Synscapes to Cityscapes
MADAN: Multi-source Adversarial Domain Aggregation Network for Domain Adaptation
Finally, feature-level alignment is performed between the aggregated domain and the target domain while training the task network.
DBA: Distributed Backdoor Attacks against Federated Learning
Compared to standard centralized backdoors, we show that DBA is substantially more persistent and stealthy against FL on diverse datasets such as finance and image data.
Towards Stable and Efficient Training of Verifiably Robust Neural Networks
In this paper, we propose a new certified adversarial training method, CROWN-IBP, by combining the fast IBP bounds in a forward bounding pass and a tight linear relaxation based bound, CROWN, in a backward bounding pass.
GrowSP: Unsupervised Semantic Segmentation of 3D Point Clouds
Our method consists of three major components, 1) the feature extractor to learn per-point features from input point clouds, 2) the superpoint constructor to progressively grow the sizes of superpoints, and 3) the semantic primitive clustering module to group superpoints into semantic elements for the final semantic segmentation.
Evaluating ChatGPT's Information Extraction Capabilities: An Assessment of Performance, Explainability, Calibration, and Faithfulness
The capability of Large Language Models (LLMs) like ChatGPT to comprehend user intent and provide reasonable responses has made them extremely popular lately.
HarmBench: A Standardized Evaluation Framework for Automated Red Teaming and Robust Refusal
Automated red teaming holds substantial promise for uncovering and mitigating the risks associated with the malicious use of large language models (LLMs), yet the field lacks a standardized evaluation framework to rigorously assess new methods.
Characterizing Adversarial Subspaces Using Local Intrinsic Dimensionality
Deep Neural Networks (DNNs) have recently been shown to be vulnerable against adversarial examples, which are carefully crafted instances that can mislead DNNs to make errors during prediction.
On the Tool Manipulation Capability of Open-source Large Language Models
In this paper, we ask can we enhance open-source LLMs to be competitive to leading closed LLM APIs in tool manipulation, with practical amount of human supervision.
QuaRot: Outlier-Free 4-Bit Inference in Rotated LLMs
We introduce QuaRot, a new Quantization scheme based on Rotations, which is able to quantize LLMs end-to-end, including all weights, activations, and KV cache in 4 bits.
Disentangled High Quality Salient Object Detection
As a pixel-wise classification task, LRSCN is designed to capture sufficient semantics at low-resolution to identify the definite salient, background and uncertain image regions.
Ranked #10 on
RGB Salient Object Detection
on DAVIS-S
Neural Attention Distillation: Erasing Backdoor Triggers from Deep Neural Networks
NAD utilizes a teacher network to guide the finetuning of the backdoored student network on a small clean subset of data such that the intermediate-layer attention of the student network aligns with that of the teacher network.
MixGen: A New Multi-Modal Data Augmentation
Data augmentation is a necessity to enhance data efficiency in deep learning.
Robust Deep Reinforcement Learning against Adversarial Perturbations on State Observations
Several works have shown this vulnerability via adversarial attacks, but existing approaches on improving the robustness of DRL under this setting have limited success and lack for theoretical principles.
Efficient Probabilistic Logic Reasoning with Graph Neural Networks
In this paper, we explore the combination of MLNs and GNNs, and use graph neural networks for variational inference in MLN.
Robust Physical-World Attacks on Deep Learning Models
We propose a general attack algorithm, Robust Physical Perturbations (RP2), to generate robust visual adversarial perturbations under different physical conditions.
SoK: Certified Robustness for Deep Neural Networks
Great advances in deep neural networks (DNNs) have led to state-of-the-art performance on a wide range of tasks.
Generating 3D Adversarial Point Clouds
Deep neural networks are known to be vulnerable to adversarial examples which are carefully crafted instances to cause the models to make wrong predictions.
PixMix: Dreamlike Pictures Comprehensively Improve Safety Measures
In real-world applications of machine learning, reliable and safe systems must consider measures of performance beyond standard test set accuracy.
FunQA: Towards Surprising Video Comprehension
Surprising videos, such as funny clips, creative performances, or visual illusions, attract significant attention.
AnimeRun: 2D Animation Visual Correspondence from Open Source 3D Movies
Existing correspondence datasets for two-dimensional (2D) cartoon suffer from simple frame composition and monotonic movements, making them insufficient to simulate real animations.
PointCloud Saliency Maps
Our motivation for constructing a saliency map is by point dropping, which is a non-differentiable operator.
Anti-Backdoor Learning: Training Clean Models on Poisoned Data
From this view, we identify two inherent characteristics of backdoor attacks as their weaknesses: 1) the models learn backdoored data much faster than learning with clean data, and the stronger the attack the faster the model converges on backdoored data; 2) the backdoor task is tied to a specific class (the backdoor target class).
UnrealPerson: An Adaptive Pipeline towards Costless Person Re-identification
The main difficulty of person re-identification (ReID) lies in collecting annotated data and transferring the model across different domains.
Spatially Transformed Adversarial Examples
Perturbations generated through spatial transformation could result in large $\mathcal{L}_p$ distance measures, but our extensive experiments show that such spatially transformed adversarial examples are perceptually realistic and more difficult to defend against with existing defense systems.
TStarBots: Defeating the Cheating Level Builtin AI in StarCraft II in the Full Game
Both TStarBot1 and TStarBot2 are able to defeat the built-in AI agents from level 1 to level 10 in a full game (1v1 Zerg-vs-Zerg game on the AbyssalReef map), noting that level 8, level 9, and level 10 are cheating agents with unfair advantages such as full vision on the whole map and resource harvest boosting.
Invisible for both Camera and LiDAR: Security of Multi-Sensor Fusion based Perception in Autonomous Driving Under Physical-World Attacks
In this paper, we present the first study of security issues of MSF-based perception in AD systems.
CRFL: Certifiably Robust Federated Learning against Backdoor Attacks
Our method exploits clipping and smoothing on model parameters to control the global model smoothness, which yields a sample-wise robustness certification on backdoors with limited magnitude.
RGM: A Robust Generalizable Matching Model
Finding corresponding pixels within a pair of images is a fundamental computer vision task with various applications.
Towards Efficient Data Valuation Based on the Shapley Value
In this paper, we study the problem of data valuation by utilizing the Shapley value, a popular notion of value which originated in cooperative game theory.
Efficient Task-Specific Data Valuation for Nearest Neighbor Algorithms
The most surprising result is that for unweighted $K$NN classifiers and regressors, the Shapley value of all $N$ data points can be computed, exactly, in $O(N\log N)$ time -- an exponential improvement on computational complexity!
G-PATE: Scalable Differentially Private Data Generator via Private Aggregation of Teacher Discriminators
In particular, we train a student data generator with an ensemble of teacher discriminators and propose a novel private gradient aggregation mechanism to ensure differential privacy on all information that flows from teacher discriminators to the student generator.
Bridging Multi-Task Learning and Meta-Learning: Towards Efficient Training and Effective Adaptation
Despite the subtle difference between MTL and meta-learning in the problem formulation, both learning paradigms share the same insight that the shared structure between existing training tasks could lead to better generalization and adaptation.
Ranked #17 on
Few-Shot Image Classification
on FC100 5-way (1-shot)
ChebNet: Efficient and Stable Constructions of Deep Neural Networks with Rectified Power Units via Chebyshev Approximations
As spectral accuracy is hard to obtain by direct training of deep neural networks, ChebNets provide a practical way to obtain spectral accuracy, it is expected to be useful in real applications that require efficient approximations of smooth functions.
Exploring the Space of Black-box Attacks on Deep Neural Networks
An iterative variant of our attack achieves close to 100% adversarial success rates for both targeted and untargeted attacks on DNNs.
Detecting Camouflaged Object in Frequency Domain
We first design a novel frequency enhancement module (FEM) to dig clues of camouflaged objects in the frequency domain.
Detecting AI Trojans Using Meta Neural Analysis
To train the meta-model without knowledge of the attack strategy, we introduce a technique called jumbo learning that samples a set of Trojaned models following a general distribution.
Referring Image Segmentation via Cross-Modal Progressive Comprehension
In addition to the CMPC module, we further leverage a simple yet effective TGFE module to integrate the reasoned multimodal features from different levels with the guidance of textual information.
Ranked #14 on
Referring Expression Segmentation
on RefCOCO testB
Self-Supervised Pretraining Improves Self-Supervised Pretraining
Through experimentation on 16 diverse vision datasets, we show HPT converges up to 80x faster, improves accuracy across tasks, and improves the robustness of the self-supervised pretraining process to changes in the image augmentation policy or amount of pretraining data.
Cross-Modal Progressive Comprehension for Referring Segmentation
In this paper, we propose a Cross-Modal Progressive Comprehension (CMPC) scheme to effectively mimic human behaviors and implement it as a CMPC-I (Image) module and a CMPC-V (Video) module to improve referring image and video segmentation models.
Ranked #7 on
Referring Expression Segmentation
on J-HMDB
Bipartite Graph Pre-training for Unsupervised Extractive Summarization with Graph Convolutional Auto-Encoders
Pre-trained sentence representations are crucial for identifying significant sentences in unsupervised document extractive summarization.
Constrained Variational Policy Optimization for Safe Reinforcement Learning
Safe reinforcement learning (RL) aims to learn policies that satisfy certain constraints before deploying them to safety-critical applications.
Efficient Multi-View Stereo by Iterative Dynamic Cost Volume
Specifically, a lightweight 3D CNN is utilized to generate the coarsest initial depth map which is essential to launch the GRU and guarantee a fast convergence.
SemanticAdv: Generating Adversarial Examples via Attribute-conditional Image Editing
In this paper, we aim to explore the impact of semantic manipulation on DNNs predictions by manipulating the semantic attributes of images and generate "unrestricted adversarial examples".
SemanticAdv: Generating Adversarial Examples via Attribute-conditioned Image Editing
Deep neural networks (DNNs) have achieved great successes in various vision applications due to their strong expressive power.
Orthogonal Weight Normalization: Solution to Optimization over Multiple Dependent Stiefel Manifolds in Deep Neural Networks
In this paper, we generalize such square orthogonal matrix to orthogonal rectangular matrix and formulating this problem in feed-forward Neural Networks (FNNs) as Optimization over Multiple Dependent Stiefel Manifolds (OMDSM).
Orthogonal Weight Normalization: Solution to Optimization overMultiple Dependent Stiefel Manifolds in Deep Neural Networks
In this paper, we generalize such square orthogonal matrix to orthogonal rectangular matrix and formulating this problem in feed-forward Neural Networks (FNNs) as Optimization over Multiple Dependent Stiefel Manifolds (OMDSM).
Panoptic Video Scene Graph Generation
PVSG relates to the existing video scene graph generation (VidSGG) problem, which focuses on temporal interactions between humans and objects grounded with bounding boxes in videos.
Target before Shooting: Accurate Anomaly Detection and Localization under One Millisecond via Cascade Patch Retrieval
In this framework, the anomaly detection problem is solved via a cascade patch retrieval procedure that retrieves the nearest neighbors for each test image patch in a coarse-to-fine fashion.
Ranked #1 on
Supervised Anomaly Detection
on BTAD
Manipulating Machine Learning: Poisoning Attacks and Countermeasures for Regression Learning
As machine learning becomes widely used for automated decisions, attackers have strong incentives to manipulate the results and models generated by machine learning algorithms.
Backbone is All Your Need: A Simplified Architecture for Visual Object Tracking
Exploiting a general-purpose neural architecture to replace hand-wired designs or inductive biases has recently drawn extensive interest.
Untargeted Backdoor Watermark: Towards Harmless and Stealthy Dataset Copyright Protection
In this paper, we revisit dataset ownership verification.
TrojDiff: Trojan Attacks on Diffusion Models with Diverse Targets
To answer these questions, we propose an effective Trojan attack against diffusion models, TrojDiff, which optimizes the Trojan diffusion and generative processes during training.
Rule-Guided Compositional Representation Learning on Knowledge Graphs
Representation learning on a knowledge graph (KG) is to embed entities and relations of a KG into low-dimensional continuous vector spaces.
UniFed: All-In-One Federated Learning Platform to Unify Open-Source Frameworks
The platform streamlines the end-to-end workflow for distributed experimentation and deployment, encompassing 11 popular open-source FL frameworks.
Invariant-Feature Subspace Recovery: A New Class of Provable Domain Generalization Algorithms
First, in the binary classification setup of Rosenfeld et al. (2021), we show that our first algorithm, ISR-Mean, can identify the subspace spanned by invariant features from the first-order moments of the class-conditional distributions, and achieve provable domain generalization with $d_s+1$ training environments.
CAKE: A Scalable Commonsense-Aware Framework For Multi-View Knowledge Graph Completion
The previous knowledge graph embedding (KGE) techniques suffer from invalid negative sampling and the uncertainty of fact-view link prediction, limiting KGC's performance.
A Quantitative Survey of Communication Optimizations in Distributed Deep Learning
In this article, we present a quantitative survey of communication optimization techniques for data parallel distributed DL.
RAB: Provable Robustness Against Backdoor Attacks
In addition, we theoretically show that it is possible to train the robust smoothed models efficiently for simple models such as K-nearest neighbor classifiers, and we propose an exact smooth-training algorithm that eliminates the need to sample from a noise distribution for such models.
DataLens: Scalable Privacy Preserving Training via Gradient Compression and Aggregation
In this paper, we aim to explore the power of generative models and gradient sparsity, and propose a scalable privacy-preserving generative model DATALENS.
On the Robustness of Safe Reinforcement Learning under Observational Perturbations
One interesting and counter-intuitive finding is that the maximum reward attack is strong, as it can both induce unsafe behaviors and make the attack stealthy by maintaining the reward.
"Bilingual Expert" Can Find Translation Errors
Recent advances in statistical machine translation via the adoption of neural sequence-to-sequence models empower the end-to-end system to achieve state-of-the-art in many WMT benchmarks.
Scalability vs. Utility: Do We Have to Sacrifice One for the Other in Data Importance Quantification?
Quantifying the importance of each training point to a learning task is a fundamental problem in machine learning and the estimated importance scores have been leveraged to guide a range of data workflows such as data summarization and domain adaption.
Relative Pose Estimation of Calibrated Cameras with Known SE(3) Invariants
In this paper, we present a complete comprehensive study of the relative pose estimation problem for a calibrated camera constrained by known $\mathrm{SE}(3)$ invariant, which involves 5 minimal problems in total.
Data Debugging with Shapley Importance over End-to-End Machine Learning Pipelines
We present DataScope (ease. ml/datascope), the first system that efficiently computes Shapley values of training examples over an end-to-end ML pipeline, and illustrate its applications in data debugging for ML training.
Cross Domain Object Detection by Target-Perceived Dual Branch Distillation
It can adaptively enhance source detector to perceive objects in a target image, by leveraging target proposal contexts from iterative cross-attention.
Robust Inference via Generative Classifiers for Handling Noisy Labels
Large-scale datasets may contain significant proportions of noisy (incorrect) class labels, and it is well-known that modern deep neural networks (DNNs) poorly generalize from such noisy training datasets.
Attack-Resistant Federated Learning with Residual-based Reweighting
Federated learning has a variety of applications in multiple domains by utilizing private training data stored on different devices.
Adversarial Texts with Gradient Methods
In addition, we empirically show that WMD is closely related to the quality of adversarial texts.
Improved Noisy Student Training for Automatic Speech Recognition
Noisy student training is an iterative self-training method that leverages augmentation to improve network performance.
Ranked #5 on
Speech Recognition
on LibriSpeech test-clean
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+2
GAPS: Generator for Automatic Polynomial Solvers
Minimal problems in computer vision raise the demand of generating efficient automatic solvers for polynomial equation systems.
Neural Abstract Style Transfer for Chinese Traditional Painting
In experiments, the proposed method shows more appealing stylized results in transferring the style of Chinese traditional painting than state-of-the-art neural style transfer methods.
Decision Boundary Analysis of Adversarial Examples
We find that the boundaries around these adversarial examples do not resemble the boundaries around benign examples.
Gotta Catch 'Em All: Using Honeypots to Catch Adversarial Attacks on Neural Networks
Attackers' optimization algorithms gravitate towards trapdoors, leading them to produce attacks similar to trapdoors in the feature space.
HALC: Object Hallucination Reduction via Adaptive Focal-Contrast Decoding
While large vision-language models (LVLMs) have demonstrated impressive capabilities in interpreting multi-modal contexts, they invariably suffer from object hallucinations (OH).
Reinforcement Learning with Perturbed Rewards
For instance, the state-of-the-art PPO algorithm is able to obtain 84. 6% and 80. 8% improvements on average score for five Atari games, with error rates as 10% and 30% respectively.
Imagine by Reasoning: A Reasoning-Based Implicit Semantic Data Augmentation for Long-Tailed Classification
Since the covariance matrix of each category represents the feature transformation directions, we can sample new directions from similar categories to generate definitely different instances.
Ranked #50 on
Long-tail Learning
on CIFAR-10-LT (ρ=10)
Generalizing Goal-Conditioned Reinforcement Learning with Variational Causal Reasoning
As a pivotal component to attaining generalizable solutions in human intelligence, reasoning provides great potential for reinforcement learning (RL) agents' generalization towards varied goals by summarizing part-to-whole arguments and discovering cause-and-effect relations.
T3: Tree-Autoencoder Constrained Adversarial Text Generation for Targeted Attack
In particular, we propose a tree-based autoencoder to embed the discrete text data into a continuous representation space, upon which we optimize the adversarial perturbation.
TextBugger: Generating Adversarial Text Against Real-world Applications
Deep Learning-based Text Understanding (DLTU) is the backbone technique behind various applications, including question answering, machine translation, and text classification.
Can Brain Signals Reveal Inner Alignment with Human Languages?
In addition, we provide interpretations of the performance improvement: (1) feature distribution shows the effectiveness of the alignment module for discovering and encoding the relationship between EEG and language; (2) alignment weights show the influence of different language semantics as well as EEG frequency features; (3) brain topographical maps provide an intuitive demonstration of the connectivity in the brain regions.
ChatTraffic: Text-to-Traffic Generation via Diffusion Model
The key challenge of the TTG task is how to associate text with the spatial structure of the road network and traffic data for generating traffic situations.
Robust Prompt Optimization for Defending Language Models Against Jailbreaking Attacks
Despite advances in AI alignment, language models (LM) remain vulnerable to adversarial attacks or jailbreaking, in which adversaries modify input prompts to induce harmful behavior.
A Novel Approach to Industrial Defect Generation through Blended Latent Diffusion Model with Online Adaptation
Effectively addressing the challenge of industrial Anomaly Detection (AD) necessitates an ample supply of defective samples, a constraint often hindered by their scarcity in industrial contexts.
BjTT: A Large-scale Multimodal Dataset for Traffic Prediction
The key challenge of the TTG task is how to associate text with the spatial structure of the road network and traffic data for generating traffic situations.
Single Camera Training for Person Re-identification
Differently, this paper investigates ReID in an unexplored single-camera-training (SCT) setting, where each person in the training set appears in only one camera.
Controllable Orthogonalization in Training DNNs
Orthogonality is widely used for training deep neural networks (DNNs) due to its ability to maintain all singular values of the Jacobian close to 1 and reduce redundancy in representation.
What Would Jiminy Cricket Do? Towards Agents That Behave Morally
When making everyday decisions, people are guided by their conscience, an internal sense of right and wrong.
MMSum: A Dataset for Multimodal Summarization and Thumbnail Generation of Videos
To address these challenges and provide a comprehensive dataset for this new direction, we have meticulously curated the \textbf{MMSum} dataset.
REFIT: A Unified Watermark Removal Framework For Deep Learning Systems With Limited Data
The experimental results demonstrate that our fine-tuning based watermark removal attacks could pose real threats to the copyright of pre-trained models, and thus highlight the importance of further investigating the watermarking problem and proposing more robust watermark embedding schemes against the attacks.
TRS: Transferability Reduced Ensemble via Encouraging Gradient Diversity and Model Smoothness
To answer these questions, in this work we first theoretically analyze and outline sufficient conditions for adversarial transferability between models; then propose a practical algorithm to reduce the transferability between base models within an ensemble to improve its robustness.
Heterogeneous Risk Minimization
In this paper, we propose Heterogeneous Risk Minimization (HRM) framework to achieve joint learning of latent heterogeneity among the data and invariant relationship, which leads to stable prediction despite distributional shifts.
Provable Domain Generalization via Invariant-Feature Subspace Recovery
Our first algorithm, ISR-Mean, can identify the subspace spanned by invariant features from the first-order moments of the class-conditional distributions, and achieve provable domain generalization with $d_s+1$ training environments under the data model of Rosenfeld et al. (2021).
Benchmarking Large Multimodal Models against Common Corruptions
This technical report aims to fill a deficiency in the assessment of large multimodal models (LMMs) by specifically examining the self-consistency of their outputs when subjected to common corruptions.
The Secret Revealer: Generative Model-Inversion Attacks Against Deep Neural Networks
This paper studies model-inversion attacks, in which the access to a model is abused to infer information about the training data.
Logic and Commonsense-Guided Temporal Knowledge Graph Completion
To address these challenges, we propose a Logic and Commonsense-Guided Embedding model (LCGE) to jointly learn the time-sensitive representation involving timeliness and causality of events, together with the time-independent representation of events from the perspective of commonsense.
Can SAM Segment Anything? When SAM Meets Camouflaged Object Detection
In this study, we try to ask if SAM can address the COD task and evaluate the performance of SAM on the COD benchmark by employing maximum segmentation evaluation and camouflage location evaluation.
Reconstructive Neuron Pruning for Backdoor Defense
Specifically, RNP first unlearns the neurons by maximizing the model's error on a small subset of clean samples and then recovers the neurons by minimizing the model's error on the same data.
TSS: Transformation-Specific Smoothing for Robustness Certification
Moreover, to the best of our knowledge, TSS is the first approach that achieves nontrivial certified robustness on the large-scale ImageNet dataset.
DENSE: Data-Free One-Shot Federated Learning
One-shot Federated Learning (FL) has recently emerged as a promising approach, which allows the central server to learn a model in a single communication round.
DeepMutation: Mutation Testing of Deep Learning Systems
To do this, by sharing the same spirit of mutation testing in traditional software, we first define a set of source-level mutation operators to inject faults to the source of DL (i. e., training data and training programs).
Software Engineering
SemAttack: Natural Textual Attacks via Different Semantic Spaces
In particular, SemAttack optimizes the generated perturbations constrained on generic semantic spaces, including typo space, knowledge space (e. g., WordNet), contextualized semantic space (e. g., the embedding space of BERT clusterings), or the combination of these spaces.
Unsupervised Learning of Accurate Siamese Tracking
As unlimited self-supervision signals can be obtained by tracking a video along a cycle in time, we investigate evolving a Siamese tracker by tracking videos forward-backward.
DP-OPT: Make Large Language Model Your Privacy-Preserving Prompt Engineer
To ensure that the prompts do not leak private information, we introduce the first private prompt generation mechanism, by a differentially-private (DP) ensemble of in-context learning with private demonstrations.
WordScape: a Pipeline to extract multilingual, visually rich Documents with Layout Annotations from Web Crawl Data
Together with the pipeline, we will additionally release 9. 5M urls to word documents which can be processed using WordScape to create a dataset of over 40M pages.
Uncovering the Connections Between Adversarial Transferability and Knowledge Transferability
Knowledge transferability, or transfer learning, has been widely adopted to allow a pre-trained model in the source domain to be effectively adapted to downstream tasks in the target domain.
Adversarial Attack Generation Empowered by Min-Max Optimization
In this paper, we show how a general framework of min-max optimization over multiple domains can be leveraged to advance the design of different types of adversarial attacks.
AI-GAN: Attack-Inspired Generation of Adversarial Examples
Deep neural networks (DNNs) are vulnerable to adversarial examples, which are crafted by adding imperceptible perturbations to inputs.
Learning Decomposed Representation for Counterfactual Inference
The fundamental problem in treatment effect estimation from observational data is confounder identification and balancing.
Sequence Generation with Label Augmentation for Relation Extraction
Sequence generation demonstrates promising performance in recent information extraction efforts, by incorporating large-scale pre-trained Seq2Seq models.
Ranked #1 on
Relation Extraction
on sciERC-sent
Shape Matters: Deformable Patch Attack
Though deep neural networks (DNNs) have demonstrated excellent performance in computer vision, they are susceptible and vulnerable to carefully crafted adversarial examples which can mislead DNNs to incorrect outputs.
Algorithmic Decision Making with Conditional Fairness
In practice, there frequently exist a certain set of variables we term as fair variables, which are pre-decision covariates such as users' choices.
Confidence Estimation for Attention-based Sequence-to-sequence Models for Speech Recognition
For various speech-related tasks, confidence scores from a speech recogniser are a useful measure to assess the quality of transcriptions.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+2
Which Style Makes Me Attractive? Interpretable Control Discovery and Counterfactual Explanation on StyleGAN
First, we propose a novel approach to disentangle latent subspace semantics by exploiting existing face analysis models, e. g., face parsers and face landmark detectors.
Towards Efficient Data Free Black-Box Adversarial Attack
The proposed method can efficiently imitate the target model through a small number of queries and achieve high attack success rate.
CU-Net: LiDAR Depth-Only Completion With Coupled U-Net
We find that existing depth-only methods can obtain satisfactory results in the areas where the measurement points are almost accurate and evenly distributed (denoted as normal areas), while the performance is limited in the areas where the foreground and background points are overlapped due to occlusion (denoted as overlap areas) and the areas where there are no measurement points around (denoted as blank areas) since the methods have no reliable input information in these areas.
Group-Wise Deep Object Co-Segmentation With Co-Attention Recurrent Neural Network
Effective feature representations which should not only express the images individual properties, but also reflect the interaction among group images are essentially crucial for real-world co-segmentation.
Rethinking Disparity: A Depth Range Free Multi-View Stereo Based on Disparity
Existing learning-based multi-view stereo (MVS) methods rely on the depth range to build the 3D cost volume and may fail when the range is too large or unreliable.
IMPRESS: Evaluating the Resilience of Imperceptible Perturbations Against Unauthorized Data Usage in Diffusion-Based Generative AI
IMPRESS is based on the key observation that imperceptible perturbations could lead to a perceptible inconsistency between the original image and the diffusion-reconstructed image, which can be used to devise a new optimization strategy for purifying the image, which may weaken the protection of the original image from unauthorized data usage (e. g., style mimicking, malicious editing).
DiffAttack: Evasion Attacks Against Diffusion-Based Adversarial Purification
In particular, we propose a deviated-reconstruction loss at intermediate diffusion steps to induce inaccurate density gradient estimation to tackle the problem of vanishing/exploding gradients.
MG-WFBP: Merging Gradients Wisely for Efficient Communication in Distributed Deep Learning
Distributed synchronous stochastic gradient descent has been widely used to train deep neural networks (DNNs) on computer clusters.
MG-WFBP: Efficient Data Communication for Distributed Synchronous SGD Algorithms
Distributed synchronous stochastic gradient descent has been widely used to train deep neural networks on computer clusters.
Distributed, Parallel, and Cluster Computing
Multi-Normal Estimation via Pair Consistency Voting
This paper presents a unified definition for point cloud normals of feature and non-feature points, which allows feature points to possess multiple normals.
Auto IV: Counterfactual Prediction via Automatic Instrumental Variable Decomposition
We also learn confounder representations by encouraging them to be relevant to both the treatment and the outcome.
Global Convergence of MAML and Theory-Inspired Neural Architecture Search for Few-Shot Learning
We show that the performance of MetaNTK-NAS is comparable or better than the state-of-the-art NAS method designed for few-shot learning while enjoying more than 100x speedup.
ArtPrompt: ASCII Art-based Jailbreak Attacks against Aligned LLMs
In this paper, we propose a novel ASCII art-based jailbreak attack and introduce a comprehensive benchmark Vision-in-Text Challenge (ViTC) to evaluate the capabilities of LLMs in recognizing prompts that cannot be solely interpreted by semantics.
On uncertainty estimation in active learning for image segmentation
This is especially critical in the data-driven active learning setting where the goal is to achieve a certain accuracy with minimum labeling effort.
Kernelized Heterogeneous Risk Minimization
The ability to generalize under distributional shifts is essential to reliable machine learning, while models optimized with empirical risk minimization usually fail on non-$i. i. d$ testing data.
Hierarchical Taxonomy-Aware and Attentional Graph Capsule RCNNs for Large-Scale Multi-Label Text Classification
In this paper, we propose a novel hierarchical taxonomy-aware and attentional graph capsule recurrent CNNs framework for large-scale multi-label text classification.
Knowledge Enhanced Machine Learning Pipeline against Diverse Adversarial Attacks
In particular, we develop KEMLP by integrating a diverse set of weak auxiliary models based on their logical relationships to the main DNN model that performs the target task.
Graph neural networks and attention-based CNN-LSTM for protein classification
This paper proposes three models for protein classification.
DeAR: Accelerating Distributed Deep Learning with Fine-Grained All-Reduce Pipelining
Communication scheduling has been shown to be effective in accelerating distributed training, which enables all-reduce communications to be overlapped with backpropagation computations.
Differentially Private Synthetic Data via Foundation Model APIs 2: Text
Lin et al. (2024) recently introduced the Private Evolution (PE) algorithm to generate DP synthetic images with only API access to diffusion models.
Deep Learning for Audio Signal Processing
Given the recent surge in developments of deep learning, this article provides a review of the state-of-the-art deep learning techniques for audio signal processing.
Reinforcement-Learning based Portfolio Management with Augmented Asset Movement Prediction States
Our framework aims to address two unique challenges in financial PM: (1) data heterogeneity -- the collected information for each asset is usually diverse, noisy and imbalanced (e. g., news articles); and (2) environment uncertainty -- the financial market is versatile and non-stationary.
Nonlinear Projection Based Gradient Estimation for Query Efficient Blackbox Attacks
We aim to bridge the gap between the two by investigating how to efficiently estimate gradient based on a projected low-dimensional space.
Competing for Shareable Arms in Multi-Player Multi-Armed Bandits
In reality, agents often have to learn and maximize the rewards of the resources at the same time.
Perceptive self-supervised learning network for noisy image watermark removal
Specifically, mentioned paired watermark images are obtained in a self supervised way, and paired noisy images (i. e., noisy and reference images) are obtained in a supervised way.
Multi-Task Dense Prediction via Mixture of Low-Rank Experts
Furthermore, to control the parameters and computational cost brought by the increase in the number of experts, we take inspiration from LoRA and propose to leverage the low-rank format of a vanilla convolution in the expert network.
Unrestricted Adversarial Examples via Semantic Manipulation
Machine learning models, especially deep neural networks (DNNs), have been shown to be vulnerable against adversarial examples which are carefully crafted samples with a small magnitude of the perturbation.
CROP: Certifying Robust Policies for Reinforcement Learning through Functional Smoothing
We then develop a local smoothing algorithm for policies derived from Q-functions to guarantee the robustness of actions taken along the trajectory; we also develop a global smoothing algorithm for certifying the lower bound of a finite-horizon cumulative reward, as well as a novel local smoothing algorithm to perform adaptive search in order to obtain tighter reward certification.
Adversarial GLUE: A Multi-Task Benchmark for Robustness Evaluation of Language Models
In this paper, we present Adversarial GLUE (AdvGLUE), a new multi-task benchmark to quantitatively and thoroughly explore and evaluate the vulnerabilities of modern large-scale language models under various types of adversarial attacks.
Ranked #1 on
Adversarial Robustness
on AdvGLUE
COPA: Certifying Robust Policies for Offline Reinforcement Learning against Poisoning Attacks
We leverage COPA to certify three RL environments trained with different algorithms and conclude: (1) The proposed robust aggregation protocols such as temporal aggregation can significantly improve the certifications; (2) Our certification for both per-state action stability and cumulative reward bound are efficient and tight; (3) The certification for different training algorithms and environments are different, implying their intrinsic robustness properties.
PhysioMTL: Personalizing Physiological Patterns using Optimal Transport Multi-Task Regression
The proposed method learns an individual-specific predictive model from heterogeneous observations, and enables estimation of an optimal transport map that yields a push forward operation onto the demographic features for each task.
Understanding Gradual Domain Adaptation: Improved Analysis, Optimal Path and Beyond
Gradual domain adaptation (GDA), on the other hand, assumes a path of $(T-1)$ unlabeled intermediate domains bridging the source and target, and aims to provide better generalization in the target domain by leveraging the intermediate ones.
Double Sampling Randomized Smoothing
To overcome this hurdle, we propose a Double Sampling Randomized Smoothing (DSRS) framework, which exploits the sampled probability from an additional smoothing distribution to tighten the robustness certification of the previous smoothed classifier.
AREA: Adaptive Reweighting via Effective Area for Long-Tailed Classification
Consequently, we estimate the size of the spanned space for each category, namely effective area, by detailedly analyzing its samples' distribution.
Ring-A-Bell! How Reliable are Concept Removal Methods for Diffusion Models?
While efforts have been made to mitigate such problems, either by implementing a safety filter at the evaluation stage or by fine-tuning models to eliminate undesirable concepts or styles, the effectiveness of these safety measures in dealing with a wide range of prompts remains largely unexplored.
Gradual Domain Adaptation: Theory and Algorithms
Unsupervised domain adaptation (UDA) adapts a model from a labeled source domain to an unlabeled target domain in a one-off way.
Projection Based Weight Normalization for Deep Neural Networks
We conduct comprehensive experiments on several widely-used image datasets including CIFAR-10, CIFAR-100, SVHN and ImageNet for supervised learning over the state-of-the-art convolutional neural networks, such as Inception, VGG and residual networks.
Adversarial Mutual Information for Text Generation
Recent advances in maximizing mutual information (MI) between the source and target have demonstrated its effectiveness in text generation.
FILTRA: Rethinking Steerable CNN by Filter Transform
It has been an intuitive and widely used technique to construct a steerable filter by augmenting a filter with its transformed copies in the past decades, which is named as filter transform in this paper.
Regulatory Instruments for Fair Personalized Pricing
Personalized pricing is a business strategy to charge different prices to individual consumers based on their characteristics and behaviors.
Evaluation and Optimization of Gradient Compression for Distributed Deep Learning
To accelerate distributed training, many gradient compression methods have been proposed to alleviate the communication bottleneck in synchronous stochastic gradient descent (S-SGD), but their efficacy in real-world applications still remains unclear.
Sparse Black-box Video Attack with Reinforcement Learning
Adversarial attacks on video recognition models have been explored recently.
FastEmit: Low-latency Streaming ASR with Sequence-level Emission Regularization
FastEmit also improves streaming ASR accuracy from 4. 4%/8. 9% to 3. 1%/7. 5% WER, meanwhile reduces 90th percentile latency from 210 ms to only 30 ms on LibriSpeech.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+2
Progressive-Scale Boundary Blackbox Attack via Projective Gradient Estimation
In this paper, we show that such efficiency highly depends on the scale at which the attack is applied, and attacking at the optimal scale significantly improves the efficiency.
How to Steer Your Adversary: Targeted and Efficient Model Stealing Defenses with Gradient Redirection
To meet these challenges, we present a new approach to model stealing defenses called gradient redirection.
Improving Long-tailed Object Detection with Image-Level Supervision by Multi-Task Collaborative Learning
Specifically, there are an object detection task (consisting of an instance-classification task and a localization task) and an image-classification task in our framework, responsible for utilizing the two types of supervision.
Rethinking the Learning Paradigm for Dynamic Facial Expression Recognition
Dynamic Facial Expression Recognition (DFER) is a rapidly developing field that focuses on recognizing facial expressions in video format.
Ranked #7 on
Dynamic Facial Expression Recognition
on FERV39k
Dynamic Facial Expression Recognition
Facial Expression Recognition
Multi-Task Learning Enhanced Single Image De-Raining
Rain removal in images is an important task in computer vision filed and attracting attentions of more and more people.
Product Ranking for Revenue Maximization with Multiple Purchases
In this paper, we assume that each consumer can purchase multiple products at will.
LOT: Layer-wise Orthogonal Training on Improving $\ell_2$ Certified Robustness
On the other hand, as existing works show that semi-supervised training helps improve empirical robustness, we aim to bridge the gap and prove that semi-supervised learning also improves the certified robustness of Lipschitz-bounded models.
On the effectiveness of partial variance reduction in federated learning with heterogeneous data
In this paper, we first revisit the widely used FedAvg algorithm in a deep neural network to understand how data heterogeneity influences the gradient updates across the neural network layers.
Invertible Convolution with Symmetric Paddings
We show that symmetrically padded convolution can be analytically inverted via DFT.
