Search Results for author:
Found 613 papers, 255 papers with code
LTF: A Label Transformation Framework for Correcting Label Shift
Distribution shift is a major obstacle to the deployment of current deep learning models on real-world problems.
Hallucinating Visual Instances in Total Absentia
This seemingly minor difference in fact makes the HVITA a much challenging task, as the restoration algorithm would have to not only infer the category of the object in total absentia, but also hallucinate an object of which the appearance is consistent with the background.
Label-Noise Robust Domain Adaptation
Domain adaptation aims to correct the classifiers when faced with distribution shift between source (training) and target (test) domains.
Polysemy Deciphering Network for Human-Object Interaction Detection
First, PD-Net augments human pose and spatial features for HOI detection using language priors, enabling the verb classifiers to receive language hints that reduce the intra-class variation of the same verb.
The USYD-JD Speech Translation System for IWSLT2021
Our constrained system is based on a pipeline framework, i. e. ASR and NMT.
Deep Streaming Label Learning
In order to fill in these research gaps, we propose a novel deep neural network (DNN) based framework, Deep Streaming Label Learning (DSLL), to classify instances with newly emerged labels effectively.
Redistributing Low-Frequency Words: Making the Most of Monolingual Data in Non-Autoregressive Translation
In this work, we provide an appealing alternative for NAT – monolingual KD, which trains NAT student on external monolingual data with AT teacher trained on the original bilingual data.
On Dropping Clusters to Regularize Graph Convolutional Neural Networks
Dropout has been widely adopted to regularize graph convolutional networks (GCNs) by randomly zeroing entries of the node feature vectors and obtains promising performance on various tasks.
Divide, Conquer, and Combine: Mixture of Semantic-Independent Experts for Zero-Shot Dialogue State Tracking
Zero-shot transfer learning for Dialogue State Tracking (DST) helps to handle a variety of task-oriented dialogue domains without the cost of collecting in-domain data.
Cocktail: Mixing Multi-Modality Controls for Text-Conditional Image Generation
In this work, we propose Cocktail, a pipeline to mix various modalities into one embedding, amalgamated with a generalized ControlNet (gControlNet), a controllable normalisation (ControlNorm), and a spatial guidance sampling method, to actualize multi-modal and spatially-refined control for text-conditional diffusion models.
DeepSolo++: Let Transformer Decoder with Explicit Points Solo for Text Spotting
On the other hand, based on the extensibility of DeepSolo, we launch DeepSolo++ for multilingual text spotting, making a further step to let Transformer decoder with explicit points solo for multilingual text detection, recognition, and script identification all at once.
Learning to Learn from APIs: Black-Box Data-Free Meta-Learning
Data-free meta-learning (DFML) aims to enable efficient learning of new tasks by meta-learning from a collection of pre-trained models without access to the training data.
All Points Matter: Entropy-Regularized Distribution Alignment for Weakly-supervised 3D Segmentation
We hypothesize that this selective usage arises from the noise in pseudo-labels generated on unlabeled data.
Learning Better with Less: Effective Augmentation for Sample-Efficient Visual Reinforcement Learning
Taking the non-stationary nature of RL into account, we propose a RL-tailored multi-type DA fusion scheme called Cycling Augmentation (CycAug), which performs periodic cycles of different DA operations to increase type diversity while maintaining data distribution consistency.
Self-Evolution Learning for Discriminative Language Model Pretraining
Masked language modeling, widely used in discriminative language model (e. g., BERT) pretraining, commonly adopts a random masking strategy.
Revisiting Token Dropping Strategy in Efficient BERT Pretraining
Token dropping is a recently-proposed strategy to speed up the pretraining of masked language models, such as BERT, by skipping the computation of a subset of the input tokens at several middle layers.
Towards More Suitable Personalization in Federated Learning via Decentralized Partial Model Training
Personalized federated learning (PFL) aims to produce the greatest personalized model for each client to face an insurmountable problem--data heterogeneity in real FL systems.
Improving Heterogeneous Model Reuse by Density Estimation
This paper studies multiparty learning, aiming to learn a model using the private data of different participants.
Self-Evolution Learning for Mixup: Enhance Data Augmentation on Few-Shot Text Classification Tasks
However, most of the mixup methods do not consider the varying degree of learning difficulty in different stages of training and generate new samples with one hot labels, resulting in the model over confidence.
VanillaNet: the Power of Minimalism in Deep Learning
In this study, we introduce VanillaNet, a neural network architecture that embraces elegance in design.
Dynamic Regularized Sharpness Aware Minimization in Federated Learning: Approaching Global Consistency and Smooth Landscape
In federated learning (FL), a cluster of local clients are chaired under the coordination of the global server and cooperatively train one model with privacy protection.
Prompt-Tuning Decision Transformer with Preference Ranking
Our work contributes to the advancement of prompt-tuning approaches in RL, providing a promising direction for optimizing large RL agents for specific preference tasks.
MMoT: Mixture-of-Modality-Tokens Transformer for Composed Multimodal Conditional Image Synthesis
To this end, we propose to generate images conditioned on the compositions of multimodal control signals, where modalities are imperfectly complementary, i. e., composed multimodal conditional image synthesis (CMCIS).
Revolutionizing Agrifood Systems with Artificial Intelligence: A Survey
With the world population rapidly increasing, transforming our agrifood systems to be more productive, efficient, safe, and sustainable is crucial to mitigate potential food shortages.
Scaling-up Remote Sensing Segmentation Dataset with Segment Anything Model
The success of the Segment Anything Model (SAM) demonstrates the significance of data-centric machine learning.
Scalable Mask Annotation for Video Text Spotting
Video text spotting refers to localizing, recognizing, and tracking textual elements such as captions, logos, license plates, signs, and other forms of text within consecutive video frames.
Towards the Flatter Landscape and Better Generalization in Federated Learning under Client-level Differential Privacy
To defend the inference attacks and mitigate the sensitive information leakages in Federated Learning (FL), client-level Differentially Private FL (DPFL) is the de-facto standard for privacy protection by clipping local updates and adding random noise.
Deep Graph Reprogramming
In this paper, we explore a novel model reusing task tailored for graph neural networks (GNNs), termed as "deep graph reprogramming".
Compositional 3D Human-Object Neural Animation
Human-object interactions (HOIs) are crucial for human-centric scene understanding applications such as human-centric visual generation, AR/VR, and robotics.
Segment Anything in Non-Euclidean Domains: Challenges and Opportunities
The recent work known as Segment Anything (SA) has made significant strides in pushing the boundaries of semantic segmentation into the era of foundation models.
HKNAS: Classification of Hyperspectral Imagery Based on Hyper Kernel Neural Architecture Search
Recent neural architecture search (NAS) based approaches have made great progress in hyperspectral image (HSI) classification tasks.
DCN-T: Dual Context Network with Transformer for Hyperspectral Image Classification
Hyperspectral image (HSI) classification is challenging due to spatial variability caused by complex imaging conditions.
Event-based Simultaneous Localization and Mapping: A Comprehensive Survey
This paper presents a timely and comprehensive review of event-based vSLAM algorithms that exploit the benefits of asynchronous and irregular event streams for localization and mapping tasks.
UVA: Towards Unified Volumetric Avatar for View Synthesis, Pose rendering, Geometry and Texture Editing
Experiments on multiple human avatars demonstrate that our UVA achieves competitive results in novel view synthesis and novel pose rendering while enabling local and independent editing of geometry and appearance.
Deep Image Matting: A Comprehensive Survey
Image matting refers to extracting precise alpha matte from natural images, and it plays a critical role in various downstream applications, such as image editing.
On Efficient Training of Large-Scale Deep Learning Models: A Literature Review
The field of deep learning has witnessed significant progress, particularly in computer vision (CV), natural language processing (NLP), and speech.
Quantum Imitation Learning
Despite remarkable successes in solving various complex decision-making tasks, training an imitation learning (IL) algorithm with deep neural networks (DNNs) suffers from the high computation burden.
VTAE: Variational Transformer Autoencoder with Manifolds Learning
This weak projection, however, can be addressed by a Riemannian metric, and we show that geodesics computation and accurate interpolations between data samples on the Riemannian manifold can substantially improve the performance of deep generative models.
BEVSimDet: Simulated Multi-modal Distillation in Bird's-Eye View for Multi-view 3D Object Detection
In this paper, we approach this challenge from the perspective of both architecture design and knowledge distillation and present a new simulated multi-modal 3D object detection method named BEVSimDet.
Vision Transformer with Quadrangle Attention
Window-based attention has become a popular choice in vision transformers due to its superior performance, lower computational complexity, and less memory footprint.
Towards Making the Most of ChatGPT for Machine Translation
We show that: 1) The performance of ChatGPT depends largely on temperature, and a lower temperature usually can achieve better performance; 2) Emphasizing the task information further improves ChatGPT's performance, particularly in complex MT tasks; 3) Introducing domain information can elicit ChatGPT's generalization ability and improve its performance in the specific domain; 4) ChatGPT tends to generate hallucinations for non-English-centric MT tasks, which can be partially addressed by our proposed prompts but still need to be highlighted for the MT/NLP community.
Error Analysis Prompting Enables Human-Like Translation Evaluation in Large Language Models: A Case Study on ChatGPT
Our results indicate that by combining Chain-of-Thoughts and Error Analysis, a new prompting method called \textbf{\texttt{Error Analysis Prompting}}, LLMs like ChatGPT can \textit{generate human-like MT evaluations at both the system and segment level}.
Make Landscape Flatter in Differentially Private Federated Learning
To defend the inference attacks and mitigate the sensitive information leakages in Federated Learning (FL), client-level Differentially Private FL (DPFL) is the de-facto standard for privacy protection by clipping local updates and adding random noise.
Architecture, Dataset and Model-Scale Agnostic Data-free Meta-Learning
The goal of data-free meta-learning is to learn useful prior knowledge from a collection of pre-trained models without accessing their training data.
Deep Learning for Camera Calibration and Beyond: A Survey
In this paper, we provide a comprehensive survey of learning-based camera calibration techniques, by analyzing their strengths and limitations.
Sensitivity-Aware Visual Parameter-Efficient Tuning
Visual Parameter-Efficient Tuning (VPET) has become a powerful alternative for full fine-tuning so as to adapt pre-trained vision models to downstream tasks, which only tunes a small number of parameters while freezing the vast majority ones to ease storage burden and optimization difficulty.
Visual Prompt Based Personalized Federated Learning
Then, the local model is trained on the input composed of raw data and a visual prompt to learn the distribution information contained in the prompt.
Upcycling Models under Domain and Category Shift
We examine the superiority of our GLC on multiple benchmarks with different category shift scenarios, including partial-set, open-set, and open-partial-set DA.
Ranked #1 on
Universal Domain Adaptation
on VisDA2017
Source-Free Domain Adaptation
Universal Domain Adaptation
+1
TriDet: Temporal Action Detection with Relative Boundary Modeling
In this paper, we present a one-stage framework TriDet for temporal action detection.
Ranked #1 on
Temporal Action Localization
on EPIC-KITCHENS-100
DDS2M: Self-Supervised Denoising Diffusion Spatio-Spectral Model for Hyperspectral Image Restoration
This is especially inappropriate for data-starved hyperspectral image (HSI) restoration.
Centroid-centered Modeling for Efficient Vision Transformer Pre-training
Our proposed approach, \textbf{CCViT}, leverages k-means clustering to obtain centroids for image modeling without supervised training of tokenizer model.
Graph Decision Transformer
Offline reinforcement learning (RL) is a challenging task, whose objective is to learn policies from static trajectory data without interacting with the environment.
Time Associated Meta Learning for Clinical Prediction
We view time-associated disease prediction as classification tasks at multiple time points.
ESceme: Vision-and-Language Navigation with Episodic Scene Memory
Vision-and-language navigation (VLN) simulates a visual agent that follows natural-language navigation instructions in real-world scenes.
OmniForce: On Human-Centered, Large Model Empowered and Cloud-Edge Collaborative AutoML System
Automated machine learning (AutoML) seeks to build ML models with minimal human effort.
AdaSAM: Boosting Sharpness-Aware Minimization with Adaptive Learning Rate and Momentum for Training Deep Neural Networks
Integrating SAM with adaptive learning rate and momentum acceleration, dubbed AdaSAM, has already been explored empirically to train large-scale deep neural networks without theoretical guarantee due to the triple difficulties in analyzing the coupled perturbation step, adaptive learning rate and momentum step.
Subspace based Federated Unlearning
Federated learning (FL) enables multiple clients to train a machine learning model collaboratively without exchanging their local data.
Learning to Generalize Provably in Learning to Optimize
While the optimizer generalization has been recently studied, the optimizee generalization (or learning to generalize) has not been rigorously studied in the L2O context, which is the aim of this paper.
Fusion of Global and Local Knowledge for Personalized Federated Learning
Personalized federated learning, as a variant of federated learning, trains customized models for clients using their heterogeneously distributed data.
FedSpeed: Larger Local Interval, Less Communication Round, and Higher Generalization Accuracy
Federated learning is an emerging distributed machine learning framework which jointly trains a global model via a large number of local devices with data privacy protections.
Pseudo Contrastive Learning for Graph-based Semi-supervised Learning
Pseudo Labeling is a technique used to improve the performance of semi-supervised Graph Neural Networks (GNNs) by generating additional pseudo-labels based on confident predictions.
X-Adv: Physical Adversarial Object Attacks against X-ray Prohibited Item Detection
In this paper, we take the first step toward the study of adversarial attacks targeted at X-ray prohibited item detection, and reveal the serious threats posed by such attacks in this safety-critical scenario.
Can ChatGPT Understand Too? A Comparative Study on ChatGPT and Fine-tuned BERT
Recently, ChatGPT has attracted great attention, as it can generate fluent and high-quality responses to human inquiries.
Bag of Tricks for Effective Language Model Pretraining and Downstream Adaptation: A Case Study on GLUE
This technical report briefly describes our JDExplore d-team's submission Vega v1 on the General Language Understanding Evaluation (GLUE) leaderboard, where GLUE is a collection of nine natural language understanding tasks, including question answering, linguistic acceptability, sentiment analysis, text similarity, paraphrase detection, and natural language inference.
Deep Learning for Event-based Vision: A Comprehensive Survey and Benchmarks
Being capable of capturing information in challenging visual conditions, event cameras have the potential to overcome the limitations of frame-based cameras in the computer vision and robotics community.
FedABC: Targeting Fair Competition in Personalized Federated Learning
In particular, we adopt the ``one-vs-all'' training strategy in each client to alleviate the unfair competition between classes by constructing a personalized binary classification problem for each class.
Enhance Local Consistency in Federated Learning: A Multi-Step Inertial Momentum Approach
Federated learning (FL), as a collaborative distributed training paradigm with several edge computing devices under the coordination of a centralized server, is plagued by inconsistent local stationary points due to the heterogeneity of the local partial participation clients, which precipitates the local client-drifts problems and sparks off the unstable and slow convergence, especially on the aggravated heterogeneous dataset.
PointWavelet: Learning in Spectral Domain for 3D Point Cloud Analysis
In this paper, we introduce a new method, PointWavelet, to explore local graphs in the spectral domain via a learnable graph wavelet transform.
Improving the Model Consistency of Decentralized Federated Learning
To mitigate the privacy leakages and communication burdens of Federated Learning (FL), decentralized FL (DFL) discards the central server and each client only communicates with its neighbors in a decentralized communication network.
AniPixel: Towards Animatable Pixel-Aligned Human Avatar
In this paper, we propose AniPixel, a novel animatable and generalizable human avatar reconstruction method that leverages pixel-aligned features for body geometry prediction and RGB color blending.
Neural Collapse Inspired Feature-Classifier Alignment for Few-Shot Class Incremental Learning
In this paper, we deal with this misalignment dilemma in FSCIL inspired by the recently discovered phenomenon named neural collapse, which reveals that the last-layer features of the same class will collapse into a vertex, and the vertices of all classes are aligned with the classifier prototypes, which are formed as a simplex equiangular tight frame (ETF).
class-incremental learning
Few-Shot Class-Incremental Learning
+1
Domain Re-Modulation for Few-Shot Generative Domain Adaptation
Specifically, DoRM freezes the source generator and introduces new mapping and affine modules (M&A modules) to capture the attributes of the target domain during GDA.
Neural Collapse Inspired Feature-Classifier Alignment for Few-Shot Class-Incremental Learning
In this paper, we deal with this misalignment dilemma in FSCIL inspired by the recently discovered phenomenon named neural collapse, which reveals that the last-layer features of the same class will collapse into a vertex, and the vertices of all classes are aligned with the classifier prototypes, which are formed as a simplex equiangular tight frame (ETF).
Ranked #2 on
Few-Shot Class-Incremental Learning
on CUB-200-2011
(Average Accuracy metric)
class-incremental learning
Few-Shot Class-Incremental Learning
+1
SaFormer: A Conditional Sequence Modeling Approach to Offline Safe Reinforcement Learning
Offline safe RL is of great practical relevance for deploying agents in real-world applications.
Global Nash Equilibrium in Non-convex Multi-player Game: Theory and Algorithms
This paper takes conjugate transformation to the formulation of non-convex multi-player games, and casts the complementary problem into a variational inequality (VI) problem with a continuous pseudo-gradient mapping.
A Survey of Self-Supervised Learning from Multiple Perspectives: Algorithms, Theory, Applications and Future Trends
To avoid the expensive cost incurred by collecting and labeling too many examples, as a subset of unsupervised learning, self-supervised learning (SSL) was proposed to learn good features from many unlabeled examples without any human-annotated labels.
A Comprehensive Survey of Dataset Distillation
Dataset distillation, a dataset reduction method, addresses this problem by synthesizing a small typical dataset from substantial data and has attracted much attention from the deep learning community.
PanopticPartFormer++: A Unified and Decoupled View for Panoptic Part Segmentation
Third, inspired by Mask2Former, based on our meta-architecture, we propose Panoptic-PartFormer++ and design a new part-whole cross-attention scheme to boost part segmentation qualities further.
From Images to Textual Prompts: Zero-Shot Visual Question Answering With Frozen Large Language Models
To address this issue, we propose Img2Prompt, a plug-and-play module that provides the prompts that can bridge the aforementioned modality and task disconnections, so that LLMs can perform zero-shot VQA tasks without end-to-end training.
Leverage Interactive Affinity for Affordance Learning
Perceiving potential "action possibilities" (i. e., affordance) regions of images and learning interactive functionalities of objects from human demonstration is a challenging task due to the diversity of human-object interactions.
Exploring the Relationship Between Architectural Design and Adversarially Robust Generalization
In particular, we comprehensively evaluated 20 most representative adversarially trained architectures on ImageNette and CIFAR-10 datasets towards multiple l_p-norm adversarial attacks.
Learnable Skeleton-Aware 3D Point Cloud Sampling
In this paper, we introduce a new skeleton-aware learning-to-sample method by learning object skeletons as the prior knowledge to preserve the object geometry and topology information during sampling.
On Transforming Reinforcement Learning by Transformer: The Development Trajectory
Transformer, originally devised for natural language processing, has also attested significant success in computer vision.
Demystify Problem-Dependent Power of Quantum Neural Networks on Multi-Class Classification
Using these findings, we propose a method that uses loss dynamics to probe whether a QC may be more effective than a classical classifier on a particular learning task.
From Images to Textual Prompts: Zero-shot VQA with Frozen Large Language Models
To address this issue, we propose \emph{Img2Prompt}, a plug-and-play module that provides the prompts that can bridge the aforementioned modality and task disconnections, so that LLMs can perform zero-shot VQA tasks without end-to-end training.
Toward Human-Like Evaluation for Natural Language Generation with Error Analysis
To this end, we augment BARTScore by incorporating the human-like error analysis strategies, namely BARTScore++, where the final score consists of both the evaluations of major errors and minor errors.
Original or Translated? On the Use of Parallel Data for Translation Quality Estimation
Machine Translation Quality Estimation (QE) is the task of evaluating translation output in the absence of human-written references.
Localizing Scan Targets from Human Pose for Autonomous Lung Ultrasound Imaging
Ultrasound is progressing toward becoming an affordable and versatile solution to medical imaging.
Safety Correction from Baseline: Towards the Risk-aware Policy in Robotics via Dual-agent Reinforcement Learning
On the other hand, the safe agent mimics the baseline agent for policy improvement and learns to fulfill safety constraints via off-policy RL tuning.
ContraFeat: Contrasting Deep Features for Semantic Discovery
In this paper, we propose a model that automates this process and achieves state-of-the-art semantic discovery performance.
Evaluating Model-free Reinforcement Learning toward Safety-critical Tasks
Despite a large number of reinforcement learning (RL) methods focusing on safety-critical tasks, there is still a lack of high-quality evaluation of those algorithms that adheres to safety constraints at each decision step under complex and unknown dynamics.
Diff-Font: Diffusion Model for Robust One-Shot Font Generation
Specifically, a large stroke-wise dataset is constructed, and a stroke-wise diffusion model is proposed to preserve the structure and the completion of each generated character.
Generating Holistic 3D Human Motion from Speech
This work addresses the problem of generating 3D holistic body motions from human speech.
ViTPose+: Vision Transformer Foundation Model for Generic Body Pose Estimation
In this paper, we show the surprisingly good properties of plain vision transformers for body pose estimation from various aspects, namely simplicity in model structure, scalability in model size, flexibility in training paradigm, and transferability of knowledge between models, through a simple baseline model dubbed ViTPose.
Ranked #1 on
Animal Pose Estimation
on AP-10K
Learning to Learn Better for Video Object Segmentation
Recently, the joint learning framework (JOINT) integrates matching based transductive reasoning and online inductive learning to achieve accurate and robust semi-supervised video object segmentation (SVOS).
Semantic Segmentation
Semi-Supervised Video Object Segmentation
+1
Toward Efficient Language Model Pretraining and Downstream Adaptation via Self-Evolution: A Case Study on SuperGLUE
This technical report briefly describes our JDExplore d-team's Vega v2 submission on the SuperGLUE leaderboard.
Improving Simultaneous Machine Translation with Monolingual Data
Preliminary experiments on En-Zh and En-Ja news domain corpora demonstrate that monolingual data can significantly improve translation quality (e. g., +3. 15 BLEU on En-Zh).
AL-iGAN: An Active Learning Framework for Tunnel Geological Reconstruction Based on TBM Operational Data
In tunnel boring machine (TBM) underground projects, an accurate description of the rock-soil types distributed in the tunnel can decrease the construction risk ({\it e. g.} surface settlement and landslide) and improve the efficiency of construction.
Unified Discrete Diffusion for Simultaneous Vision-Language Generation
The recently developed discrete diffusion models perform extraordinarily well in the text-to-image task, showing significant promise for handling the multi-modality signals.
3DDesigner: Towards Photorealistic 3D Object Generation and Editing with Text-guided Diffusion Models
Text-guided diffusion models have shown superior performance in image/video generation and editing.
Knowledge-Aware Federated Active Learning with Non-IID Data
In this paper, we propose a federated active learning paradigm to efficiently learn a global model with limited annotation budget while protecting data privacy in a decentralized learning way.
1st Workshop on Maritime Computer Vision (MaCVi) 2023: Challenge Results
The 1$^{\text{st}}$ Workshop on Maritime Computer Vision (MaCVi) 2023 focused on maritime computer vision for Unmanned Aerial Vehicles (UAV) and Unmanned Surface Vehicle (USV), and organized several subchallenges in this domain: (i) UAV-based Maritime Object Detection, (ii) UAV-based Maritime Object Tracking, (iii) USV-based Maritime Obstacle Segmentation and (iv) USV-based Maritime Obstacle Detection.
Responsible Active Learning via Human-in-the-loop Peer Study
To further enhance the active learner via large-scale unlabelled data, we introduce multiple peer students into the active learner which is trained by a novel learning paradigm, including the In-Class Peer Study on labelled data and the Out-of-Class Peer Study on unlabelled data.
SLLEN: Semantic-aware Low-light Image Enhancement Network
How to effectively explore semantic feature is vital for low-light image enhancement (LLE).
Cross-Modal Contrastive Learning for Robust Reasoning in VQA
For each positive pair, we regard the images from different graphs as negative samples and deduct the version of multi-positive contrastive learning.
Adaptive Edge-to-Edge Interaction Learning for Point Cloud Analysis
To explore the role of the relation between edges, this paper proposes a novel Adaptive Edge-to-Edge Interaction Learning module, which aims to enhance the point-to-point relation through modelling the edge-to-edge interaction in the local region adaptively.
DeepSolo: Let Transformer Decoder with Explicit Points Solo for Text Spotting
In this paper, we present DeepSolo, a simple DETR-like baseline that lets a single Decoder with Explicit Points Solo for text detection and recognition simultaneously.
PAD-Net: An Efficient Framework for Dynamic Networks
The main contributions of our work are challenging the basic commonsense in dynamic networks and proposing a partially dynamic network, namely PAD-Net, to transform the redundant dynamic parameters into static ones.
Unifying Flow, Stereo and Depth Estimation
We present a unified formulation and model for three motion and 3D perception tasks: optical flow, rectified stereo matching and unrectified stereo depth estimation from posed images.
Ranked #1 on
Optical Flow Estimation
on Sintel-clean
Rethinking Hierarchies in Pre-trained Plain Vision Transformer
Self-supervised pre-training vision transformer (ViT) via masked image modeling (MIM) has been proven very effective.
Adversarial Auto-Augment with Label Preservation: A Representation Learning Principle Guided Approach
In experiments, we show that our method consistently brings non-trivial improvements to the three aforementioned learning tasks from both efficiency and final performance, either or not combined with strong pre-defined augmentations, e. g., on medical images when domain knowledge is unavailable and the existing augmentation techniques perform poorly.
TASA: Deceiving Question Answering Models by Twin Answer Sentences Attack
We present Twin Answer Sentences Attack (TASA), an adversarial attack method for question answering (QA) models that produces fluent and grammatical adversarial contexts while maintaining gold answers.
Identifiability and Asymptotics in Learning Homogeneous Linear ODE Systems from Discrete Observations
This paper derives a sufficient condition for the identifiability of homogeneous linear ODE systems from a sequence of equally-spaced error-free observations sampled from a single trajectory.
Make Sharpness-Aware Minimization Stronger: A Sparsified Perturbation Approach
One of the popular solutions is Sharpness-Aware Minimization (SAM), which smooths the loss landscape via minimizing the maximized change of training loss when adding a perturbation to the weight.
Improving Sharpness-Aware Minimization with Fisher Mask for Better Generalization on Language Models
Fine-tuning large pretrained language models on a limited training corpus usually suffers from poor generalization.
Benefits of Permutation-Equivariance in Auction Mechanisms
Designing an incentive-compatible auction mechanism that maximizes the auctioneer's revenue while minimizes the bidders' ex-post regret is an important yet intricate problem in economics.
A Comprehensive Survey of Data Augmentation in Visual Reinforcement Learning
Visual reinforcement learning (RL), which makes decisions directly from high-dimensional visual inputs, has demonstrated significant potential in various domains.
SparseAdapter: An Easy Approach for Improving the Parameter-Efficiency of Adapters
Adapter Tuning, which freezes the pretrained language models (PLMs) and only fine-tunes a few extra modules, becomes an appealing efficient alternative to the full model fine-tuning.
Bridged Transformer for Vision and Point Cloud 3D Object Detection
Recently, there is a trend of leveraging multiple sources of input data, such as complementing the 3D point cloud with 2D images that often have richer color and fewer noises.
Alternating Differentiation for Optimization Layers
In this paper, we developed a new framework, named Alternating Differentiation (Alt-Diff), that differentiates optimization problems (here, specifically in the form of convex optimization problems with polyhedral constraints) in a fast and recursive way.
Exploring the Relationship between Architecture and Adversarially Robust Generalization
Inparticular, we comprehensively evaluated 20 most representative adversarially trained architectures on ImageNette and CIFAR-10 datasets towards multiple `p-norm adversarial attacks.
Shuffle-QUDIO: accelerate distributed VQE with trainability enhancement and measurement reduction
To gain such computational advantages on large-scale problems, a feasible solution is the QUantum DIstributed Optimization (QUDIO) scheme, which partitions the original problem into $K$ subproblems and allocates them to $K$ quantum machines followed by the parallel optimization.
Towards Robust Referring Image Segmentation
Our proposed RefSegformer achieves the new state-of-the-art results on three regular RIS datasets and three R-RIS datasets, which serves as a new solid baseline for further research.
Vega-MT: The JD Explore Academy Translation System for WMT22
As for model sizes, we scale the Transformer-Big up to the extremely large model that owns nearly 4. 7 Billion parameters, to fully enhance the model capacity for our Vega-MT.
Ranked #1 on
Machine Translation
on WMT 2022 Chinese-English
On Robust Cross-View Consistency in Self-Supervised Monocular Depth Estimation
Compared with the photometric consistency loss as well as the rigid point cloud alignment loss, the proposed DFA and VDA losses are more robust owing to the strong representation power of deep features as well as the high tolerance of voxel density to the aforementioned challenges.
On the Complementarity between Pre-Training and Random-Initialization for Resource-Rich Machine Translation
Pre-Training (PT) of text representations has been successfully applied to low-resource Neural Machine Translation (NMT).
Not All Instances Contribute Equally: Instance-adaptive Class Representation Learning for Few-Shot Visual Recognition
To address the above issues, we propose a novel metric-based meta-learning framework termed instance-adaptive class representation learning network (ICRL-Net) for few-shot visual recognition.
Super-model ecosystem: A domain-adaptation perspective
We model the super-model paradigm as a two-stage diffusion process: (1) in the pre-training stage, the model parameter diffuses from random initials and converges to a steady distribution; and (2) in the fine-tuning stage, the model parameter is transported to another steady distribution.
Symmetric Pruning in Quantum Neural Networks
To fill this knowledge gap, here we propose the effective quantum neural tangent kernel (EQNTK) and connect this concept with over-parameterization theory to quantify the convergence of QNNs towards the global optima.
Grounded Affordance from Exocentric View
Due to the diversity of interactive affordance, the uniqueness of different individuals leads to diverse interactions, which makes it difficult to establish an explicit link between object parts and affordance labels.
Shortcut Learning of Large Language Models in Natural Language Understanding
We first introduce the concepts of shortcut learning of language models.
Hierarchical Perceptual Noise Injection for Social Media Fingerprint Privacy Protection
The threat of fingerprint leakage from social media raises a strong desire for anonymizing shared images while maintaining image qualities, since fingerprints act as a lifelong individual biometric password.
PANDA: Prompt Transfer Meets Knowledge Distillation for Efficient Model Adaptation
In response to these problems, we propose a new metric to accurately predict the prompt transferability (regarding (i)), and a novel PoT approach (namely PANDA) that leverages the knowledge distillation technique to transfer the "knowledge" from the source prompt to the target prompt in a subtle manner and alleviate the catastrophic forgetting effectively (regarding (ii)).
Domain-Specific Risk Minimization for Out-of-Distribution Generalization
Our bound motivates two strategies to reduce the gap: the first one is ensembling multiple classifiers to enrich the hypothesis space, then we propose effective gap estimation methods for guiding the selection of a better hypothesis for the target.
Advancing Plain Vision Transformer Towards Remote Sensing Foundation Model
Large-scale vision foundation models have made significant progress in visual tasks on natural images, with vision transformers being the primary choice due to their good scalability and representation ability.
Balancing Stability and Plasticity through Advanced Null Space in Continual Learning
Continual learning is a learning paradigm that learns tasks sequentially with resources constraints, in which the key challenge is stability-plasticity dilemma, i. e., it is uneasy to simultaneously have the stability to prevent catastrophic forgetting of old tasks and the plasticity to learn new tasks well.
Learning Graph Neural Networks for Image Style Transfer
To this end, we develop an elaborated GNN model with content and style local patches as the graph vertices.
Online Continual Learning with Contrastive Vision Transformer
Based on the learnable focuses, we design a focal contrastive loss to rebalance contrastive learning between new and past classes and consolidate previously learned representations.
JPerceiver: Joint Perception Network for Depth, Pose and Layout Estimation in Driving Scenes
A naive way is to accomplish them independently in a sequential or parallel manner, but there are many drawbacks, i. e., 1) the depth and VO results suffer from the inherent scale ambiguity issue; 2) the BEV layout is directly predicted from the front-view image without using any depth-related information, although the depth map contains useful geometry clues for inferring scene layouts.
ST-P3: End-to-end Vision-based Autonomous Driving via Spatial-Temporal Feature Learning
In particular, we propose a spatial-temporal feature learning scheme towards a set of more representative features for perception, prediction and planning tasks simultaneously, which is called ST-P3.
ReAct: Temporal Action Detection with Relational Queries
Moreover, we propose two losses to facilitate and stabilize the training of action classification.
Ranked #7 on
Temporal Action Localization
on THUMOS’14
Transformer-based Context Condensation for Boosting Feature Pyramids in Object Detection
Current object detectors typically have a feature pyramid (FP) module for multi-level feature fusion (MFF) which aims to mitigate the gap between features from different levels and form a comprehensive object representation to achieve better detection performance.
Towards Scale-Aware, Robust, and Generalizable Unsupervised Monocular Depth Estimation by Integrating IMU Motion Dynamics
Unsupervised monocular depth and ego-motion estimation has drawn extensive research attention in recent years.
SFNet: Faster, Accurate, and Domain Agnostic Semantic Segmentation via Semantic Flow
In this paper, we focus on exploring effective methods for faster, accurate, and domain agnostic semantic segmentation.
DPText-DETR: Towards Better Scene Text Detection with Dynamic Points in Transformer
However, these methods built upon detection transformer framework might achieve sub-optimal training efficiency and performance due to coarse positional query modeling. In addition, the point label form exploited in previous works implies the reading order of humans, which impedes the detection robustness from our observation.
Ranked #1 on
Scene Text Detection
on SCUT-CTW1500
GFNet: Geometric Flow Network for 3D Point Cloud Semantic Segmentation
However, recent projection-based methods for point cloud semantic segmentation usually utilize a vanilla late fusion strategy for the predictions of different views, failing to explore the complementary information from a geometric perspective during the representation learning.
Ranked #1 on
Robust 3D Semantic Segmentation
on nuScenes-C
Dynamic Contrastive Distillation for Image-Text Retrieval
Although the vision-and-language pretraining (VLP) equipped cross-modal image-text retrieval (ITR) has achieved remarkable progress in the past two years, it suffers from a major drawback: the ever-increasing size of VLP models restricts its deployment to real-world search scenarios (where the high latency is unacceptable).
Topology-aware Generalization of Decentralized SGD
Our theory indicates that the generalizability of D-SGD is positively correlated with the spectral gap, and can explain why consensus control in initial training phase can ensure better generalization.
CLAMP: Prompt-based Contrastive Learning for Connecting Language and Animal Pose
Motivated by the progress of visual-language research, we propose that pre-trained language models (e. g., CLIP) can facilitate animal pose estimation by providing rich prior knowledge for describing animal keypoints in text.
Variational Distillation for Multi-View Learning
Information Bottleneck (IB) based multi-view learning provides an information theoretic principle for seeking shared information contained in heterogeneous data descriptions.
EATFormer: Improving Vision Transformer Inspired by Evolutionary Algorithm
Motivated by biological evolution, this paper explains the rationality of Vision Transformer by analogy with the proven practical Evolutionary Algorithm (EA) and derives that both have consistent mathematical formulation.
Boosting Factorization Machines via Saliency-Guided Mixup
Second, considering that MixFM may generate redundant or even detrimental instances, we further put forward a novel Factorization Machine powered by Saliency-guided Mixup (denoted as SMFM).
A Survey on Gradient Inversion: Attacks, Defenses and Future Directions
In particular, we dig out some critical ingredients from the iteration-based attacks, including data initialization, model training and gradient matching.
APT-36K: A Large-scale Benchmark for Animal Pose Estimation and Tracking
Based on APT-36K, we benchmark several representative models on the following three tracks: (1) supervised animal pose estimation on a single frame under intra- and inter-domain transfer learning settings, (2) inter-species domain generalization test for unseen animals, and (3) animal pose estimation with animal tracking.
Toward Real-world Single Image Deraining: A New Benchmark and Beyond
To address these issues, we establish a new high-quality dataset named RealRain-1k, consisting of $1, 120$ high-resolution paired clean and rainy images with low- and high-density rain streaks, respectively.
Referring Image Matting
Different from conventional image matting, which either requires user-defined scribbles/trimap to extract a specific foreground object or directly extracts all the foreground objects in the image indiscriminately, we introduce a new task named Referring Image Matting (RIM) in this paper, which aims to extract the meticulous alpha matte of the specific object that best matches the given natural language description, thus enabling a more natural and simpler instruction for image matting.
Ranked #1 on
Referring Image Matting (RefMatte-RW100)
on RefMatte
A Relational Intervention Approach for Unsupervised Dynamics Generalization in Model-Based Reinforcement Learning
However, because environments are not labelled, the extracted information inevitably contains redundant information unrelated to the dynamics in transition segments and thus fails to maintain a crucial property of $Z$: $Z$ should be similar in the same environment and dissimilar in different ones.
Model-based Reinforcement Learning
reinforcement-learning
+1
Recent Advances for Quantum Neural Networks in Generative Learning
Due to the intrinsic probabilistic nature of quantum mechanics, it is reasonable to postulate that quantum generative learning models (QGLMs) may surpass their classical counterparts.
Understanding deep learning via decision boundary
From the theoretical view, two lower bounds based on algorithm DB variability are proposed and do not explicitly depend on the sample size.
Modeling Image Composition for Complex Scene Generation
Compared to existing CNN-based and Transformer-based generation models that entangled modeling on pixel-level&patch-level and object-level&patch-level respectively, the proposed focal attention predicts the current patch token by only focusing on its highly-related tokens that specified by the spatial layout, thereby achieving disambiguation during training.
DisPFL: Towards Communication-Efficient Personalized Federated Learning via Decentralized Sparse Training
In this work, we propose a novel personalized federated learning framework in a decentralized (peer-to-peer) communication protocol named Dis-PFL, which employs personalized sparse masks to customize sparse local models on the edge.