Search Results for author:
Found 99 papers, 59 papers with code
Scalable Differential Privacy with Certified Robustness in Adversarial Learning
In this paper, we aim to develop a scalable algorithm to preserve differential privacy (DP) in adversarial learning for deep neural networks (DNNs), with certified robustness to adversarial examples.
小样本关系分类研究综述(Few-Shot Relation Classification: A Survey)
关系分类作为构建结构化知识的重要一环, 在自然语言处理领域备受关注。但在很多应用领域中(医疗、金融领域), 收集充足的用于训练关系分类模型的数据是十分困难的。近年来, 仅需要少量训练样本的小样本学习研究逐渐新兴于各大领域。本文对近期小样本关系分类模型与方法进行了系统的综述。根据度量方法的不同, 将现有方法分为原型式和分布式两大类。根据是否利用额外信息, 将模型分为预训练和非预训练两大类。此外, 除了常规设定下的小样本学习, 本文还梳理了跨领域和稀缺资源场景下的小样本学习, 并探讨了目前小样本关系分类方法的局限性, 分析了跨领域小样本 学习面临的技术挑战。最后, 展望了小样本关系分类未来的发展方向。
Human Pose as Compositional Tokens
Human pose is typically represented by a coordinate vector of body joints or their heatmap embeddings.
Ranked #1 on
Pose Estimation
on MPII Human Pose
Pseudo Supervised Metrics: Evaluating Unsupervised Image to Image Translation Models In Unsupervised Cross-Domain Classification Frameworks
In this paper, we introduce a new method called Pseudo Supervised Metrics that was designed specifically to support cross-domain classification applications contrary to other typically used metrics such as the FID which was designed to evaluate the model in terms of the quality of the generated image from a human-eye perspective.
Efficient Diffusion Training via Min-SNR Weighting Strategy
Denoising diffusion models have been a mainstream approach for image generation, however, training these models often suffers from slow convergence.
Ranked #1 on
Unconditional Image Generation
on ImageNet 256x256
DeepMIM: Deep Supervision for Masked Image Modeling
Deep supervision, which involves extra supervisions to the intermediate features of a neural network, was widely used in image classification in the early deep learning era since it significantly reduces the training difficulty and eases the optimization like avoiding gradient vanish over the vanilla training.
SGDA: Towards 3D Universal Pulmonary Nodule Detection via Slice Grouped Domain Attention
To address this issue, we propose a slice grouped domain attention (SGDA) module to enhance the generalization capability of the pulmonary nodule detection networks.
Subspace based Federated Unlearning
Federated learning (FL) enables multiple clients to train a machine learning model collaboratively without exchanging their local data.
Side Adapter Network for Open-Vocabulary Semantic Segmentation
A side network is attached to a frozen CLIP model with two branches: one for predicting mask proposals, and the other for predicting attention bias which is applied in the CLIP model to recognize the class of masks.
FedABC: Targeting Fair Competition in Personalized Federated Learning
In particular, we adopt the ``one-vs-all'' training strategy in each client to alleviate the unfair competition between classes by constructing a personalized binary classification problem for each class.
Training-free Lexical Backdoor Attacks on Language Models
In this work, we propose Training-Free Lexical Backdoor Attack (TFLexAttack) as the first training-free backdoor attack on language models.
All in Tokens: Unifying Output Space of Visual Tasks via Soft Token
With these new techniques and other designs, we show that the proposed general-purpose task-solver can perform both instance segmentation and depth estimation well.
Ranked #3 on
Monocular Depth Estimation
on NYU-Depth V2
TinyMIM: An Empirical Study of Distilling MIM Pre-trained Models
Our TinyMIM model of tiny size achieves 79. 6% top-1 accuracy on ImageNet-1K image classification, which sets a new record for small vision models of the same size and computation budget.
A Framework for Generalizing Critical Heat Flux Detection Models Using Unsupervised Image-to-Image Translation
To deal with datasets from new domains a model needs to be trained from scratch.
Attentive Mask CLIP
To address this issue, we propose an attentive token removal approach for CLIP training, which retains tokens with a high semantic correlation to the text description.
ResFormer: Scaling ViTs with Multi-Resolution Training
Vision Transformers (ViTs) have achieved overwhelming success, yet they suffer from vulnerable resolution scalability, i. e., the performance drops drastically when presented with input resolutions that are unseen during training.
SVFormer: Semi-supervised Video Transformer for Action Recognition
In this paper, we investigate the use of transformer models under the SSL setting for action recognition.
ClipCrop: Conditioned Cropping Driven by Vision-Language Model
Image cropping has progressed tremendously under the data-driven paradigm.
Exploring Discrete Diffusion Models for Image Captioning
The image captioning task is typically realized by an auto-regressive method that decodes the text tokens one by one.
Could Giant Pretrained Image Models Extract Universal Representations?
In this paper, we present a study of frozen pretrained models when applied to diverse and representative computer vision tasks, including object detection, semantic segmentation and video action recognition.
Ranked #3 on
Action Recognition In Videos
on Kinetics-400
Expediting Large-Scale Vision Transformer for Dense Prediction without Fine-tuning
Vision transformers have recently achieved competitive results across various vision tasks but still suffer from heavy computation costs when processing a large number of tokens.
One-to-Many Semantic Communication Systems: Design, Implementation, Performance Evaluation
To effectively exploit the benefits enabled by semantic communication, in this paper, we propose a one-to-many semantic communication system.
Not All Instances Contribute Equally: Instance-adaptive Class Representation Learning for Few-Shot Visual Recognition
To address the above issues, we propose a novel metric-based meta-learning framework termed instance-adaptive class representation learning network (ICRL-Net) for few-shot visual recognition.
Leveraging GAN Priors for Few-Shot Part Segmentation
Overall, this work is an attempt to explore the internal relevance between generation tasks and perception tasks by prompt designing.
DETRs with Hybrid Matching
This end-to-end signature is important for the versatility of DETR, and it has been generalized to a wide range of visual problems, including instance/semantic segmentation, human pose estimation, and point cloud/multi-view-images based detection, etc.
Lifelong DP: Consistently Bounded Differential Privacy in Lifelong Machine Learning
In this paper, we show that the process of continually learning new tasks and memorizing previous tasks introduces unknown privacy risks and challenges to bound the privacy loss.
On Data Scaling in Masked Image Modeling
Our study reveals that: (i) Masked image modeling is also demanding on larger data.
Tutel: Adaptive Mixture-of-Experts at Scale
On effectiveness, the SwinV2-MoE model achieves superior accuracy in both pre-training and down-stream computer vision tasks such as COCO object detection than the counterpart dense model, indicating the readiness of Tutel for end-to-end real-world model training and inference.
Contrastive Learning Rivals Masked Image Modeling in Fine-tuning via Feature Distillation
These properties, which we aggregately refer to as optimization friendliness, are identified and analyzed by a set of attention- and optimization-related diagnosis tools.
Ranked #2 on
Instance Segmentation
on COCO test-dev
(using extra training data)
Revealing the Dark Secrets of Masked Image Modeling
In this paper, we compare MIM with the long-dominant supervised pre-trained models from two perspectives, the visualizations and the experiments, to uncover their key representational differences.
Ranked #1 on
Monocular Depth Estimation
on KITTI Eigen split
CDFKD-MFS: Collaborative Data-free Knowledge Distillation via Multi-level Feature Sharing
To tackle this challenge, we propose a framework termed collaborative data-free knowledge distillation via multi-level feature sharing (CDFKD-MFS), which consists of a multi-header student module, an asymmetric adversarial data-free KD module, and an attention-based aggregation module.
Multi-Agent Collaborative Inference via DNN Decoupling: Intermediate Feature Compression and Edge Learning
In this paper, we study the multi-agent collaborative inference scenario, where a single edge server coordinates the inference of multiple UEs.
Deeper Insights into the Robustness of ViTs towards Common Corruptions
With Vision Transformers (ViTs) making great advances in a variety of computer vision tasks, recent literature have proposed various variants of vanilla ViTs to achieve better efficiency and efficacy.
iCAR: Bridging Image Classification and Image-text Alignment for Visual Recognition
Second, we convert the image classification problem from learning parametric category classifier weights to learning a text encoder as a meta network to generate category classifier weights.
Enhancing the Robustness, Efficiency, and Diversity of Differentiable Architecture Search
Differentiable architecture search (DARTS) has attracted much attention due to its simplicity and significant improvement in efficiency.
Region Rebalance for Long-Tailed Semantic Segmentation
In this paper, we study the problem of class imbalance in semantic segmentation.
Ranked #18 on
Semantic Segmentation
on ADE20K
RankSeg: Adaptive Pixel Classification with Image Category Ranking for Segmentation
Given an input image or video, our framework first conducts multi-label classification over the complete label, then sorts the complete label and selects a small subset according to their class confidence scores.
Energy Efficiency and Delay Tradeoff in an MEC-Enabled Mobile IoT Network
Theoretical analysis proves that the proposed algorithm can achieve a $[O(1/V), O(V)]$ tradeoff between EE and service delay.
Semi-Supervised Adversarial Recognition of Refined Window Structures for Inverse Procedural Façade Modeling
Unfortunately, the large amount of interactive sample labeling efforts has dramatically hindered the application of deep learning methods, especially for 3D modeling tasks, which require heterogeneous samples.
A Simple Baseline for Open-Vocabulary Semantic Segmentation with Pre-trained Vision-language Model
However, semantic segmentation and the CLIP model perform on different visual granularity, that semantic segmentation processes on pixels while CLIP performs on images.
Ranked #1 on
Open Vocabulary Semantic Segmentation
on Cityscapes
Swin Transformer V2: Scaling Up Capacity and Resolution
Three main techniques are proposed: 1) a residual-post-norm method combined with cosine attention to improve training stability; 2) A log-spaced continuous position bias method to effectively transfer models pre-trained using low-resolution images to downstream tasks with high-resolution inputs; 3) A self-supervised pre-training method, SimMIM, to reduce the needs of vast labeled images.
Ranked #4 on
Instance Segmentation
on COCO minival
(using extra training data)
SimMIM: A Simple Framework for Masked Image Modeling
We also leverage this approach to facilitate the training of a 3B model (SwinV2-G), that by $40\times$ less data than that in previous practice, we achieve the state-of-the-art on four representative vision benchmarks.
Representation Learning
Self-Supervised Image Classification
FLSys: Toward an Open Ecosystem for Federated Learning Mobile Apps
This article presents the design, implementation, and evaluation of FLSys, a mobile-cloud federated learning (FL) system, which can be a key component for an open ecosystem of FL models and apps.
Bootstrap Your Object Detector via Mixed Training
We introduce MixTraining, a new training paradigm for object detection that can improve the performance of existing detectors for free.
Joint Task Offloading and Resource Allocation for IoT Edge Computing with Sequential Task Dependency
To solve this challenging problem, we decompose it as a one-dimensional search of task offloading decision problem and a non-convex optimization problem with task offloading decision given.
Semi-Supervised Semantic Segmentation via Adaptive Equalization Learning
The confidence bank is leveraged as an indicator to tilt training towards under-performing categories, instantiated in three strategies: 1) adaptive Copy-Paste and CutMix data augmentation approaches which give more chance for under-performing categories to be copied or cut; 2) an adaptive data sampling approach to encourage pixels from under-performing category to be sampled; 3) a simple yet effective re-weighting method to alleviate the training noise raised by pseudo-labeling.
Meta-learning an Intermediate Representation for Few-shot Block-wise Prediction of Landslide Susceptibility
Then, the critical problem is that in each block with limited samples, conducting training and testing a model is impossible for a satisfactory LSM prediction, especially in dangerous mountainous areas where landslide surveying is expensive.
Energy-Efficient Design for IRS-Assisted MEC Networks with NOMA
Energy-efficient design is of crucial importance in wireless internet of things (IoT) networks.
Learning Efficient Vision Transformers via Fine-Grained Manifold Distillation
Specifically, we train a tiny student model to match a pre-trained teacher model in the patch-level manifold space.
Video Swin Transformer
The vision community is witnessing a modeling shift from CNNs to Transformers, where pure Transformer architectures have attained top accuracy on the major video recognition benchmarks.
Ranked #21 on
Action Classification
on Kinetics-600
(using extra training data)
End-to-End Semi-Supervised Object Detection with Soft Teacher
This paper presents an end-to-end semi-supervised object detection approach, in contrast to previous more complex multi-stage methods.
Ranked #4 on
Semi-Supervised Object Detection
on COCO 100% labeled data
(using extra training data)
Aligning Pretraining for Detection via Object-Level Contrastive Learning
Image-level contrastive representation learning has proven to be highly effective as a generic model for transfer learning.
TENSILE: A Tensor granularity dynamic GPU memory scheduling method toward multiple dynamic workloads system
In this paper, we demonstrated TENSILE, a method of managing GPU memory in tensor granularity to reduce the GPU memory peak, considering the multiple dynamic workloads.
Breaking Shortcut: Exploring Fully Convolutional Cycle-Consistency for Video Correspondence Learning
Previous cycle-consistency correspondence learning methods usually leverage image patches for training.
Self-Supervised Learning with Swin Transformers
We are witnessing a modeling shift from CNN to Transformers in computer vision.
Ranked #53 on
Self-Supervised Image Classification
on ImageNet
Group-Free 3D Object Detection via Transformers
Instead of grouping local points to each object candidate, our method computes the feature of an object from all the points in the point cloud with the help of an attention mechanism in the Transformers \cite{vaswani2017attention}, where the contribution of each point is automatically learned in the network training.
Ranked #3 on
3D Object Detection
on SUN-RGBD
Capsule Network is Not More Robust than Convolutional Network
The examination reveals five major new/different components in CapsNet: a transformation process, a dynamic routing layer, a squashing function, a marginal loss other than cross-entropy loss, and an additional class-conditional reconstruction loss for regularization.
Swin Transformer: Hierarchical Vision Transformer using Shifted Windows
This paper presents a new vision Transformer, called Swin Transformer, that capably serves as a general-purpose backbone for computer vision.
Ranked #2 on
Image Classification
on OmniBenchmark
Boosting Adversarial Transferability through Enhanced Momentum
Various momentum iterative gradient-based methods are shown to be effective to improve the adversarial transferability.
Mobility-Aware Offloading and Resource Allocation in MEC-Enabled IoT Networks
In this paper, the offloading decision and resource allocation problem is studied with mobility consideration.
Secure and Energy-Efficient Offloading and Resource Allocation in a NOMA-Based MEC Network
In this paper, we study the task offloading and resource allocation problem in a non-orthogonal multiple access (NOMA) assisted MEC network with security and energy efficiency considerations.
Robustness of on-device Models: Adversarial Attack to Deep Learning Models on Android Apps
Deep learning has shown its power in many applications, including object detection in images, natural-language understanding, and speech recognition.
Global Context Networks
The Non-Local Network (NLNet) presents a pioneering approach for capturing long-range dependencies within an image, via aggregating query-specific global context to each query position.
Ranked #34 on
Instance Segmentation
on COCO minival
Evading Web Application Firewalls with Reinforcement Learning
Our framework successfully discovered numbers of evasion payloads for each WAF in our experiments and can significantly outperform baseline policy.
Depth-Enhanced Feature Pyramid Network for Occlusion-Aware Verification of Buildings from Oblique Images
Finally, multi-view information from both the nadir and oblique images is used in a robust voting procedure to label changes in existing buildings.
Joint Task Offloading and Resource Allocation for IoT Edge Computing with Sequential Task Dependency
To solve this challenging problem, we decompose it as a one-dimensional search of task offloading decision problem and a non-convex optimization problem with task offloading decision given.
Edge-computing
Information Theory
Information Theory
Structure-Aware Completion of Photogrammetric Meshes in Urban Road Environment
Finally, the completed rendered image is deintegrated to the original texture atlas and the triangles for the vehicles are also flattened for improved meshes.
Propagate Yourself: Exploring Pixel-Level Consistency for Unsupervised Visual Representation Learning
We argue that the power of contrastive learning has yet to be fully unleashed, as current methods are trained only on instance-level pretext tasks, leading to representations that may be sub-optimal for downstream tasks requiring dense pixel predictions.
RelationNet++: Bridging Visual Representations for Object Detection via Transformer Decoder
The proposed module is named \emph{bridging visual representations} (BVR).
Ranked #62 on
Object Detection
on COCO test-dev
RepPoints V2: Verification Meets Regression for Object Detection
Though RepPoints provides high performance, we find that its heavy reliance on regression for object localization leaves room for improvement.
Ranked #69 on
Object Detection
on COCO test-dev
A Closer Look at Local Aggregation Operators in Point Cloud Analysis
Our investigation reveals that despite the different designs of these operators, all of these operators make surprisingly similar contributions to the network performance under the same network input and feature numbers and result in the state-of-the-art accuracy on standard benchmarks.
Ranked #3 on
3D Semantic Segmentation
on PartNet
Parametric Instance Classification for Unsupervised Visual Feature Learning
This paper presents parametric instance classification (PIC) for unsupervised visual feature learning.
Disentangled Non-Local Neural Networks
This paper first studies the non-local block in depth, where we find that its attention computation can be split into two terms, a whitened pairwise term accounting for the relationship between two pixels and a unary term representing the saliency of every pixel.
Ranked #14 on
Semantic Segmentation
on Cityscapes test
Ontology-based Interpretable Machine Learning for Textual Data
In this paper, we introduce a novel interpreting framework that learns an interpretable model based on an ontology-based sampling technique to explain agnostic prediction models.
Memory Enhanced Global-Local Aggregation for Video Object Detection
We argue that there are two important cues for humans to recognize objects in videos: the global semantic information and the local localization information.
Ranked #6 on
Video Object Detection
on ImageNet VID
Negative Margin Matters: Understanding Margin in Few-shot Classification
This paper introduces a negative margin loss to metric learning based few-shot learning methods.
Leveraging Photogrammetric Mesh Models for Aerial-Ground Feature Point Matching Toward Integrated 3D Reconstruction
Second, aerial models are rendered to the initial ground views, in which the color, depth and normal images are obtained.
Fast and Regularized Reconstruction of Building Façades from Street-View Images using Binary Integer Programming
Regularized arrangement of primitives on building fa\c{c}ades to aligned locations and consistent sizes is important towards structured reconstruction of urban environment.
Deep Fusion of Local and Non-Local Features for Precision Landslide Recognition
The second uses a scale attention mechanism to guide the up-sampling of features from the coarse level by a learned weight map.
Dense RepPoints: Representing Visual Objects with Dense Point Sets
We present a new object representation, called Dense RepPoints, that utilizes a large set of points to describe an object at multiple levels, including both box level and pixel level.
MAP-Net: Multi Attending Path Neural Network for Building Footprint Extraction from Remote Sensed Imagery
This paper proposes a novel multi attending path neural network (MAP-Net) for accurately extracting multiscale building footprints and precise boundaries.
Differential Privacy in Adversarial Learning with Provable Robustness
In this paper, we aim to develop a novel mechanism to preserve differential privacy (DP) in adversarial learning for deep neural networks, with provable robustness to adversarial examples.
XCMRC: Evaluating Cross-lingual Machine Reading Comprehension
Chinese and English are rich-resource language pairs, in order to study low-resource cross-lingual machine reading comprehension (XMRC), besides defining the common XCMRC task which has no restrictions on use of external language resources, we also define the pseudo low-resource XCMRC task by limiting the language resources to be used.
Local Relation Networks for Image Recognition
The convolution layer has been the dominant feature extractor in computer vision for years.
Ranked #762 on
Image Classification
on ImageNet
Spatial-Temporal Relation Networks for Multi-Object Tracking
Recent progress in multiple object tracking (MOT) has shown that a robust similarity score is key to the success of trackers.
RepPoints: Point Set Representation for Object Detection
They furthermore do not require the use of anchors to sample a space of bounding boxes.
Ranked #82 on
Object Detection
on COCO minival
GCNet: Non-local Networks Meet Squeeze-Excitation Networks and Beyond
In this paper, we take advantage of this finding to create a simplified network based on a query-independent formulation, which maintains the accuracy of NLNet but with significantly less computation.
Ranked #52 on
Instance Segmentation
on COCO minival
Preserving Differential Privacy in Adversarial Learning with Provable Robustness
In this paper, we aim to develop a novel mechanism to preserve differential privacy (DP) in adversarial learning for deep neural networks, with provable robustness to adversarial examples.
Cryptography and Security
Deep Metric Transfer for Label Propagation with Limited Annotated Data
In this paper, we propose a generic framework that utilizes unlabeled data to aid generalization for all three tasks.
Deformable ConvNets v2: More Deformable, Better Results
The superior performance of Deformable Convolutional Networks arises from its ability to adapt to the geometric variations of objects.
Ranked #119 on
Object Detection
on COCO minival
Learning Region Features for Object Detection
While most steps in the modern object detection methods are learnable, the region feature extraction step remains largely hand-crafted, featured by RoI pooling methods.
Relation Networks for Object Detection
Although it is well believed for years that modeling relations between objects would help object recognition, there has not been evidence that the idea is working in the deep learning era.
Adaptive Laplace Mechanism: Differential Privacy Preservation in Deep Learning
In this paper, we focus on developing a novel mechanism to preserve differential privacy in deep neural networks, such that: (1) The privacy budget consumption is totally independent of the number of training steps; (2) It has the ability to adaptively inject noise into features based on the contribution of each to the output; and (3) It could be applied in a variety of different deep neural networks.
WordSup: Exploiting Word Annotations for Character based Text Detection
When applied in scene text detection, we are thus able to train a robust character detector by exploiting word annotations in the rich large-scale real scene text datasets, e. g. ICDAR15 and COCO-text.
Ranked #4 on
Scene Text Detection
on ICDAR 2013
Deformable Convolutional Networks
Convolutional neural networks (CNNs) are inherently limited to model geometric transformations due to the fixed geometric structures in its building modules.
Ranked #215 on
Object Detection
on COCO test-dev
Power Data Classification: A Hybrid of a Novel Local Time Warping and LSTM
Finally, using the power consumption data from a real data center, we show that the proposed LTW can improve the classification accuracy of DTW from about 84% to 90%.
Smooth Representation Clustering
Based on our analysis, we propose the SMooth Representation (SMR) model.
Pose from Flow and Flow from Pose
The pose labeled segments and corresponding articulated joints are used to improve the motion flow fields by proposing kinematically constrained affine displacements on body parts.
