Search Results for author:
Found 149 papers, 92 papers with code
Pose from Flow and Flow from Pose
The pose labeled segments and corresponding articulated joints are used to improve the motion flow fields by proposing kinematically constrained affine displacements on body parts.
Smooth Representation Clustering
Based on our analysis, we propose the SMooth Representation (SMR) model.
Power Data Classification: A Hybrid of a Novel Local Time Warping and LSTM
Finally, using the power consumption data from a real data center, we show that the proposed LTW can improve the classification accuracy of DTW from about 84% to 90%.
Deformable Convolutional Networks
Convolutional neural networks (CNNs) are inherently limited to model geometric transformations due to the fixed geometric structures in its building modules.
Ranked #3 on
Vessel Detection
on Vessel detection Dateset
WordSup: Exploiting Word Annotations for Character based Text Detection
When applied in scene text detection, we are thus able to train a robust character detector by exploiting word annotations in the rich large-scale real scene text datasets, e. g. ICDAR15 and COCO-text.
Ranked #4 on
Scene Text Detection
on ICDAR 2013
Adaptive Laplace Mechanism: Differential Privacy Preservation in Deep Learning
In this paper, we focus on developing a novel mechanism to preserve differential privacy in deep neural networks, such that: (1) The privacy budget consumption is totally independent of the number of training steps; (2) It has the ability to adaptively inject noise into features based on the contribution of each to the output; and (3) It could be applied in a variety of different deep neural networks.
Relation Networks for Object Detection
Although it is well believed for years that modeling relations between objects would help object recognition, there has not been evidence that the idea is working in the deep learning era.
Learning Region Features for Object Detection
While most steps in the modern object detection methods are learnable, the region feature extraction step remains largely hand-crafted, featured by RoI pooling methods.
Deformable ConvNets v2: More Deformable, Better Results
The superior performance of Deformable Convolutional Networks arises from its ability to adapt to the geometric variations of objects.
Ranked #116 on
Object Detection
on COCO test-dev
Deep Metric Transfer for Label Propagation with Limited Annotated Data
In this paper, we propose a generic framework that utilizes unlabeled data to aid generalization for all three tasks.
Preserving Differential Privacy in Adversarial Learning with Provable Robustness
In this paper, we aim to develop a novel mechanism to preserve differential privacy (DP) in adversarial learning for deep neural networks, with provable robustness to adversarial examples.
Cryptography and Security
RepPoints: Point Set Representation for Object Detection
They furthermore do not require the use of anchors to sample a space of bounding boxes.
Ranked #95 on
Object Detection
on COCO minival
GCNet: Non-local Networks Meet Squeeze-Excitation Networks and Beyond
In this paper, we take advantage of this finding to create a simplified network based on a query-independent formulation, which maintains the accuracy of NLNet but with significantly less computation.
Ranked #25 on
Object Detection
on COCO-O
Local Relation Networks for Image Recognition
The convolution layer has been the dominant feature extractor in computer vision for years.
Ranked #867 on
Image Classification
on ImageNet
Spatial-Temporal Relation Networks for Multi-Object Tracking
Recent progress in multiple object tracking (MOT) has shown that a robust similarity score is key to the success of trackers.
XCMRC: Evaluating Cross-lingual Machine Reading Comprehension
Chinese and English are rich-resource language pairs, in order to study low-resource cross-lingual machine reading comprehension (XMRC), besides defining the common XCMRC task which has no restrictions on use of external language resources, we also define the pseudo low-resource XCMRC task by limiting the language resources to be used.
Differential Privacy in Adversarial Learning with Provable Robustness
In this paper, we aim to develop a novel mechanism to preserve differential privacy (DP) in adversarial learning for deep neural networks, with provable robustness to adversarial examples.
MAP-Net: Multi Attending Path Neural Network for Building Footprint Extraction from Remote Sensed Imagery
This paper proposes a novel multi attending path neural network (MAP-Net) for accurately extracting multiscale building footprints and precise boundaries.
Dense RepPoints: Representing Visual Objects with Dense Point Sets
We present a new object representation, called Dense RepPoints, that utilizes a large set of points to describe an object at multiple levels, including both box level and pixel level.
Deep Fusion of Local and Non-Local Features for Precision Landslide Recognition
The second uses a scale attention mechanism to guide the up-sampling of features from the coarse level by a learned weight map.
Fast and Regularized Reconstruction of Building Façades from Street-View Images using Binary Integer Programming
Regularized arrangement of primitives on building fa\c{c}ades to aligned locations and consistent sizes is important towards structured reconstruction of urban environment.
Leveraging Photogrammetric Mesh Models for Aerial-Ground Feature Point Matching Toward Integrated 3D Reconstruction
Second, aerial models are rendered to the initial ground views, in which the color, depth and normal images are obtained.
Memory Enhanced Global-Local Aggregation for Video Object Detection
We argue that there are two important cues for humans to recognize objects in videos: the global semantic information and the local localization information.
Ranked #12 on
Video Object Detection
on ImageNet VID
Negative Margin Matters: Understanding Margin in Few-shot Classification
This paper introduces a negative margin loss to metric learning based few-shot learning methods.
Ontology-based Interpretable Machine Learning for Textual Data
In this paper, we introduce a novel interpreting framework that learns an interpretable model based on an ontology-based sampling technique to explain agnostic prediction models.
Disentangled Non-Local Neural Networks
This paper first studies the non-local block in depth, where we find that its attention computation can be split into two terms, a whitened pairwise term accounting for the relationship between two pixels and a unary term representing the saliency of every pixel.
Ranked #20 on
Semantic Segmentation
on Cityscapes test
(using extra training data)
Parametric Instance Classification for Unsupervised Visual Feature Learning
This paper presents parametric instance classification (PIC) for unsupervised visual feature learning.
A Closer Look at Local Aggregation Operators in Point Cloud Analysis
Our investigation reveals that despite the different designs of these operators, all of these operators make surprisingly similar contributions to the network performance under the same network input and feature numbers and result in the state-of-the-art accuracy on standard benchmarks.
Ranked #4 on
3D Semantic Segmentation
on PartNet
RepPoints V2: Verification Meets Regression for Object Detection
Though RepPoints provides high performance, we find that its heavy reliance on regression for object localization leaves room for improvement.
Ranked #27 on
Object Detection
on COCO-O
RelationNet++: Bridging Visual Representations for Object Detection via Transformer Decoder
The proposed module is named \emph{bridging visual representations} (BVR).
Ranked #53 on
Object Detection
on COCO test-dev
Propagate Yourself: Exploring Pixel-Level Consistency for Unsupervised Visual Representation Learning
We argue that the power of contrastive learning has yet to be fully unleashed, as current methods are trained only on instance-level pretext tasks, leading to representations that may be sub-optimal for downstream tasks requiring dense pixel predictions.
Structure-Aware Completion of Photogrammetric Meshes in Urban Road Environment
Finally, the completed rendered image is deintegrated to the original texture atlas and the triangles for the vehicles are also flattened for improved meshes.
Joint Task Offloading and Resource Allocation for IoT Edge Computing with Sequential Task Dependency
To solve this challenging problem, we decompose it as a one-dimensional search of task offloading decision problem and a non-convex optimization problem with task offloading decision given.
Edge-computing
Information Theory
Information Theory
Depth-Enhanced Feature Pyramid Network for Occlusion-Aware Verification of Buildings from Oblique Images
Finally, multi-view information from both the nadir and oblique images is used in a robust voting procedure to label changes in existing buildings.
Evading Web Application Firewalls with Reinforcement Learning
Our framework successfully discovered numbers of evasion payloads for each WAF in our experiments and can significantly outperform baseline policy.
Global Context Networks
The Non-Local Network (NLNet) presents a pioneering approach for capturing long-range dependencies within an image, via aggregating query-specific global context to each query position.
Ranked #40 on
Instance Segmentation
on COCO test-dev
Leveraging Batch Normalization for Vision Transformers
Transformer-based vision architectures have attracted great attention because of the strong performance over the convolutional neural networks (CNNs).
Robustness of on-device Models: Adversarial Attack to Deep Learning Models on Android Apps
Deep learning has shown its power in many applications, including object detection in images, natural-language understanding, and speech recognition.
Secure and Energy-Efficient Offloading and Resource Allocation in a NOMA-Based MEC Network
In this paper, we study the task offloading and resource allocation problem in a non-orthogonal multiple access (NOMA) assisted MEC network with security and energy efficiency considerations.
Mobility-Aware Offloading and Resource Allocation in MEC-Enabled IoT Networks
In this paper, the offloading decision and resource allocation problem is studied with mobility consideration.
Boosting Adversarial Transferability through Enhanced Momentum
Various momentum iterative gradient-based methods are shown to be effective to improve the adversarial transferability.
Swin Transformer: Hierarchical Vision Transformer using Shifted Windows
This paper presents a new vision Transformer, called Swin Transformer, that capably serves as a general-purpose backbone for computer vision.
Ranked #2 on
Image Classification
on OmniBenchmark
Capsule Network is Not More Robust than Convolutional Network
The examination reveals five major new/different components in CapsNet: a transformation process, a dynamic routing layer, a squashing function, a marginal loss other than cross-entropy loss, and an additional class-conditional reconstruction loss for regularization.
Group-Free 3D Object Detection via Transformers
Instead of grouping local points to each object candidate, our method computes the feature of an object from all the points in the point cloud with the help of an attention mechanism in the Transformers \cite{vaswani2017attention}, where the contribution of each point is automatically learned in the network training.
Ranked #3 on
3D Object Detection
on SUN-RGBD
Self-Supervised Learning with Swin Transformers
We are witnessing a modeling shift from CNN to Transformers in computer vision.
Ranked #75 on
Self-Supervised Image Classification
on ImageNet
Breaking Shortcut: Exploring Fully Convolutional Cycle-Consistency for Video Correspondence Learning
Previous cycle-consistency correspondence learning methods usually leverage image patches for training.
TENSILE: A Tensor granularity dynamic GPU memory scheduling method toward multiple dynamic workloads system
In this paper, we demonstrated TENSILE, a method of managing GPU memory in tensor granularity to reduce the GPU memory peak, considering the multiple dynamic workloads.
Aligning Pretraining for Detection via Object-Level Contrastive Learning
Image-level contrastive representation learning has proven to be highly effective as a generic model for transfer learning.
End-to-End Semi-Supervised Object Detection with Soft Teacher
This paper presents an end-to-end semi-supervised object detection approach, in contrast to previous more complex multi-stage methods.
Ranked #6 on
Semi-Supervised Object Detection
on COCO 100% labeled data
(using extra training data)
Video Swin Transformer
The vision community is witnessing a modeling shift from CNNs to Transformers, where pure Transformer architectures have attained top accuracy on the major video recognition benchmarks.
Ranked #28 on
Action Classification
on Kinetics-600
(using extra training data)
Learning Efficient Vision Transformers via Fine-Grained Manifold Distillation
Specifically, we train a tiny student model to match a pre-trained teacher model in the patch-level manifold space.
Energy-Efficient Design for IRS-Assisted MEC Networks with NOMA
Energy-efficient design is of crucial importance in wireless internet of things (IoT) networks.
Meta-learning an Intermediate Representation for Few-shot Block-wise Prediction of Landslide Susceptibility
Then, the critical problem is that in each block with limited samples, conducting training and testing a model is impossible for a satisfactory LSM prediction, especially in dangerous mountainous areas where landslide surveying is expensive.
Semi-Supervised Semantic Segmentation via Adaptive Equalization Learning
The confidence bank is leveraged as an indicator to tilt training towards under-performing categories, instantiated in three strategies: 1) adaptive Copy-Paste and CutMix data augmentation approaches which give more chance for under-performing categories to be copied or cut; 2) an adaptive data sampling approach to encourage pixels from under-performing category to be sampled; 3) a simple yet effective re-weighting method to alleviate the training noise raised by pseudo-labeling.
Joint Task Offloading and Resource Allocation for IoT Edge Computing with Sequential Task Dependency
To solve this challenging problem, we decompose it as a one-dimensional search of task offloading decision problem and a non-convex optimization problem with task offloading decision given.
Bootstrap Your Object Detector via Mixed Training
We introduce MixTraining, a new training paradigm for object detection that can improve the performance of existing detectors for free.
FLSys: Toward an Open Ecosystem for Federated Learning Mobile Apps
This article presents the design, implementation, and evaluation of FLSys, a mobile-cloud federated learning (FL) system, which can be a key component for an open ecosystem of FL models and apps.
Swin Transformer V2: Scaling Up Capacity and Resolution
Three main techniques are proposed: 1) a residual-post-norm method combined with cosine attention to improve training stability; 2) A log-spaced continuous position bias method to effectively transfer models pre-trained using low-resolution images to downstream tasks with high-resolution inputs; 3) A self-supervised pre-training method, SimMIM, to reduce the needs of vast labeled images.
Ranked #4 on
Image Classification
on ImageNet V2
(using extra training data)
SimMIM: A Simple Framework for Masked Image Modeling
We also leverage this approach to facilitate the training of a 3B model (SwinV2-G), that by $40\times$ less data than that in previous practice, we achieve the state-of-the-art on four representative vision benchmarks.
Representation Learning
Self-Supervised Image Classification
+1
A Simple Baseline for Open-Vocabulary Semantic Segmentation with Pre-trained Vision-language Model
However, semantic segmentation and the CLIP model perform on different visual granularity, that semantic segmentation processes on pixels while CLIP performs on images.
Semi-Supervised Adversarial Recognition of Refined Window Structures for Inverse Procedural Façade Modeling
Unfortunately, the large amount of interactive sample labeling efforts has dramatically hindered the application of deep learning methods, especially for 3D modeling tasks, which require heterogeneous samples.
Energy Efficiency and Delay Tradeoff in an MEC-Enabled Mobile IoT Network
Theoretical analysis proves that the proposed algorithm can achieve a $[O(1/V), O(V)]$ tradeoff between EE and service delay.
RankSeg: Adaptive Pixel Classification with Image Category Ranking for Segmentation
Given an input image or video, our framework first conducts multi-label classification over the complete label, then sorts the complete label and selects a small subset according to their class confidence scores.
Region Rebalance for Long-Tailed Semantic Segmentation
In this paper, we study the problem of class imbalance in semantic segmentation.
Ranked #21 on
Semantic Segmentation
on ADE20K
Enhancing the Robustness, Efficiency, and Diversity of Differentiable Architecture Search
Differentiable architecture search (DARTS) has attracted much attention due to its simplicity and significant improvement in efficiency.
iCAR: Bridging Image Classification and Image-text Alignment for Visual Recognition
Second, we convert the image classification problem from learning parametric category classifier weights to learning a text encoder as a meta network to generate category classifier weights.
Deeper Insights into the Robustness of ViTs towards Common Corruptions
With Vision Transformers (ViTs) making great advances in a variety of computer vision tasks, recent literature have proposed various variants of vanilla ViTs to achieve better efficiency and efficacy.
Multi-Agent Collaborative Inference via DNN Decoupling: Intermediate Feature Compression and Edge Learning
In this paper, we study the multi-agent collaborative inference scenario, where a single edge server coordinates the inference of multiple UEs.
CDFKD-MFS: Collaborative Data-free Knowledge Distillation via Multi-level Feature Sharing
To tackle this challenge, we propose a framework termed collaborative data-free knowledge distillation via multi-level feature sharing (CDFKD-MFS), which consists of a multi-header student module, an asymmetric adversarial data-free KD module, and an attention-based aggregation module.
Revealing the Dark Secrets of Masked Image Modeling
In this paper, we compare MIM with the long-dominant supervised pre-trained models from two perspectives, the visualizations and the experiments, to uncover their key representational differences.
Ranked #3 on
Depth Estimation
on NYU-Depth V2
Contrastive Learning Rivals Masked Image Modeling in Fine-tuning via Feature Distillation
These properties, which we aggregately refer to as optimization friendliness, are identified and analyzed by a set of attention- and optimization-related diagnosis tools.
Ranked #2 on
Instance Segmentation
on COCO test-dev
(using extra training data)
Tutel: Adaptive Mixture-of-Experts at Scale
On efficiency, Flex accelerates SwinV2-MoE, achieving up to 1. 55x and 2. 11x speedup in training and inference over Fairseq, respectively.
On Data Scaling in Masked Image Modeling
Our study reveals that: (i) Masked image modeling is also demanding on larger data.
DETRs with Hybrid Matching
One-to-one set matching is a key design for DETR to establish its end-to-end capability, so that object detection does not require a hand-crafted NMS (non-maximum suppression) to remove duplicate detections.
Lifelong DP: Consistently Bounded Differential Privacy in Lifelong Machine Learning
In this paper, we show that the process of continually learning new tasks and memorizing previous tasks introduces unknown privacy risks and challenges to bound the privacy loss.
Leveraging GAN Priors for Few-Shot Part Segmentation
Overall, this work is an attempt to explore the internal relevance between generation tasks and perception tasks by prompt designing.
Not All Instances Contribute Equally: Instance-adaptive Class Representation Learning for Few-Shot Visual Recognition
To address the above issues, we propose a novel metric-based meta-learning framework termed instance-adaptive class representation learning network (ICRL-Net) for few-shot visual recognition.
One-to-Many Semantic Communication Systems: Design, Implementation, Performance Evaluation
To effectively exploit the benefits enabled by semantic communication, in this paper, we propose a one-to-many semantic communication system.
Expediting Large-Scale Vision Transformer for Dense Prediction without Fine-tuning
Vision transformers have recently achieved competitive results across various vision tasks but still suffer from heavy computation costs when processing a large number of tokens.
Could Giant Pretrained Image Models Extract Universal Representations?
In this paper, we present a study of frozen pretrained models when applied to diverse and representative computer vision tasks, including object detection, semantic segmentation and video action recognition.
Ranked #3 on
Action Recognition In Videos
on Kinetics-400
Exploring Discrete Diffusion Models for Image Captioning
The image captioning task is typically realized by an auto-regressive method that decodes the text tokens one by one.
ClipCrop: Conditioned Cropping Driven by Vision-Language Model
Image cropping has progressed tremendously under the data-driven paradigm.
SVFormer: Semi-supervised Video Transformer for Action Recognition
In this paper, we investigate the use of transformer models under the SSL setting for action recognition.
ResFormer: Scaling ViTs with Multi-Resolution Training
We introduce, ResFormer, a framework that is built upon the seminal idea of multi-resolution training for improved performance on a wide spectrum of, mostly unseen, testing resolutions.
Attentive Mask CLIP
To address this issue, we propose an attentive token removal approach for CLIP training, which retains tokens with a high semantic correlation to the text description.
A Framework for Generalizing Critical Heat Flux Detection Models Using Unsupervised Image-to-Image Translation
To deal with datasets from new domains a model needs to be trained from scratch.
Improving CLIP Fine-tuning Performance
Experiments suggest that the feature map distillation approach significantly boosts the fine-tuning performance of CLIP models on several typical downstream vision tasks.
DETR Does Not Need Multi-Scale or Locality Design
This paper presents an improved DETR detector that maintains a "plain" nature: using a single-scale feature map and global cross-attention calculations without specific locality constraints, in contrast to previous leading DETR-based detectors that reintroduce architectural inductive biases of multi-scale and locality into the decoder.
iCLIP: Bridging Image Classification and Contrastive Language-Image Pre-Training for Visual Recognition
This paper presents a method that effectively combines two prevalent visual recognition methods, i. e., image classification and contrastive language-image pre-training, dubbed iCLIP.
TinyMIM: An Empirical Study of Distilling MIM Pre-trained Models
Our TinyMIM model of tiny size achieves 79. 6% top-1 accuracy on ImageNet-1K image classification, which sets a new record for small vision models of the same size and computation budget.
All in Tokens: Unifying Output Space of Visual Tasks via Soft Token
With these new techniques and other designs, we show that the proposed general-purpose task-solver can perform both instance segmentation and depth estimation well.
Ranked #14 on
Monocular Depth Estimation
on NYU-Depth V2
Training-free Lexical Backdoor Attacks on Language Models
In this work, we propose Training-Free Lexical Backdoor Attack (TFLexAttack) as the first training-free backdoor attack on language models.
FedABC: Targeting Fair Competition in Personalized Federated Learning
In particular, we adopt the ``one-vs-all'' training strategy in each client to alleviate the unfair competition between classes by constructing a personalized binary classification problem for each class.
Side Adapter Network for Open-Vocabulary Semantic Segmentation
A side network is attached to a frozen CLIP model with two branches: one for predicting mask proposals, and the other for predicting attention bias which is applied in the CLIP model to recognize the class of masks.
Subspace based Federated Unlearning
Federated learning (FL) enables multiple clients to train a machine learning model collaboratively without exchanging their local data.
SGDA: Towards 3D Universal Pulmonary Nodule Detection via Slice Grouped Domain Attention
To address this issue, we propose a slice grouped domain attention (SGDA) module to enhance the generalization capability of the pulmonary nodule detection networks.
DeepMIM: Deep Supervision for Masked Image Modeling
Deep supervision, which involves extra supervisions to the intermediate features of a neural network, was widely used in image classification in the early deep learning era since it significantly reduces the training difficulty and eases the optimization like avoiding gradient vanish over the vanilla training.
Efficient Diffusion Training via Min-SNR Weighting Strategy
Denoising diffusion models have been a mainstream approach for image generation, however, training these models often suffers from slow convergence.
Ranked #1 on
Image Generation
on ImageNet 256x256
Domain-knowledge Inspired Pseudo Supervision (DIPS) for Unsupervised Image-to-Image Translation Models to Support Cross-Domain Classification
Cross-domain classification frameworks were developed to handle this data domain shift problem by utilizing unsupervised image-to-image translation models to translate an input image from the unlabeled domain to the labeled domain.
Human Pose as Compositional Tokens
Human pose is typically represented by a coordinate vector of body joints or their heatmap embeddings.
Ranked #1 on
Pose Estimation
on MPII Human Pose
Semantic Image Translation for Repairing the Texture Defects of Building Models
Experimental results on publicly available fa\c{c}ade image and 3D model datasets demonstrate that our method yields superior results and effectively addresses issues associated with flawed textures.
SuperpixelGraph: Semi-automatic generation of building footprint through semantic-sensitive superpixel and neural graph networks
Most urban applications necessitate building footprints in the form of concise vector graphics with sharp boundaries rather than pixel-wise raster images.
BCE-Net: Reliable Building Footprints Change Extraction based on Historical Map and Up-to-Date Images using Contrastive Learning
This contrastive learning strategy allowed us to inject the semantics of buildings into a pipeline for the detection of changes, which is achieved by increasing the distinguishability of features of buildings from those of non-buildings.
Implicit Temporal Modeling with Learnable Alignment for Video Recognition
While modeling temporal information within straight through tube is widely adopted in literature, we find that simple frame alignment already provides enough essence without temporal attention.
Ranked #16 on
Action Classification
on Kinetics-400
EfficientViT: Memory Efficient Vision Transformer with Cascaded Group Attention
Comprehensive experiments demonstrate EfficientViT outperforms existing efficient models, striking a good trade-off between speed and accuracy.
Improving Heterogeneous Model Reuse by Density Estimation
This paper studies multiparty learning, aiming to learn a model using the private data of different participants.
VanillaKD: Revisit the Power of Vanilla Knowledge Distillation from Small Scale to Large Scale
The tremendous success of large models trained on extensive datasets demonstrates that scale is a key ingredient in achieving superior results.
GlyphControl: Glyph Conditional Control for Visual Text Generation
We evaluate the effectiveness of our approach by measuring OCR-based metrics, CLIP score, and FID of the generated visual text.
StructuredMesh: 3D Structured Optimization of Façade Components on Photogrammetric Mesh Models using Binary Integer Programming
The lack of fa\c{c}ade structures in photogrammetric mesh models renders them inadequate for meeting the demands of intricate applications.
FHA-Kitchens: A Novel Dataset for Fine-Grained Hand Action Recognition in Kitchen Scenes
We represent the action information in each hand interaction region as a triplet, resulting in a total of 878 action triplets.
LGViT: Dynamic Early Exiting for Accelerating Vision Transformer
Recently, the efficient deployment and acceleration of powerful vision transformers (ViTs) on resource-limited edge devices for providing multimedia services have become attractive tasks.
DETR Doesn't Need Multi-Scale or Locality Design
This paper presents an improved DETR detector that maintains a "plain" nature: using a single-scale feature map and global cross-attention calculations without specific locality constraints, in contrast to previous leading DETR-based detectors that reintroduce architectural inductive biases of multi-scale and locality into the decoder.
Cross-Silo Prototypical Calibration for Federated Learning with Non-IID Data
To address this issue, this paper presents a cross-silo prototypical calibration method (FedCSPC), which takes additional prototype information from the clients to learn a unified feature space on the server side.
V-DETR: DETR with Vertex Relative Position Encoding for 3D Object Detection
We introduce a highly performant 3D object detector for point clouds using the DETR framework.
Ranked #2 on
3D Object Detection
on ScanNetV2
Rethinking the Localization in Weakly Supervised Object Localization
Weakly supervised object localization (WSOL) is one of the most popular and challenging tasks in computer vision.
Over-the-Air Computation Aided Federated Learning with the Aggregation of Normalized Gradient
In case of smooth and strongly convex loss function, we prove our proposed method can achieve minimal training loss at linear rate with any small positive tolerance.
Joint Power Control and Data Size Selection for Over-the-Air Computation Aided Federated Learning
For parameter aggregating in FL, over-the-air computation is a spectrum-efficient solution, which allows all mobile devices to transmit their parameter-mapped signals concurrently to a BS.
SimDA: Simple Diffusion Adapter for Efficient Video Generation
In this work, we propose a Simple Diffusion Adapter (SimDA) that fine-tunes only 24M out of 1. 1B parameters of a strong T2I model, adapting it to video generation in a parameter-efficient way.
Federated Learning Robust to Byzantine Attacks: Achieving Zero Optimality Gap
In this paper, we propose a robust aggregation method for federated learning (FL) that can effectively tackle malicious Byzantine attacks.
PartSeg: Few-shot Part Segmentation via Part-aware Prompt Learning
In this work, we address the task of few-shot part segmentation, which aims to segment the different parts of an unseen object using very few labeled examples.
Mask-Attention-Free Transformer for 3D Instance Segmentation
Therefore, we abandon the mask attention design and resort to an auxiliary center regression task instead.
InstructDiffusion: A Generalist Modeling Interface for Vision Tasks
We present InstructDiffusion, a unifying and generic framework for aligning computer vision tasks with human instructions.
DeViT: Decomposing Vision Transformers for Collaborative Inference in Edge Devices
Our objective is to achieve fast and energy-efficient collaborative inference while maintaining comparable accuracy compared with large ViTs.
Exploring Non-additive Randomness on ViT against Query-Based Black-Box Attacks
In this work, we first taxonomize the stochastic defense strategies against QBBA.
TinyCLIP: CLIP Distillation via Affinity Mimicking and Weight Inheritance
In this paper, we propose a novel cross-modal distillation method, called TinyCLIP, for large-scale language-image pre-trained models.
Deep Model Fusion: A Survey
Specifically, we categorize existing deep model fusion methods as four-fold: (1) "Mode connectivity", which connects the solutions in weight space via a path of non-increasing loss, in order to obtain better initialization for model fusion; (2) "Alignment" matches units between neural networks to create better conditions for fusion; (3) "Weight average", a classical model fusion method, averages the weights of multiple models to obtain more accurate results closer to the optimal solution; (4) "Ensemble learning" combines the outputs of diverse models, which is a foundational technique for improving the accuracy and robustness of the final model.
Parameter Efficient Multi-task Model Fusion with Partial Linearization
We demonstrate that our partial linearization technique enables a more effective fusion of multiple tasks into a single model, outperforming standard adapter tuning and task arithmetic alone.
Learn From Model Beyond Fine-Tuning: A Survey
LFM focuses on the research, modification, and design of FM based on the model interface, so as to better understand the model structure and weights (in a black box environment), and to generalize the model to downstream tasks.
A Survey on Video Diffusion Models
However, existing surveys mainly focus on diffusion models in the context of image generation, with few up-to-date reviews on their application in the video domain.
FP8-LM: Training FP8 Large Language Models
In this paper, we explore FP8 low-bit data formats for efficient training of large language models (LLMs).
One-for-All: Bridge the Gap Between Heterogeneous Architectures in Knowledge Distillation
To tackle the challenge in distilling heterogeneous models, we propose a simple yet effective one-for-all KD framework called OFA-KD, which significantly improves the distillation performance between heterogeneous architectures.
Multiple View Geometry Transformers for 3D Human Pose Estimation
In this work, we aim to improve the 3D reasoning ability of Transformers in multi-view 3D human pose estimation.
GAIA: Zero-shot Talking Avatar Generation
Zero-shot talking avatar generation aims at synthesizing natural talking videos from speech and a single portrait image.
Viewport Prediction for Volumetric Video Streaming by Exploring Video Saliency and Trajectory Information
To our knowledge, this is the first comprehensive study of viewport prediction in volumetric video streaming.
MotionEditor: Editing Video Motion via Content-Aware Diffusion
This mechanism enables the editing branch to query the key and value from the reconstruction branch in a decoupled manner, making the editing branch retain the original background and protagonist appearance.
VIDiff: Translating Videos via Multi-Modal Instructions with Diffusion Models
Our model can edit and translate the desired results within seconds based on user instructions.
Segment and Caption Anything
We propose a method to efficiently equip the Segment Anything Model (SAM) with the ability to generate regional captions.
Concrete Subspace Learning based Interference Elimination for Multi-task Model Fusion
At the upper level, we focus on learning a shared Concrete mask to identify the subspace, while at the inner level, model merging is performed to maximize the performance of the merged model.
Dynamic Routing for Integrated Satellite-Terrestrial Networks: A Constrained Multi-Agent Reinforcement Learning Approach
To address these challenges, we study packet routing with ground stations and satellites working jointly to transmit packets, while prioritizing fast communication and meeting energy efficiency and packet loss requirements.
WisdoM: Improving Multimodal Sentiment Analysis by Fusing Contextual World Knowledge
Sentiment analysis is rapidly advancing by utilizing various data modalities (e. g., text, image).
Data-efficient Large Vision Models through Sequential Autoregression
Training general-purpose vision models on purely sequential visual data, eschewing linguistic inputs, has heralded a new frontier in visual understanding.
StarCoder 2 and The Stack v2: The Next Generation
Our large model, StarCoder2- 15B, significantly outperforms other models of comparable size.
Ranked #25 on
Code Generation
on MBPP
Common 7B Language Models Already Possess Strong Math Capabilities
This paper shows that the LLaMA-2 7B model with common pre-training already exhibits strong mathematical abilities, as evidenced by its impressive accuracy of 97. 7% and 72. 0% on the GSM8K and MATH benchmarks, respectively, when selecting the best response from 256 random generations.
Byzantine-resilient Federated Learning With Adaptivity to Data Heterogeneity
This paper deals with federated learning (FL) in the presence of malicious Byzantine attacks and data heterogeneity.
Federated Learning with Only Positive Labels by Exploring Label Correlations
Specifically, FedALC estimates the label correlations in the class embedding learning for different label pairs and utilizes it to improve the model training.
小样本关系分类研究综述(Few-Shot Relation Classification: A Survey)
关系分类作为构建结构化知识的重要一环, 在自然语言处理领域备受关注。但在很多应用领域中(医疗、金融领域), 收集充足的用于训练关系分类模型的数据是十分困难的。近年来, 仅需要少量训练样本的小样本学习研究逐渐新兴于各大领域。本文对近期小样本关系分类模型与方法进行了系统的综述。根据度量方法的不同, 将现有方法分为原型式和分布式两大类。根据是否利用额外信息, 将模型分为预训练和非预训练两大类。此外, 除了常规设定下的小样本学习, 本文还梳理了跨领域和稀缺资源场景下的小样本学习, 并探讨了目前小样本关系分类方法的局限性, 分析了跨领域小样本 学习面临的技术挑战。最后, 展望了小样本关系分类未来的发展方向。
Scalable Differential Privacy with Certified Robustness in Adversarial Learning
In this paper, we aim to develop a scalable algorithm to preserve differential privacy (DP) in adversarial learning for deep neural networks (DNNs), with certified robustness to adversarial examples.
