Search Results for author:
Found 82 papers, 45 papers with code
UV-Attack: Physical-World Adversarial Attacks for Person Detection via Dynamic-NeRF-based UV Mapping
This work highlights the potential of dynamic NeRF-based UV mapping for creating more effective adversarial attacks on person detectors, addressing key challenges in modeling human movement and texture modification.
Mobile Augmented Reality Framework with Fusional Localization and Pose Estimation
This paper first conducts a comprehensive study of the state-of-the-art AR and localization systems on mobile platforms.
CustomTTT: Motion and Appearance Customized Video Generation via Test-Time Training
Besides, given some reference images or videos, the parameter-efficient fine-tuning method, i. e. LoRA, can generate high-quality customized concepts, e. g., the specific subject or the motions from a reference video.
Dynamic Ensemble Reasoning for LLM Experts
Ensemble reasoning for the strengths of different LLM experts is critical to achieving consistent and satisfactory performance on diverse inputs across a wide range of tasks.
Jointly RS Image Deblurring and Super-Resolution with Adjustable-Kernel and Multi-Domain Attention
In the deblurring branch, we design a pixel-adjustable kernel block (PAKB) to estimate the local and spatial-varying blur kernels.
Customize Segment Anything Model for Multi-Modal Semantic Segmentation with Mixture of LoRA Experts
The recent Segment Anything Model (SAM) represents a significant breakthrough in scaling segmentation models, delivering strong performance across various downstream applications in the RGB modality.
Florence-VL: Enhancing Vision-Language Models with Generative Vision Encoder and Depth-Breadth Fusion
We present Florence-VL, a new family of multimodal large language models (MLLMs) with enriched visual representations produced by Florence-2, a generative vision foundation model.
Improving Transferable Targeted Attacks with Feature Tuning Mixup
Recent clean feature mixup methods use random clean features to perturb the feature space but lack optimization for disrupting adversarial examples, overlooking the advantages of attack-specific perturbations.
Cross-Organ Domain Adaptive Neural Network for Pancreatic Endoscopic Ultrasound Image Segmentation
Accurate segmentation of lesions in pancreatic endoscopic ultrasound (EUS) images is crucial for effective diagnosis and treatment.
ISO: Overlap of Computation and Communication within Seqenence For LLM Inference
In the realm of Large Language Model (LLM) inference, the inherent structure of transformer models coupled with the multi-GPU tensor parallelism strategy leads to a sequential execution of computation and communication.
Boosting Lossless Speculative Decoding via Feature Sampling and Partial Alignment Distillation
In this paper, we reassess these approaches and propose FSPAD (Feature Sampling and Partial Alignment Distillation for Lossless Speculative Decoding), which introduces two straightforward and effective components within the existing framework to boost lossless speculative decoding.
Transferable 3D Adversarial Shape Completion using Diffusion Models
Extensive experiments demonstrate that our proposed attacks outperform state-of-the-art adversarial attack methods against both black-box models and defenses.
Efficient Prompting for LLM-based Generative Internet of Things
Large language models (LLMs) have demonstrated remarkable capacities on various tasks, and integrating the capacities of LLMs into the Internet of Things (IoT) applications has drawn much research attention recently.
Ranked #6 on
Table-based Fact Verification
on TabFact
ZeroPur: Succinct Training-Free Adversarial Purification
In this work, we suppose that adversarial images are outliers of the natural image manifold and the purification process can be considered as returning them to this manifold.
Score-based Generative Priors Guided Model-driven Network for MRI Reconstruction
However, the hyperparameters in the sampling process require subtle tuning, otherwise the results can be severely corrupted by hallucination artifacts, especially with out-of-distribution test data.
Clover: Regressive Lightweight Speculative Decoding with Sequential Knowledge
Consequently, the GPU spends most of its time on memory transfer instead of computation.
Phi-3 Technical Report: A Highly Capable Language Model Locally on Your Phone
We introduce phi-3-mini, a 3. 8 billion parameter language model trained on 3. 3 trillion tokens, whose overall performance, as measured by both academic benchmarks and internal testing, rivals that of models such as Mixtral 8x7B and GPT-3. 5 (e. g., phi-3-mini achieves 69% on MMLU and 8. 38 on MT-bench), despite being small enough to be deployed on a phone.
Ranked #5 on
MMR total
on MRR-Benchmark
(using extra training data)
Efficient Modulation for Vision Networks
We demonstrate that the modulation mechanism is particularly well suited for efficient networks and further tailor the modulation design by proposing the efficient modulation (EfficientMod) block, which is considered the essential building block for our networks.
Real-Time Image Segmentation via Hybrid Convolutional-Transformer Architecture Search
Manually replacing convolution layers with multi-head self-attention is non-trivial due to the costly overhead in memory to maintain high resolution.
Rethinking Detection Based Table Structure Recognition for Visually Rich Document Images
However, existing detection-based models usually cannot perform as well as other types of solutions regarding cell-level TSR metrics, such as TEDS, and the underlying reasons limiting the performance of these models on the TSR task are also not well-explored.
Florence-2: Advancing a Unified Representation for a Variety of Vision Tasks
We introduce Florence-2, a novel vision foundation model with a unified, prompt-based representation for a variety of computer vision and vision-language tasks.
Ranked #1 on
Visual Grounding
on RefCOCO+ val
(using extra training data)
Detecting Generated Images by Real Images Only
This paper approaches the generated image detection problem from a new perspective: Start from real images.
VFedMH: Vertical Federated Learning for Training Multiple Heterogeneous Models
Then the passive party, who owns only features of the sample, injects the blinding factor into the local embedding and sends it to the active party.
A Survey of Robustness and Safety of 2D and 3D Deep Learning Models Against Adversarial Attacks
Lots of papers have emerged to investigate the robustness and safety of deep learning models against adversarial attacks.
TinyCLIP: CLIP Distillation via Affinity Mimicking and Weight Inheritance
In this paper, we propose a novel cross-modal distillation method, called TinyCLIP, for large-scale language-image pre-trained models.
Baichuan 2: Open Large-scale Language Models
Large language models (LLMs) have demonstrated remarkable performance on a variety of natural language tasks based on just a few examples of natural language instructions, reducing the need for extensive feature engineering.
Generating Transferable and Stealthy Adversarial Patch via Attention-guided Adversarial Inpainting
In the first stage, we extract multi-scale style embeddings by a pyramid-like network and identity embeddings by a pretrained FR model and propose a novel Attention-guided Adaptive Instance Normalization layer (AAIN) to merge them via background-patch cross-attention maps.
AdvDiff: Generating Unrestricted Adversarial Examples using Diffusion Models
Unrestricted adversarial attacks present a serious threat to deep learning models and adversarial defense techniques.
Fast Continual Multi-View Clustering with Incomplete Views
Such an incomplete continual data problem (ICDP) in MVC is tough to solve since incomplete information with continual data increases the difficulty of extracting consistent and complementary knowledge among views.
Table Detection for Visually Rich Document Images
Table Detection (TD) is a fundamental task to enable visually rich document understanding, which requires the model to extract information without information loss.
i-Code V2: An Autoregressive Generation Framework over Vision, Language, and Speech Data
The convergence of text, visual, and audio data is a key step towards human-like artificial intelligence, however the current Vision-Language-Speech landscape is dominated by encoder-only models which lack generative abilities.
Revisiting Table Detection Datasets for Visually Rich Documents
Moreover, to enrich the data sources, we propose a new ICT-TD dataset using the PDF files of Information and Communication Technologies (ICT) commodities, a different domain containing unique samples that hardly appear in open datasets.
StyLess: Boosting the Transferability of Adversarial Examples
Our method can prevent adversarial examples from using non-robust style features and help generate transferable perturbations.
SARF: Aliasing Relation Assisted Self-Supervised Learning for Few-shot Relation Reasoning
Moreover, most of the existing methods ignore leveraging the beneficial information from aliasing relations (AR), i. e., data-rich relations with similar contextual semantics to the target data-poor relation.
DAA: A Delta Age AdaIN operation for age estimation via binary code transformer
Inspired by the transfer learning, we designed the Delta Age AdaIN (DAA) operation to obtain the feature difference with each age, which obtains the style map of each age through the learned values representing the mean and standard deviation.
Revisiting Initializing Then Refining: An Incomplete and Missing Graph Imputation Network
With the development of various applications, such as social networks and knowledge graphs, graph data has been ubiquitous in the real world.
DLBD: A Self-Supervised Direct-Learned Binary Descriptor
Since their binarization processes are not a component of the network, the learning-based binary descriptor cannot fully utilize the advances of deep learning.
MCF: Mutual Correction Framework for Semi-Supervised Medical Image Segmentation
Inspired by the plain contrast idea, MCF introduces two different subnets to explore and utilize the discrepancies between subnets to correct cognitive bias of the model.
ARISE: Graph Anomaly Detection on Attributed Networks via Substructure Awareness
The average node-pair similarity can be regarded as the topology anomaly degree of nodes within substructures.
Efficient Information Sharing in ICT Supply Chain Social Network via Table Structure Recognition
Table Structure Recognition (TSR) aims to represent tables with complex structures in a machine-interpretable format so that the tabular data can be processed automatically.
Semantic Cross Attention for Few-shot Learning
Our proposed model uses word-embedding representations as semantic features to help train the embedding network and a semantic cross-attention module to bridge the semantic features into the typical visual modal.
Handling big tabular data of ICT supply chains: a multi-task, machine-interpretable approach
To transform the tabular data in electronic documents into a machine-interpretable format and provide layout and semantic information for information extraction and interpretation, we define a Table Structure Recognition (TSR) task and a Table Cell Type Classification (CTC) task.
Learning Visual Representation from Modality-Shared Contrastive Language-Image Pre-training
Large-scale multi-modal contrastive pre-training has demonstrated great utility to learn transferable features for a range of downstream tasks by mapping multiple modalities into a shared embedding space.
TinyViT: Fast Pretraining Distillation for Small Vision Transformers
It achieves a top-1 accuracy of 84. 8% on ImageNet-1k with only 21M parameters, being comparable to Swin-B pretrained on ImageNet-21k while using 4. 2 times fewer parameters.
Ranked #145 on
Image Classification
on ImageNet
Image Synthesis with Disentangled Attributes for Chest X-Ray Nodule Augmentation and Detection
Moreover, we propose to synthesize nodule CXR images by controlling the disentangled nodule attributes for data augmentation, in order to better compensate for the nodules that are easily missed in the detection task.
Physical-World Optical Adversarial Attacks on 3D Face Recognition
2D face recognition has been proven insecure for physical adversarial attacks.
i-Code: An Integrative and Composable Multimodal Learning Framework
Human intelligence is multimodal; we integrate visual, linguistic, and acoustic signals to maintain a holistic worldview.
Multimodal Adaptive Distillation for Leveraging Unimodal Encoders for Vision-Language Tasks
Experiments demonstrate that MAD leads to consistent gains in the low-shot, domain-shifted, and fully-supervised conditions on VCR, SNLI-VE, and VQA, achieving SOTA performance on VCR compared to other single models pretrained with image-text data.
Ranked #4 on
Visual Question Answering (VQA)
on VCR (Q-A) test
MiniViT: Compressing Vision Transformers with Weight Multiplexing
The central idea of MiniViT is to multiplex the weights of consecutive transformer blocks.
Ranked #227 on
Image Classification
on ImageNet
Unified Contrastive Learning in Image-Text-Label Space
Particularly, it attains gains up to 9. 2% and 14. 5% in average on zero-shot recognition benchmarks over the language-image contrastive learning and supervised learning methods, respectively.
DaViT: Dual Attention Vision Transformers
We show that these two self-attentions complement each other: (i) since each channel token contains an abstract representation of the entire image, the channel attention naturally captures global interactions and representations by taking all spatial positions into account when computing attention scores between channels; (ii) the spatial attention refines the local representations by performing fine-grained interactions across spatial locations, which in turn helps the global information modeling in channel attention.
Ranked #1 on
Medical Image Classification
on ImageNet
Table Structure Recognition with Conditional Attention
Table Structure Recognition (TSR) problem aims to recognize the structure of a table and transform the unstructured tables into a structured and machine-readable format so that the tabular data can be further analysed by the down-stream tasks, such as semantic modeling and information retrieval.
CLIP-TD: CLIP Targeted Distillation for Vision-Language Tasks
Experiments demonstrate that our proposed CLIP-TD leads to exceptional gains in the low-shot (up to 51. 9%) and domain-shifted (up to 71. 3%) conditions of VCR, while simultaneously improving performance under standard fully-supervised conditions (up to 2%), achieving state-of-art performance on VCR compared to other single models that are pretrained with image-text data only.
Focal Attention for Long-Range Interactions in Vision Transformers
With focal attention, we propose a new variant of Vision Transformer models, called Focal Transformers, which achieve superior performance over the state-of-the-art (SoTA) Vision Transformers on a range of public image classification and object detection benchmarks.
Florence: A New Foundation Model for Computer Vision
Computer vision foundation models, which are trained on diverse, large-scale dataset and can be adapted to a wide range of downstream tasks, are critical for this mission to solve real-world computer vision applications.
Ranked #1 on
Action Recognition In Videos
on Kinetics-600
Generating Unrestricted 3D Adversarial Point Clouds
The unrestricted adversarial attack loss is incorporated in the special adversarial training of GAN, which enables the generator to generate the adversarial examples to spoof the target network.
MA-CLIP: Towards Modality-Agnostic Contrastive Language-Image Pre-training
Large-scale multimodal contrastive pretraining has demonstrated great utility to support high performance in a range of downstream tasks by mapping multiple modalities into a shared embedding space.
Focal Self-attention for Local-Global Interactions in Vision Transformers
With focal self-attention, we propose a new variant of Vision Transformer models, called Focal Transformer, which achieves superior performance over the state-of-the-art vision Transformers on a range of public image classification and object detection benchmarks.
Ranked #19 on
Instance Segmentation
on COCO test-dev
Long-term Cross Adversarial Training: A Robust Meta-learning Method for Few-shot Classification Tasks
Although adversarial training (AT) methods such as Adversarial Query (AQ) can improve the adversarially robust performance of meta-learning models, AT is still computationally expensive training.
Efficient Self-supervised Vision Transformers for Representation Learning
This paper investigates two techniques for developing efficient self-supervised vision transformers (EsViT) for visual representation learning.
Ranked #17 on
Self-Supervised Image Classification
on ImageNet
Representation Learning
Self-Supervised Image Classification
Dynamic Head: Unifying Object Detection Heads with Attentions
In this paper, we present a novel dynamic head framework to unify object detection heads with attentions.
Ranked #7 on
Object Detection
on COCO 2017 val
(AP75 metric)
Lite-HRNet: A Lightweight High-Resolution Network
We introduce a lightweight unit, conditional channel weighting, to replace costly pointwise (1x1) convolutions in shuffle blocks.
Ranked #33 on
Pose Estimation
on COCO test-dev
(using extra training data)
Bottom-Up Human Pose Estimation Via Disentangled Keypoint Regression
Our motivation is that regressing keypoint positions accurately needs to learn representations that focus on the keypoint regions.
Multi-Scale Vision Longformer: A New Vision Transformer for High-Resolution Image Encoding
This paper presents a new Vision Transformer (ViT) architecture Multi-Scale Vision Longformer, which significantly enhances the ViT of \cite{dosovitskiy2020image} for encoding high-resolution images using two techniques.
Ranked #47 on
Instance Segmentation
on COCO minival
CvT: Introducing Convolutions to Vision Transformers
We present in this paper a new architecture, named Convolutional vision Transformer (CvT), that improves Vision Transformer (ViT) in performance and efficiency by introducing convolutions into ViT to yield the best of both designs.
Ranked #2 on
Image Classification
on Flowers-102
(using extra training data)
Reality Transform Adversarial Generators for Image Splicing Forgery Detection and Localization
For detecting the tampered regions, a forgery localization generator GM is proposed based on a multi-decoder-single-task strategy.
DTMNet: A Discrete Tchebichef Moments-Based Deep Neural Network for Multi-Focus Image Fusion
The proposed DTMNet is an end-to-end deep neural network with only one convolutional layer and three fully connected layers.
Consistent Instance Classification for Unsupervised Representation Learning
The proposed consistent instance classification (ConIC) approach simultaneously optimizes the classification loss and an additional consistency loss explicitly penalizing the feature dissimilarity between the augmented views from the same instance.
Color-related Local Binary Pattern: A Learned Local Descriptor for Color Image Recognition
In this paper, a color-related local binary pattern (cLBP) which learns the dominant patterns from the decoded LBP is proposed for color images recognition.
D-Unet: A Dual-encoder U-Net for Image Splicing Forgery Detection and Localization
The unfixed encoder autonomously learns the image fingerprints that differentiate between the tampered and non-tampered regions, whereas the fixed encoder intentionally provides the direction information that assists the learning and detection of the network.
Proxy Network for Few Shot Learning
We conclude that the success of metric-learning based approaches lies in the data embedding, the representative of each class, and the distance metric.
Bottom-Up Human Pose Estimation by Ranking Heatmap-Guided Adaptive Keypoint Estimates
The typical bottom-up human pose estimation framework includes two stages, keypoint detection and grouping.
3D Human Pose Estimation via Explicit Compositional Depth Maps
n this work, we tackle the problem of estimating 3D human pose in camera space from a monocular image.
Ranked #22 on
3D Human Pose Estimation
on MPI-INF-3DHP
(PCK metric)
RRU-Net: The Ringed Residual U-Net for Image Splicing Forgery Detection
The core idea of the RRU-Net is to strengthen the learning way of CNN, which is inspired by the recall and the consolidation mechanism of the human brain and implemented by the propagation and the feedback process of the residual in CNN.
HigherHRNet: Scale-Aware Representation Learning for Bottom-Up Human Pose Estimation
HigherHRNet even surpasses all top-down methods on CrowdPose test (67. 6% AP), suggesting its robustness in crowded scene.
Ranked #2 on
Pose Estimation
on UAV-Human
Deep High-Resolution Representation Learning for Visual Recognition
High-resolution representations are essential for position-sensitive vision problems, such as human pose estimation, semantic segmentation, and object detection.
Ranked #1 on
Object Detection
on COCO test-dev
(Hardware Burden metric, using extra
training data)
High-Resolution Representations for Labeling Pixels and Regions
The proposed approach achieves superior results to existing single-model networks on COCO object detection.
Ranked #7 on
Semantic Segmentation
on LIP val
Deep High-Resolution Representation Learning for Human Pose Estimation
We start from a high-resolution subnetwork as the first stage, gradually add high-to-low resolution subnetworks one by one to form more stages, and connect the mutli-resolution subnetworks in parallel.
Ranked #1 on
3D Pose Estimation
on HARPER
Simple Baselines for Human Pose Estimation and Tracking
There has been significant progress on pose estimation and increasing interests on pose tracking in recent years.
Ranked #2 on
2D Human Pose Estimation
on JHMDB (2D poses only)
Integral Human Pose Regression
State-of-the-art human pose estimation methods are based on heat map representation.
Ranked #23 on
Pose Estimation
on MPII Human Pose
Interleaved Group Convolutions
The main point lies in a novel building block, a pair of two successive interleaved group convolutions: primary group convolution and secondary group convolution.
Interleaved Group Convolutions for Deep Neural Networks
The main point lies in a novel building block, a pair of two successive interleaved group convolutions: primary group convolution and secondary group convolution.
