Search Results for author:
Found 55 papers, 38 papers with code
Fast Autoregressive Models for Continuous Latent Generation
Autoregressive models have demonstrated remarkable success in sequential data generation, particularly in NLP, but their extension to continuous-domain image generation presents significant challenges.
EAGLE-3: Scaling up Inference Acceleration of Large Language Models via Training-Time Test
We identify that this limitation arises from EAGLE's feature prediction constraints.
Modelling Multi-modal Cross-interaction for ML-FSIC Based on Local Feature Selection
A key feature of the multi-label setting is that images often have several labels, which typically refer to objects appearing in different regions of the image.
CogACT: A Foundational Vision-Language-Action Model for Synergizing Cognition and Action in Robotic Manipulation
Unlike previous works that directly repurpose VLM for action prediction by simple action quantization, we propose a omponentized VLA architecture that has a specialized action module conditioned on VLM output.
Expressive and Generalizable Low-rank Adaptation for Large Models via Slow Cascaded Learning
Efficient fine-tuning plays a fundamental role in modern large models, with low-rank adaptation emerging as a particularly promising approach.
EAGLE-2: Faster Inference of Language Models with Dynamic Draft Trees
Inference with modern Large Language Models (LLMs) is expensive and time-consuming, and speculative sampling has proven to be an effective solution.
Revisiting Referring Expression Comprehension Evaluation in the Era of Large Multimodal Models
Recent advancements in REC have been driven by large multimodal models (LMMs) like CogVLM, which achieved 92. 44% accuracy on RefCOCO.
Scaling the Codebook Size of VQGAN to 100,000 with a Utilization Rate of 99%
We demonstrate the superior performance of our model over its counterparts across a variety of tasks, including image reconstruction, image classification, auto-regressive image generation using GPT, and image creation with diffusion- and flow-based generative models.
Ranked #12 on
Image Reconstruction
on ImageNet
A Hong Kong Sign Language Corpus Collected from Sign-interpreted TV News
The dataset is collected to enrich resources for HKSL and support research in large-vocabulary continuous sign language recognition (SLR) and translation (SLT).
Rethinking Generative Large Language Model Evaluation for Semantic Comprehension
Through a comprehensive evaluation of 24 models across 11 benchmarks, we highlight several potential drawbacks of MCQA, for instance, the inconsistency between the MCQA evaluation and the generation of open-ended responses in practical scenarios.
Beyond Text: Frozen Large Language Models in Visual Signal Comprehension
To achieve this, we present the Vision-to-Language Tokenizer, abbreviated as V2T Tokenizer, which transforms an image into a ``foreign language'' with the combined aid of an encoder-decoder, the LLM vocabulary, and a CLIP model.
AnyTool: Self-Reflective, Hierarchical Agents for Large-Scale API Calls
We also revisit the evaluation protocol introduced by previous works and identify a limitation in this protocol that leads to an artificially high pass rate.
EAGLE: Speculative Sampling Requires Rethinking Feature Uncertainty
In this paper, we reconsider speculative sampling and derive two key observations.
Towards Online Continuous Sign Language Recognition and Translation
Our approach consists of three phases: 1) developing a sign dictionary; 2) training an isolated sign language recognition model on the dictionary; and 3) employing a sliding window approach on the input sign sequence, feeding each sign clip to the optimized model for online recognition.
A Simple Baseline for Spoken Language to Sign Language Translation with 3D Avatars
The objective of this paper is to develop a functional system for translating spoken languages into sign languages, referred to as Spoken2Sign translation.
Hybrid Proposal Refiner: Revisiting DETR Series from the Faster R-CNN Perspective
We systematically adapt the Faster R-CNN towards the Deformable DETR by integrating or repurposing each component of Deformable DETR and note that Deformable DETR's improved performance over Faster R-CNN is attributed to the adoption of advanced modules such as a superior proposal refiner (e. g. deformable attention rather than RoI Align).
RAIN: Your Language Models Can Align Themselves without Finetuning
We discover that by integrating self-evaluation and rewind mechanisms, unaligned LLMs can directly produce responses consistent with human preferences via self-boosting.
Exploring Non-additive Randomness on ViT against Query-Based Black-Box Attacks
In this work, we first taxonomize the stochastic defense strategies against QBBA.
AniPortraitGAN: Animatable 3D Portrait Generation from 2D Image Collections
For the new task, we base our method on the generative radiance manifold representation and equip it with learnable facial and head-shoulder deformations.
Improving Continuous Sign Language Recognition with Cross-Lingual Signs
Experimentally, our approach achieves state-of-the-art performance on two widely-used CSLR datasets: Phoenix-2014 and Phoenix-2014T.
CiCo: Domain-Aware Sign Language Retrieval via Cross-Lingual Contrastive Learning
Our framework, termed as domain-aware sign language retrieval via Cross-lingual Contrastive learning or CiCo for short, outperforms the pioneering method by large margins on various datasets, e. g., +22. 4 T2V and +28. 0 V2T R@1 improvements on How2Sign dataset, and +13. 7 T2V and +17. 1 V2T R@1 improvements on PHOENIX-2014T dataset.
Ranked #1 on
Sign Language Retrieval
on CSL-Daily
Natural Language-Assisted Sign Language Recognition
Sign languages are visual languages which convey information by signers' handshape, facial expression, body movement, and so forth.
Ranked #2 on
Sign Language Recognition
on WLASL-2000
Two-shot Video Object Segmentation
The underlying idea is to generate pseudo labels for unlabeled frames during training and to optimize the model on the combination of labeled and pseudo-labeled data.
DeepMIM: Deep Supervision for Masked Image Modeling
Deep supervision, which involves extra supervisions to the intermediate features of a neural network, was widely used in image classification in the early deep learning era since it significantly reduces the training difficulty and eases the optimization like avoiding gradient vanish over the vanilla training.
Side Adapter Network for Open-Vocabulary Semantic Segmentation
A side network is attached to a frozen CLIP model with two branches: one for predicting mask proposals, and the other for predicting attention bias which is applied in the CLIP model to recognize the class of masks.
Ranked #5 on
Zero Shot Segmentation
on Segmentation in the Wild
TinyMIM: An Empirical Study of Distilling MIM Pre-trained Models
Our TinyMIM model of tiny size achieves 79. 6% top-1 accuracy on ImageNet-1K image classification, which sets a new record for small vision models of the same size and computation budget.
Iterative Proposal Refinement for Weakly-Supervised Video Grounding
Weakly-Supervised Video Grounding (WSVG) aims to localize events of interest in untrimmed videos with only video-level annotations.
Attentive Mask CLIP
To address this issue, we propose an attentive token removal approach for CLIP training, which retains tokens with a high semantic correlation to the text description.
Two-Stream Network for Sign Language Recognition and Translation
RGB videos, however, are raw signals with substantial visual redundancy, leading the encoder to overlook the key information for sign language understanding.
Ranked #1 on
Sign Language Translation
on CSL-Daily
AniFaceGAN: Animatable 3D-Aware Face Image Generation for Video Avatars
To achieve meaningful control over facial expressions via deformation, we propose a 3D-level imitative learning scheme between the generator and a parametric 3D face model during adversarial training of the 3D-aware GAN.
Conditional DETR V2: Efficient Detection Transformer with Box Queries
Inspired by Conditional DETR, an improved DETR with fast training convergence, that presented box queries (originally called spatial queries) for internal decoder layers, we reformulate the object query into the format of the box query that is a composition of the embeddings of the reference point and the transformation of the box with respect to the reference point.
Boosting Zero-shot Learning via Contrastive Optimization of Attribute Representations
In this paper, we propose a new framework to boost ZSL by explicitly learning attribute prototypes beyond images and contrastively optimizing them with attribute-level features within images.
Unsupervised Prompt Learning for Vision-Language Models
In this paper, we explore a different scenario, in which the labels of the target datasets are unprovided, and we present an unsupervised prompt learning (UPL) approach to avoid prompt engineering while simultaneously improving transfer performance of CLIP-like vision-language models.
Frame-wise Action Representations for Long Videos via Sequence Contrastive Learning
In this paper, we introduce a novel contrastive action representation learning (CARL) framework to learn frame-wise action representations, especially for long videos, in a self-supervised manner.
Learning to Prompt for Open-Vocabulary Object Detection with Vision-Language Model
In this paper, we introduce a novel method, detection prompt (DetPro), to learn continuous prompt representations for open-vocabulary object detection based on the pre-trained vision-language model.
A Simple Multi-Modality Transfer Learning Baseline for Sign Language Translation
Concretely, we pretrain the sign-to-gloss visual network on the general domain of human actions and the within-domain of a sign-to-gloss dataset, and pretrain the gloss-to-text translation network on the general domain of a multilingual corpus and the within-domain of a gloss-to-text corpus.
Ranked #3 on
Sign Language Translation
on CSL-Daily
A Simple Baseline for Open-Vocabulary Semantic Segmentation with Pre-trained Vision-language Model
However, semantic segmentation and the CLIP model perform on different visual granularity, that semantic segmentation processes on pixels while CLIP performs on images.
Cross-Model Pseudo-Labeling for Semi-Supervised Action Recognition
Typically in recent work, the pseudo-labels are obtained by training a model on the labeled data, and then using confident predictions from the model to teach itself.
Towards Tokenized Human Dynamics Representation
For human action understanding, a popular research direction is to analyze short video clips with unambiguous semantic content, such as jumping and drinking.
Bootstrap Your Object Detector via Mixed Training
We introduce MixTraining, a new training paradigm for object detection that can improve the performance of existing detectors for free.
Semi-Supervised Semantic Segmentation via Adaptive Equalization Learning
The confidence bank is leveraged as an indicator to tilt training towards under-performing categories, instantiated in three strategies: 1) adaptive Copy-Paste and CutMix data augmentation approaches which give more chance for under-performing categories to be copied or cut; 2) an adaptive data sampling approach to encourage pixels from under-performing category to be sampled; 3) a simple yet effective re-weighting method to alleviate the training noise raised by pseudo-labeling.
Self-supervised Discovery of Human Actons from Long Kinematic Videos
However, methods for understanding short semantic actions cannot be directly translated to long kinematic sequences such as dancing, where it becomes challenging even to semantically label the human movements.
Particle Based Stochastic Policy Optimization
Stochastic polic have been widely applied for their good property in exploration and uncertainty quantification.
Ranked #1 on
MuJoCo Games
on Ant-v3
ADNet: Leveraging Error-Bias Towards Normal Direction in Face Alignment
On the other hand, AAM is an attention module which can get anisotropic attention mask focusing on the region of point and its local edge connected by adjacent points, it has a stronger response in tangent than in normal, which means relaxed constraints in the tangent.
Ranked #5 on
Face Alignment
on 300W
Dual Path Learning for Domain Adaptation of Semantic Segmentation
In this paper, based on the observation that domain adaptation frameworks performed in the source and target domain are almost complementary in terms of image translation and SSL, we propose a novel dual path learning (DPL) framework to alleviate visual inconsistency.
End-to-End Semi-Supervised Object Detection with Soft Teacher
This paper presents an end-to-end semi-supervised object detection approach, in contrast to previous more complex multi-stage methods.
Ranked #6 on
Semi-Supervised Object Detection
on COCO 100% labeled data
(using extra training data)
Aligning Pretraining for Detection via Object-Level Contrastive Learning
Image-level contrastive representation learning has proven to be highly effective as a generic model for transfer learning.
High-Fidelity and Arbitrary Face Editing
However, we observe that the generator tends to find a tricky way to hide information from the original image to satisfy the constraint of cycle consistency, making it impossible to maintain the rich details (e. g., wrinkles and moles) of non-editing areas.
Global Context Networks
The Non-Local Network (NLNet) presents a pioneering approach for capturing long-range dependencies within an image, via aggregating query-specific global context to each query position.
Ranked #42 on
Instance Segmentation
on COCO test-dev
RelationNet++: Bridging Visual Representations for Object Detection via Transformer Decoder
The proposed module is named \emph{bridging visual representations} (BVR).
Ranked #66 on
Object Detection
on COCO test-dev
Restoring Negative Information in Few-Shot Object Detection
In this paper, we restore the negative information in few-shot object detection by introducing a new negative- and positive-representative based metric learning framework and a new inference scheme with negative and positive representatives.
Point-Set Anchors for Object Detection, Instance Segmentation and Pose Estimation
A recent approach for object detection and human pose estimation is to regress bounding boxes or human keypoints from a central point on the object or person.
Design and Interpretation of Universal Adversarial Patches in Face Detection
We consider universal adversarial patches for faces -- small visual elements whose addition to a face image reliably destroys the performance of face detectors.
GCNet: Non-local Networks Meet Squeeze-Excitation Networks and Beyond
In this paper, we take advantage of this finding to create a simplified network based on a query-independent formulation, which maintains the accuracy of NLNet but with significantly less computation.
Ranked #25 on
Object Detection
on COCO-O
WIDER Face and Pedestrian Challenge 2018: Methods and Results
This paper presents a review of the 2018 WIDER Challenge on Face and Pedestrian.
