Search Results for author:
Found 304 papers, 170 papers with code
FANG-COVID: A New Large-Scale Benchmark Dataset for Fake News Detection in German
As the world continues to fight the COVID-19 pandemic, it is simultaneously fighting an ‘infodemic’ – a flood of disinformation and spread of conspiracy theories leading to health threats and the division of society.
Automated Classification of Written Proficiency Levels on the CEFR-Scale through Complexity Contours and RNNs
The key to the present paper is the combined use of what we refer to as ‘complexity contours’, a series of measurements of indices of L2 proficiency obtained by a computational tool that implements a sliding window technique, and recurrent neural network (RNN) classifiers that adequately capture the sequential information in those contours.
The Best of Both Worlds: Combining Engineered Features with Transformers for Improved Mental Health Prediction from Reddit Posts
In recent years, there has been increasing interest in the application of natural language processing and machine learning techniques to the detection of mental health conditions (MHC) based on social media data.
MANTIS at SMM4H’2022: Pre-Trained Language Models Meet a Suite of Psycholinguistic Features for the Detection of Self-Reported Chronic Stress
This paper describes our submission to Social Media Mining for Health (SMM4H) 2022 Shared Task 8, aimed at detecting self-reported chronic stress on Twitter.
SPADE: A Big Five-Mturk Dataset of Argumentative Speech Enriched with Socio-Demographics for Personality Detection
In recent years, there has been increasing interest in automatic personality detection based on language.
Language that Captivates the Audience: Predicting Affective Ratings of TED Talks in a Multi-Label Classification Task
The aim of the paper is twofold: (1) to automatically predict the ratings assigned by viewers to 14 categories available for TED talks in a multi-label classification task and (2) to determine what types of features drive classification accuracy for each of the categories.
A Language-Based Approach to Fake News Detection Through Interpretable Features and BRNN
We demonstrate that our approach is promising as it achieves similar results on these two datasets as the best performing black box models reported in the literature.
RBF-Softmax: Learning Deep Representative Prototypes with Radial Basis Function Softmax
To address this problem, this paper introduces a novel Radial Basis Function (RBF) distances to replace the commonly used inner products in the softmax loss function, such that it can adaptively assign losses to regularize the intra-class and inter-class distances by reshaping the relative differences, and thus creating more representative prototypes of classes to improve optimization.
Mining Inter-Video Proposal Relations for Video Object Detection
Recent studies have shown that, context aggregating information from proposals in different frames can clearly enhance the performance of video object detection.
Ranked #6 on
Video Object Detection
on ImageNet VID
DiLu: A Knowledge-Driven Approach to Autonomous Driving with Large Language Models
Recent advancements in autonomous driving have relied on data-driven approaches, which are widely adopted but face challenges including dataset bias, overfitting, and uninterpretability.
InternLM-XComposer: A Vision-Language Large Model for Advanced Text-image Comprehension and Composition
We propose InternLM-XComposer, a vision-language large model that enables advanced image-text comprehension and composition.
LAVIE: High-Quality Video Generation with Cascaded Latent Diffusion Models
To this end, we propose LaVie, an integrated video generation framework that operates on cascaded video latent diffusion models, comprising a base T2V model, a temporal interpolation model, and a video super-resolution model.
StructChart: Perception, Structuring, Reasoning for Visual Chart Understanding
Charts are common in literature across different scientific fields, conveying rich information easily accessible to readers.
SPOT: Scalable 3D Pre-training via Occupancy Prediction for Autonomous Driving
Our contributions are threefold: (1) Occupancy prediction is shown to be promising for learning general representations, which is demonstrated by extensive experiments on plenty of datasets and tasks.
HAT: Hybrid Attention Transformer for Image Restoration
In the training stage, we additionally adopt a same-task pre-training strategy to further exploit the potential of the model for further improvement.
ReSimAD: Zero-Shot 3D Domain Transfer for Autonomous Driving with Source Reconstruction and Target Simulation
Domain shifts such as sensor type changes and geographical situation variations are prevalent in Autonomous Driving (AD), which poses a challenge since AD model relying on the previous-domain knowledge can be hardly directly deployed to a new domain without additional costs.
UniSeg: A Unified Multi-Modal LiDAR Segmentation Network and the OpenPCSeg Codebase
Besides, we construct the OpenPCSeg codebase, which is the largest and most comprehensive outdoor LiDAR segmentation codebase.
Towards Efficient SDRTV-to-HDRTV by Learning from Image Formation
However, the majority of available resources are still in standard dynamic range (SDR).
A-Eval: A Benchmark for Cross-Dataset Evaluation of Abdominal Multi-Organ Segmentation
To address these questions, we introduce A-Eval, a benchmark for the cross-dataset Evaluation ('Eval') of Abdominal ('A') multi-organ segmentation.
ImageBind-LLM: Multi-modality Instruction Tuning
During training, we adopt a learnable bind network to align the embedding space between LLaMA and ImageBind's image encoder.
SEAL: A Framework for Systematic Evaluation of Real-World Super-Resolution
Next, we propose a coarse-to-fine evaluation protocol to measure the distributed and relative performance of real-SR methods on the test set.
SAM-Med2D
To bridge this gap, we introduce SAM-Med2D, the most comprehensive studies on applying SAM to medical 2D images.
DiffBIR: Towards Blind Image Restoration with Generative Diffusion Prior
We present DiffBIR, which leverages pretrained text-to-image diffusion models for blind image restoration problem.
Ranked #1 on
Blind Face Restoration
on LFW
OmniQuant: Omnidirectionally Calibrated Quantization for Large Language Models
To tackle this issue, we introduce an Omnidirectionally calibrated Quantization (OmniQuant) technique for LLMs, which achieves good performance in diverse quantization settings while maintaining the computational efficiency of PTQ by efficiently optimizing various quantization parameters.
MGMAE: Motion Guided Masking for Video Masked Autoencoding
Based on this masking volume, we can track the unmasked tokens in time and sample a set of temporal consistent cubes from videos.
Shrinking Class Space for Enhanced Certainty in Semi-Supervised Learning
To mitigate potentially incorrect pseudo labels, recent frameworks mostly set a fixed confidence threshold to discard uncertain samples.
Foundation Model is Efficient Multimodal Multitask Model Selector
This paper investigates an under-explored but important problem: given a collection of pre-trained neural networks, predicting their performance on each multi-modal task without fine-tuning them, such as image recognition, referring, captioning, visual question answering, and text question answering.
Tiny LVLM-eHub: Early Multimodal Experiments with Bard
Secondly, it conducts an in-depth analysis of LVLMs' predictions using the ChatGPT Ensemble Evaluation (CEE), which leads to a robust and accurate evaluation and exhibits improved alignment with human evaluation compared to the word matching approach.
The All-Seeing Project: Towards Panoptic Visual Recognition and Understanding of the Open World
We present the All-Seeing (AS) project: a large-scale data and model for recognizing and understanding everything in the open world.
Scaling Data Generation in Vision-and-Language Navigation
Recent research in language-guided visual navigation has demonstrated a significant demand for the diversity of traversable environments and the quantity of supervision for training generalizable agents.
Scaling TransNormer to 175 Billion Parameters
TransNormerLLM evolves from the previous linear attention architecture TransNormer by making advanced modifications that include positional embedding, linear attention acceleration, gating mechanism, tensor normalization, inference acceleration and stabilization.
Text-guided Foundation Model Adaptation for Pathological Image Classification
In this paper, we propose to Connect Image and Text Embeddings (CITE) to enhance pathological image classification.
FedMEKT: Distillation-based Embedding Knowledge Transfer for Multimodal Federated Learning
Bringing this concept into a system, we develop a distillation-based multimodal embedding knowledge transfer mechanism, namely FedMEKT, which allows the server and clients to exchange the joint knowledge of their learning models extracted from a small multimodal proxy dataset.
Meta-Transformer: A Unified Framework for Multimodal Learning
Multimodal learning aims to build models that can process and relate information from multiple modalities.
Boosting Federated Learning Convergence with Prototype Regularization
As a distributed machine learning technique, federated learning (FL) requires clients to collaboratively train a shared model with an edge server without leaking their local data.
Drive Like a Human: Rethinking Autonomous Driving with Large Language Models
In this paper, we explore the potential of using a large language model (LLM) to understand the driving environment in a human-like manner and analyze its ability to reason, interpret, and memorize when facing complex scenarios.
LimSim: A Long-term Interactive Multi-scenario Traffic Simulator
With the growing popularity of digital twin and autonomous driving in transportation, the demand for simulation systems capable of generating high-fidelity and reliable scenarios is increasing.
InternVid: A Large-scale Video-Text Dataset for Multimodal Understanding and Generation
Specifically, we utilize a multi-scale approach to generate video-related descriptions.
AnimateDiff: Animate Your Personalized Text-to-Image Diffusion Models without Specific Tuning
With the advance of text-to-image models (e. g., Stable Diffusion) and corresponding personalization techniques such as DreamBooth and LoRA, everyone can manifest their imagination into high-quality images at an affordable cost.
JourneyDB: A Benchmark for Generative Image Understanding
We further design 4 benchmarks to quantify the performance of generated image understanding in terms of both content and style interpretation.
Faster Segment Anything: Towards Lightweight SAM for Mobile Applications
Concretely, we distill the knowledge from the heavy image encoder (ViT-H in the original SAM) to a lightweight image encoder, which can be automatically compatible with the mask decoder in the original SAM.
Align, Adapt and Inject: Sound-guided Unified Image Generation
Then, we propose the audio adapter to adapt audio representation into an audio token enriched with specific semantics, which can be injected into a frozen T2I model flexibly.
MedFMC: A Real-world Dataset and Benchmark For Foundation Model Adaptation in Medical Image Classification
Foundation models, often pre-trained with large-scale data, have achieved paramount success in jump-starting various vision and language applications.
LVLM-eHub: A Comprehensive Evaluation Benchmark for Large Vision-Language Models
Large Vision-Language Models (LVLMs) have recently played a dominant role in multimodal vision-language learning.
Retrieving-to-Answer: Zero-Shot Video Question Answering with Frozen Large Language Models
Video Question Answering (VideoQA) has been significantly advanced from the scaling of recent Large Language Models (LLMs).
Ranked #2 on
Temporal/Casual QA
on NExT-QA
(using extra training data)
Robustness of SAM: Segment Anything Under Corruptions and Beyond
Following by interpreting the effects of synthetic corruption as style changes, we proceed to conduct a comprehensive evaluation for its robustness against 15 types of common corruption.
DetZero: Rethinking Offboard 3D Object Detection with Long-term Sequential Point Clouds
Extensive experiments on Waymo Open Dataset show our DetZero outperforms all state-of-the-art onboard and offboard 3D detection methods.
Denoising Diffusion Semantic Segmentation with Mask Prior Modeling
In this paper, we propose to ameliorate the semantic segmentation quality of existing discriminative approaches with a mask prior modeled by a recently-developed denoising diffusion generative model.
AD-PT: Autonomous Driving Pre-Training with Large-scale Point Cloud Dataset
It is a long-term vision for Autonomous Driving (AD) community that the perception models can learn from a large-scale point cloud dataset, to obtain unified representations that can achieve promising results on different tasks or benchmarks.
DiffRoom: Diffusion-based High-Quality 3D Room Reconstruction and Generation with Occupancy Prior
We present DiffRoom, a novel framework for tackling the problem of high-quality 3D indoor room reconstruction and generation, both of which are challenging due to the complexity and diversity of the room geometry.
DiffRate : Differentiable Compression Rate for Efficient Vision Transformers
Token compression aims to speed up large-scale vision transformers (e. g. ViTs) by pruning (dropping) or merging tokens.
Ghost in the Minecraft: Generally Capable Agents for Open-World Environments via Large Language Models with Text-based Knowledge and Memory
These agents, equipped with the logic and common sense capabilities of LLMs, can skillfully navigate complex, sparse-reward environments with text-based interactions.
Referred by Multi-Modality: A Unified Temporal Transformer for Video Object Segmentation
In this paper, we propose MUTR, a Multi-modal Unified Temporal transformer for Referring video object segmentation.
Referring Expression Segmentation
Referring Video Object Segmentation
+2
Networks are Slacking Off: Understanding Generalization Problem in Image Deraining
A prevailing perspective in deep learning encourages the use of highly complex training data, with the expectation that a richer image content knowledge will facilitate overcoming the generalization problem.
EmbodiedGPT: Vision-Language Pre-Training via Embodied Chain of Thought
In this work, we introduce EmbodiedGPT, an end-to-end multi-modal foundation model for embodied AI, empowering embodied agents with multi-modal understanding and execution capabilities.
VideoLLM: Modeling Video Sequence with Large Language Models
Building upon this insight, we propose a novel framework called VideoLLM that leverages the sequence reasoning capabilities of pre-trained LLMs from natural language processing (NLP) for video sequence understanding.
VisionLLM: Large Language Model is also an Open-Ended Decoder for Vision-Centric Tasks
We hope this model can set a new baseline for generalist vision and language models.
Instruct2Act: Mapping Multi-modality Instructions to Robotic Actions with Large Language Model
This paper presents Instruct2Act, a framework that utilizes Large Language Models to map multi-modal instructions to sequential actions for robotic manipulation tasks.
A Survey on Segment Anything Model (SAM): Vision Foundation Model Meets Prompt Engineering
This is an ongoing project and we intend to update the manuscript on a regular basis.
VideoChat: Chat-Centric Video Understanding
In this study, we initiate an exploration into video understanding by introducing VideoChat, an end-to-end chat-centric video understanding system.
Video-based Generative Performance Benchmarking (Consistency)
Video-based Generative Performance Benchmarking (Contextual Understanding)
+5
InternGPT: Solving Vision-Centric Tasks by Interacting with ChatGPT Beyond Language
Different from existing interactive systems that rely on pure language, by incorporating pointing instructions, the proposed iGPT significantly improves the efficiency of communication between users and chatbots, as well as the accuracy of chatbots in vision-centric tasks, especially in complicated visual scenarios where the number of objects is greater than 2.
Causal Discovery with Unobserved Variables: A Proxy Variable Approach
Our observation is that discretizing continuous variables can can lead to serious errors and comprise the power of the proxy.
LEO: Generative Latent Image Animator for Human Video Synthesis
Our key idea is to represent motion as a sequence of flow maps in the generation process, which inherently isolate motion from appearance.
Long-Term Rhythmic Video Soundtracker
To this end, we present Long-Term Rhythmic Video Soundtracker (LORIS), a novel framework to synthesize long-term conditional waveforms.
Segment Anything Model (SAM) Meets Glass: Mirror and Transparent Objects Cannot Be Easily Detected
Meta AI Research has recently released SAM (Segment Anything Model) which is trained on a large segmentation dataset of over 1 billion masks.
LLaMA-Adapter V2: Parameter-Efficient Visual Instruction Model
This strategy effectively alleviates the interference between the two tasks of image-text alignment and instruction following and achieves strong multi-modal reasoning with only a small-scale image-text and instruction dataset.
Ranked #1 on
Video-based Generative Performance Benchmarking (Temporal Understanding)
on VideoInstruct
Instruction Following
Optical Character Recognition (OCR)
+6
Hierarchical Diffusion Autoencoders and Disentangled Image Manipulation
To mitigate those limitations, we propose Hierarchical Diffusion Autoencoders (HDAE) that exploit the fine-grained-to-abstract and lowlevel-to-high-level feature hierarchy for the latent space of diffusion models.
Perception Imitation: Towards Synthesis-free Simulator for Autonomous Vehicles
We propose a perception imitation method to simulate results of a certain perception model, and discuss a new heuristic route of autonomous driving simulator without data synthesis.
STU-Net: Scalable and Transferable Medical Image Segmentation Models Empowered by Large-Scale Supervised Pre-training
However, the state-of-the-art models for medical image segmentation are still small-scale, with their parameters only in the tens of millions.
One Small Step for Generative AI, One Giant Leap for AGI: A Complete Survey on ChatGPT in AIGC Era
Overall, this work is the first to survey ChatGPT with a comprehensive review of its underlying technology, applications, and challenges.
MP-FedCL: Multi-Prototype Federated Contrastive Learning for Edge Intelligence
The existing single prototype-based strategy represents a sample by using the mean of the feature space.
VideoMAE V2: Scaling Video Masked Autoencoders with Dual Masking
Finally, we successfully train a video ViT model with a billion parameters, which achieves a new state-of-the-art performance on the datasets of Kinetics (90. 0% on K400 and 89. 9% on K600) and Something-Something (68. 7% on V1 and 77. 0% on V2).
Ranked #1 on
Temporal Action Localization
on FineAction
(using extra training data)
LLaMA-Adapter: Efficient Fine-tuning of Language Models with Zero-init Attention
We present LLaMA-Adapter, a lightweight adaption method to efficiently fine-tune LLaMA into an instruction-following model.
Ranked #2 on
Music Question Answering
on MusicQA Dataset
Unmasked Teacher: Towards Training-Efficient Video Foundation Models
Previous VFMs rely on Image Foundation Models (IFMs), which face challenges in transferring to the video domain.
Ranked #1 on
Zero-Shot Video Retrieval
on LSMDC
(using extra training data)
Fine-grained Audible Video Description
We explore a new task for audio-visual-language modeling called fine-grained audible video description (FAVD).
Prototype Helps Federated Learning: Towards Faster Convergence
In this paper, a prototype-based federated learning framework is proposed, which can achieve better inference performance with only a few changes to the last global iteration of the typical federated learning process.
A Complete Survey on Generative AI (AIGC): Is ChatGPT from GPT-4 to GPT-5 All You Need?
As ChatGPT goes viral, generative AI (AIGC, a. k. a AI-generated content) has made headlines everywhere because of its ability to analyze and create text, images, and beyond.
Video Dehazing via a Multi-Range Temporal Alignment Network with Physical Prior
Video dehazing aims to recover haze-free frames with high visibility and contrast.
Uni3D: A Unified Baseline for Multi-dataset 3D Object Detection
In this paper, we study the task of training a unified 3D detector from multiple datasets.
SCPNet: Semantic Scene Completion on Point Cloud
We propose a simple yet effective label rectification strategy, which uses off-the-shelf panoptic segmentation labels to remove the traces of dynamic objects in completion labels, greatly improving the performance of deep models especially for those moving objects.
Ranked #1 on
3D Semantic Scene Completion
on SemanticKITTI
Bi3D: Bi-domain Active Learning for Cross-domain 3D Object Detection
Unsupervised Domain Adaptation (UDA) technique has been explored in 3D cross-domain tasks recently.
Aleth-NeRF: Low-light Condition View Synthesis with Concealing Fields
Common capture low-light scenes are challenging for most computer vision techniques, including Neural Radiance Fields (NeRF).
Rethinking Range View Representation for LiDAR Segmentation
We show that, for the first time, a range view method is able to surpass the point, voxel, and multi-view fusion counterparts in the competing LiDAR semantic and panoptic segmentation benchmarks, i. e., SemanticKITTI, nuScenes, and ScribbleKITTI.
Ranked #2 on
3D Semantic Segmentation
on SemanticKITTI
FCN+: Global Receptive Convolution Makes FCN Great Again
The motivation of GRC is that different channels of a convolutional filter can have different grid sampling locations across the whole input feature map.
LoGoNet: Towards Accurate 3D Object Detection with Local-to-Global Cross-Modal Fusion
Notably, LoGoNet ranks 1st on Waymo 3D object detection leaderboard and obtains 81. 02 mAPH (L2) detection performance.
OpenICL: An Open-Source Framework for In-context Learning
However, the implementation of ICL is sophisticated due to the diverse retrieval and inference methods involved, as well as the varying pre-processing requirements for different models, datasets, and tasks.
Prompt, Generate, then Cache: Cascade of Foundation Models makes Strong Few-shot Learners
Our CaFo incorporates CLIP's language-contrastive knowledge, DINO's vision-contrastive knowledge, DALL-E's vision-generative knowledge, and GPT-3's language-generative knowledge.
Uncertainty-Estimation with Normalized Logits for Out-of-Distribution Detection
However, neural networks often suffer from the overconfidence issue, making high confidence for OOD data which are never seen during training process and may be irrelevant to training data, namely in-distribution (ID) data.
Learning Open-vocabulary Semantic Segmentation Models From Natural Language Supervision
The former aims to infer all masked entities in the caption given the group tokens, that enables the model to learn fine-grained alignment between visual groups and text entities.
CLIP2Scene: Towards Label-efficient 3D Scene Understanding by CLIP
For the first time, our pre-trained network achieves annotation-free 3D semantic segmentation with 20. 8% and 25. 08% mIoU on nuScenes and ScanNet, respectively.
Policy Pre-training for Autonomous Driving via Self-supervised Geometric Modeling
Witnessing the impressive achievements of pre-training techniques on large-scale data in the field of computer vision and natural language processing, we wonder whether this idea could be adapted in a grab-and-go spirit, and mitigate the sample inefficiency problem for visuomotor driving.
Learning Weather-General and Weather-Specific Features for Image Restoration Under Multiple Adverse Weather Conditions
Inspired by this observation, we design an efficient unified framework with a two-stage training strategy to explore the weather-general and weather-specific features.
UniFormerV2: Unlocking the Potential of Image ViTs for Video Understanding
The prolific performances of Vision Transformers (ViTs) in image tasks have prompted research into adapting the image ViTs for video tasks.
HTML: Hybrid Temporal-scale Multimodal Learning Framework for Referring Video Object Segmentation
To tackle this problem, we propose a concise Hybrid Temporal-scale Multimodal Learning (HTML) framework, which can effectively align lingual and visual features to discover core object semantics in the video, by learning multimodal interaction hierarchically from different temporal scales.
Referring Video Object Segmentation
Semantic Segmentation
+1
Neural Transformation Fields for Arbitrary-Styled Font Generation
Few-shot font generation (FFG), aiming at generating font images with a few samples, is an emerging topic in recent years due to the academic and commercial values.
Distilling Focal Knowledge From Imperfect Expert for 3D Object Detection
Specifically, a set of queries are leveraged to locate the instance-level areas for masked feature generation, to intensify feature representation ability in these areas.
DegAE: A New Pretraining Paradigm for Low-Level Vision
However, we argue that pretraining is more significant for high-cost tasks, where data acquisition is more challenging.
Multi-view Spectral Polarization Propagation for Video Glass Segmentation
In this paper, we present the first polarization-guided video glass segmentation propagation solution (PGVS-Net) that can robustly and coherently propagate glass segmentation in RGB-P video sequences.
Content Rating Classification for Fan Fiction
The problem is to take fan fiction text and determine the appropriate content rating.
Lego-MT: Learning Detachable Models for Massively Multilingual Machine Translation
To address the needs of learning representations for all languages in a unified space, we propose a novel efficient training recipe, upon which we build an effective detachable model, Lego-MT.
MM-3DScene: 3D Scene Understanding by Customizing Masked Modeling with Informative-Preserved Reconstruction and Self-Distilled Consistency
Besides, such scenes with progressive masking ratios can also serve to self-distill their intrinsic spatial consistency, requiring to learn the consistent representations from unmasked areas.
Planning-oriented Autonomous Driving
Oriented at this, we revisit the key components within perception and prediction, and prioritize the tasks such that all these tasks contribute to planning.
Improving the Generalizability of Text-Based Emotion Detection by Leveraging Transformers with Psycholinguistic Features
In recent years, there has been increased interest in building predictive models that harness natural language processing and machine learning techniques to detect emotions from various text sources, including social media posts, micro-blogs or news articles.
MANTIS at TSAR-2022 Shared Task: Improved Unsupervised Lexical Simplification with Pretrained Encoders
In this paper we present our contribution to the TSAR-2022 Shared Task on Lexical Simplification of the EMNLP 2022 Workshop on Text Simplification, Accessibility, and Readability.
Exploring Hybrid and Ensemble Models for Multiclass Prediction of Mental Health Status on Social Media
In recent years, there has been a surge of interest in research on automatic mental health detection (MHD) from social media data leveraging advances in natural language processing and machine learning techniques.
(Psycho-)Linguistic Features Meet Transformer Models for Improved Explainable and Controllable Text Simplification
State-of-the-art text simplification (TS) systems adopt end-to-end neural network models to directly generate the simplified version of the input text, and usually function as a blackbox.
Learning 3D Representations from 2D Pre-trained Models via Image-to-Point Masked Autoencoders
Pre-training by numerous image data has become de-facto for robust 2D representations.
Ranked #1 on
3D Point Cloud Linear Classification
on ModelNet40
(using extra training data)
3D Point Cloud Linear Classification
Few-Shot 3D Point Cloud Classification
Diff-Font: Diffusion Model for Robust One-Shot Font Generation
Specifically, a large stroke-wise dataset is constructed, and a stroke-wise diffusion model is proposed to preserve the structure and the completion of each generated character.
InternVideo: General Video Foundation Models via Generative and Discriminative Learning
Specifically, InternVideo efficiently explores masked video modeling and video-language contrastive learning as the pretraining objectives, and selectively coordinates video representations of these two complementary frameworks in a learnable manner to boost various video applications.
Ranked #1 on
Temporal Action Localization
on THUMOS’14
(using extra training data)
Toward Efficient Language Model Pretraining and Downstream Adaptation via Self-Evolution: A Case Study on SuperGLUE
This technical report briefly describes our JDExplore d-team's Vega v2 submission on the SuperGLUE leaderboard.
Improving Training and Inference of Face Recognition Models via Random Temperature Scaling
Intuitively, FR algorithms can benefit from both the estimation of uncertainty and the detection of out-of-distribution (OOD) samples.
ResFormer: Scaling ViTs with Multi-Resolution Training
We introduce, ResFormer, a framework that is built upon the seminal idea of multi-resolution training for improved performance on a wide spectrum of, mostly unseen, testing resolutions.
BEVFormer v2: Adapting Modern Image Backbones to Bird's-Eye-View Recognition via Perspective Supervision
The proposed method is verified with a wide spectrum of traditional and modern image backbones and achieves new SoTA results on the large-scale nuScenes dataset.
Ranked #4 on
3D Object Detection
on nuScenes Camera Only
InternVideo-Ego4D: A Pack of Champion Solutions to Ego4D Challenges
In this report, we present our champion solutions to five tracks at Ego4D challenge.
Ranked #1 on
State Change Object Detection
on Ego4D
Towards All-in-one Pre-training via Maximizing Multi-modal Mutual Information
It has been proved that combining multiple pre-training strategies and data from various modalities/sources can greatly boost the training of large-scale models.
Ranked #2 on
Object Detection
on LVIS v1.0 minival
(using extra training data)
Uni-Perceiver v2: A Generalist Model for Large-Scale Vision and Vision-Language Tasks
In this paper, we propose Uni-Perceiver v2, which is the first generalist model capable of handling major large-scale vision and vision-language tasks with competitive performance.
UniFormerV2: Spatiotemporal Learning by Arming Image ViTs with Video UniFormer
UniFormer has successfully alleviated this issue, by unifying convolution and self-attention as a relation aggregator in the transformer format.
Stare at What You See: Masked Image Modeling without Reconstruction
However, unlike the low-level features such as pixel values, we argue the features extracted by powerful teacher models already encode rich semantic correlation across regions in an intact image. This raises one question: is reconstruction necessary in Masked Image Modeling (MIM) with a teacher model?
Demystify Transformers & Convolutions in Modern Image Deep Networks
Although the novel feature transformation designs are often claimed as the source of gain, some backbones may benefit from advanced engineering techniques, which makes it hard to identify the real gain from the key feature transformation operators.
InternImage: Exploring Large-Scale Vision Foundation Models with Deformable Convolutions
Compared to the great progress of large-scale vision transformers (ViTs) in recent years, large-scale models based on convolutional neural networks (CNNs) are still in an early state.
Ranked #1 on
Instance Segmentation
on COCO test-dev
(using extra training data)
PalGAN: Image Colorization with Palette Generative Adversarial Networks
Multimodal ambiguity and color bleeding remain challenging in colorization.
VideoPipe 2022 Challenge: Real-World Video Understanding for Urban Pipe Inspection
For this reason, we propose to advance research areas of video understanding, with a shift from traditional action recognition to industrial anomaly analysis.
Wider and Higher: Intensive Integration and Global Foreground Perception for Image Matting
This paper reviews recent deep-learning-based matting research and conceives our wider and higher motivation for image matting.
Hierarchical and Progressive Image Matting
In this paper, we propose an end-to-end Hierarchical and Progressive Attention Matting Network (HAttMatting++), which can better predict the opacity of the foreground from single RGB images without additional input.
Efficient Image Super-Resolution using Vast-Receptive-Field Attention
In this work, we design an efficient SR network by improving the attention mechanism.
Low-Resolution Action Recognition for Tiny Actions Challenge
Second, these activities are naturally distributed in a long-tailed way.
Collaboration of Pre-trained Models Makes Better Few-shot Learner
In this paper, we propose CoMo, a Collaboration of pre-trained Models that incorporates diverse prior knowledge from various pre-training paradigms for better few-shot learning.
Delving into the Devils of Bird's-eye-view Perception: A Review, Evaluation and Recipe
As sensor configurations get more complex, integrating multi-source information from different sensors and representing features in a unified view come of vital importance.
Recurrent Bilinear Optimization for Binary Neural Networks
To address this issue, Recurrent Bilinear Optimization is proposed to improve the learning process of BNNs (RBONNs) by associating the intrinsic bilinear variables in the back propagation process.
Frozen CLIP Models are Efficient Video Learners
Video recognition has been dominated by the end-to-end learning paradigm -- first initializing a video recognition model with weights of a pretrained image model and then conducting end-to-end training on videos.
Ranked #20 on
Action Classification
on Kinetics-400
(using extra training data)
Vision-Centric BEV Perception: A Survey
In recent years, vision-centric Bird's Eye View (BEV) perception has garnered significant interest from both industry and academia due to its inherent advantages, such as providing an intuitive representation of the world and being conducive to data fusion.
GenText: Unsupervised Artistic Text Generation via Decoupled Font and Texture Manipulation
Specifically, our current work incorporates three different stages, stylization, destylization, and font transfer, respectively, into a unified platform with a single powerful encoder network and two separate style generator networks, one for font transfer, the other for stylization and destylization.
Tip-Adapter: Training-free Adaption of CLIP for Few-shot Classification
On top of that, the performance of Tip-Adapter can be further boosted to be state-of-the-art on ImageNet by fine-tuning the cache model for 10$\times$ fewer epochs than existing methods, which is both effective and efficient.
HQANN: Efficient and Robust Similarity Search for Hybrid Queries with Structured and Unstructured Constraints
The in-memory approximate nearest neighbor search (ANNS) algorithms have achieved great success for fast high-recall query processing, but are extremely inefficient when handling hybrid queries with unstructured (i. e., feature vectors) and structured (i. e., related attributes) constraints.
CP3: Unifying Point Cloud Completion by Pretrain-Prompt-Predict Paradigm
Inspired by prompting approaches from NLP, we creatively reinterpret point cloud generation and refinement as the prompting and predicting stages, respectively.
Trajectory-guided Control Prediction for End-to-end Autonomous Driving: A Simple yet Strong Baseline
The two branches are connected so that the control branch receives corresponding guidance from the trajectory branch at each time step.
Ranked #2 on
Autonomous Driving
on CARLA Leaderboard
Level 2 Autonomous Driving on a Single Device: Diving into the Devils of Openpilot
Equipped with a wide span of sensors, predominant autonomous driving solutions are becoming more modular-oriented for safe system design.
Siamese Image Modeling for Self-Supervised Vision Representation Learning
Driven by these analysis, we propose Siamese Image Modeling (SiameseIM), which predicts the dense representations of an augmented view, based on another masked view from the same image but with different augmentations.
You Only Need 90K Parameters to Adapt Light: A Light Weight Transformer for Image Enhancement and Exposure Correction
Challenging illumination conditions (low-light, under-exposure and over-exposure) in the real world not only cast an unpleasant visual appearance but also taint the computer vision tasks.
Ranked #2 on
Image Enhancement
on Exposure-Errors
Point-M2AE: Multi-scale Masked Autoencoders for Hierarchical Point Cloud Pre-training
By fine-tuning on downstream tasks, Point-M2AE achieves 86. 43% accuracy on ScanObjectNN, +3. 36% to the second-best, and largely benefits the few-shot classification, part segmentation and 3D object detection with the hierarchical pre-training scheme.
Ranked #3 on
3D Point Cloud Linear Classification
on ModelNet40
(using extra training data)
Vision Transformer Adapter for Dense Predictions
This work investigates a simple yet powerful dense prediction task adapter for Vision Transformer (ViT).
Ranked #3 on
Semantic Segmentation
on PASCAL Context
Evaluating the Generalization Ability of Super-Resolution Networks
However, research on the generalization ability of Super-Resolution (SR) networks is currently absent.
Blueprint Separable Residual Network for Efficient Image Super-Resolution
One is the usage of blueprint separable convolution (BSConv), which takes place of the redundant convolution operation.