Search Results for author:
Found 44 papers, 12 papers with code
RoboGSim: A Real2Sim2Real Robotic Gaussian Splatting Simulator
To address these limitations, we introduce the RoboGSim, a real2sim2real robotic simulator, powered by 3D Gaussian Splatting and the physics engine.
EMMA: Empowering Multi-modal Mamba with Structural and Hierarchical Alignment
In this work, we propose Empowering Multi-modal Mamba with Structural and Hierarchical Alignment (EMMA), which enables the MLLM to extract fine-grained visual information.
Blocks as Probes: Dissecting Categorization Ability of Large Multimodal Models
Categorization, a core cognitive ability in humans that organizes objects based on common features, is essential to cognitive science as well as computer vision.
GPSFormer: A Global Perception and Local Structure Fitting-based Transformer for Point Cloud Understanding
The core of GPSFormer is the Global Perception Module (GPM) and the Local Structure Fitting Convolution (LSFConv).
Ranked #4 on
3D Point Cloud Classification
on ScanObjectNN
Q-Sparse: All Large Language Models can be Fully Sparsely-Activated
We introduce, Q-Sparse, a simple yet effective approach to training sparsely-activated large language models (LLMs).
AIC MLLM: Autonomous Interactive Correction MLLM for Robust Robotic Manipulation
The ability to reflect on and correct failures is crucial for robotic systems to interact stably with real-life objects. Observing the generalization and reasoning capabilities of Multimodal Large Language Models (MLLMs), previous approaches have aimed to utilize these models to enhance robotic systems accordingly. However, these methods typically focus on high-level planning corrections using an additional MLLM, with limited utilization of failed samples to correct low-level contact poses which is particularly prone to occur during articulated object manipulation. To address this gap, we propose an Autonomous Interactive Correction (AIC) MLLM, which makes use of previous low-level interaction experiences to correct SE(3) pose predictions for articulated object.
M4U: Evaluating Multilingual Understanding and Reasoning for Large Multimodal Models
In this work, we introduce M4U, a novel and challenging benchmark for assessing the capability of multi-discipline multilingual multimodal understanding and reasoning.
The Era of 1-bit LLMs: All Large Language Models are in 1.58 Bits
Recent research, such as BitNet, is paving the way for a new era of 1-bit Large Language Models (LLMs).
Glance and Focus: Memory Prompting for Multi-Event Video Question Answering
Instead of that, we train an Encoder-Decoder to generate a set of dynamic event memories at the glancing stage.
Ranked #1 on
Video Question Answering
on AGQA 2.0 balanced
BitNet: Scaling 1-bit Transformers for Large Language Models
The increasing size of large language models has posed challenges for deployment and raised concerns about environmental impact due to high energy consumption.
From Node to Graph: Joint Reasoning on Visual-Semantic Relational Graph for Zero-Shot Detection
Zero-Shot Detection (ZSD), which aims at localizing andrecognizing unseen objects in a complicated scene, usuallyleverages the visual and semantic information of individ-ual objects alone.
Ranked #6 on
Generalized Zero-Shot Object Detection
on MS-COCO
Generalized Zero-Shot Object Detection
Scene Understanding
+1
SEGA: Semantic Guided Attention on Visual Prototype for Few-Shot Learning
However, human can learn new classes quickly even given few samples since human can tell what discriminative features should be focused on about each category based on both the visual and semantic prior knowledge.
FAIEr: Fidelity and Adequacy Ensured Image Caption Evaluation
Then for evaluating the adequacy of the candidate caption, it highlights the image gist on the visual scene graph under the guidance of the reference captions.
FAIR1M: A Benchmark Dataset for Fine-grained Object Recognition in High-Resolution Remote Sensing Imagery
In this paper, we propose a novel benchmark dataset with more than 1 million instances and more than 15, 000 images for Fine-grAined object recognItion in high-Resolution remote sensing imagery which is named as FAIR1M.
Holistic Pose Graph: Modeling Geometric Structure Among Objects in a Scene Using Graph Inference for 3D Object Prediction
The inference of the HPG uses GRU to encode the pose features from their corresponding regions in a single RGB image, and passes messages along the graph structure iteratively to improve the predicted poses.
Topic Scene Graph Generation by Attention Distillation From Caption
To this end, we let the scene graph borrow the ability from the image caption so that it can be a specialist on the basis of remaining all-around, resulting in the so-called Topic Scene Graph.
Env-QA: A Video Question Answering Benchmark for Comprehensive Understanding of Dynamic Environments
Visual understanding goes well beyond the study of images or videos on the web.
CVPR 2020 Continual Learning in Computer Vision Competition: Approaches, Results, Current Challenges and Future Directions
In the last few years, we have witnessed a renewed and fast-growing interest in continual learning with deep neural networks with the shared objective of making current AI systems more adaptive, efficient and autonomous.
Sketching Image Gist: Human-Mimetic Hierarchical Scene Graph Generation
Scene graph aims to faithfully reveal humans' perception of image content.
Multi-Modal Graph Neural Network for Joint Reasoning on Vision and Scene Text
Then, we introduce three aggregators which guide the message passing from one graph to another to utilize the contexts in various modalities, so as to refine the features of nodes.
Deep Heterogeneous Hashing for Face Video Retrieval
To tackle the key challenge of hashing on the manifold, a well-studied Riemannian kernel mapping is employed to project data (i. e. covariance matrices) into Euclidean space and thus enables to embed the two heterogeneous representations into a common Hamming space, where both intra-space discriminability and inter-space compatibility are considered.
Cross-modal Scene Graph Matching for Relationship-aware Image-Text Retrieval
In the light of recent success of scene graph in many CV and NLP tasks for describing complex natural scenes, we propose to represent image and text with two kinds of scene graphs: visual scene graph (VSG) and textual scene graph (TSG), each of which is exploited to jointly characterize objects and relationships in the corresponding modality.
Hierarchical Disentangle Network for Object Representation Learning
In this paper, we propose the hierarchical disentangle network (HDN) to exploit the rich hierarchical characteristics among categories to divide the disentangling process in a coarse-to-fine manner, such that each level only focuses on learning the specific representations in its granularity and finally the common and unique representations in all granularities jointly constitute the raw object.
Transferable Contrastive Network for Generalized Zero-Shot Learning
Zero-shot learning (ZSL) is a challenging problem that aims to recognize the target categories without seen data, where semantic information is leveraged to transfer knowledge from some source classes.
Ranked #7 on
Zero-Shot Learning
on SUN Attribute
CRIC: A VQA Dataset for Compositional Reasoning on Vision and Commonsense
To comprehensively evaluate such abilities, we propose a VQA benchmark, CRIC, which introduces new types of questions about Compositional Reasoning on vIsion and Commonsense, and an evaluation metric integrating the correctness of answering and commonsense grounding.
WIDER Face and Pedestrian Challenge 2018: Methods and Results
This paper presents a review of the 2018 WIDER Challenge on Face and Pedestrian.
Learning Class Prototypes via Structure Alignment for Zero-Shot Recognition
Zero-shot learning (ZSL) aims to recognize objects of novel classes without any training samples of specific classes, which is achieved by exploiting the semantic information and auxiliary datasets.
Structure Inference Net: Object Detection Using Scene-Level Context and Instance-Level Relationships
Context is important for accurate visual recognition.
Learning Discriminative Latent Attributes for Zero-Shot Classification
Zero-shot learning (ZSL) aims to transfer knowledge from observed classes to the unseen classes, based on the assumption that both the seen and unseen classes share a common semantic space, among which attributes enjoy a great popularity.
Learning Multifunctional Binary Codes for Both Category and Attribute Oriented Retrieval Tasks
In this paper we propose a unified framework to address multiple realistic image retrieval tasks concerning both category and attributes.
Discriminative Covariance Oriented Representation Learning for Face Recognition With Image Sets
For face recognition with image sets, while most existing works mainly focus on building robust set models with hand-crafted feature, it remains a research gap to learn better image representations which can closely match the subsequent image set modeling and classification.
Geometry-aware Similarity Learning on SPD Manifolds for Visual Recognition
Specifically, by exploiting the Riemannian geometry of the manifold of fixed-rank Positive Semidefinite (PSD) matrices, we present a new solution to reduce optimizing over the space of column full-rank transformation matrices to optimizing on the PSD manifold which has a well-established Riemannian structure.
Cross Euclidean-to-Riemannian Metric Learning with Application to Face Recognition from Video
With this mapping, the problem of learning a cross-view metric between the two source heterogeneous spaces can be expressed as learning a single-view Euclidean distance metric in the target common Euclidean space.
Dual Purpose Hashing
Recent years have seen more and more demand for a unified framework to address multiple realistic image retrieval tasks concerning both category and attributes.
Deep Supervised Hashing for Fast Image Retrieval
In this paper, we present a new hashing method to learn compact binary codes for highly efficient image retrieval on large-scale datasets.
Ranked #1 on
Image Retrieval
on CIFAR-10
Two Birds, One Stone: Jointly Learning Binary Code for Large-Scale Face Image Retrieval and Attributes Prediction
In this way, the learned binary codes can be applied to not only fine-grained face image retrieval, but also facial attributes prediction, which is the very innovation of this work, just like killing two birds with one stone.
Learning Mid-level Words on Riemannian Manifold for Action Recognition
Human action recognition remains a challenging task due to the various sources of video data and large intra-class variations.
Learning Expressionlets via Universal Manifold Model for Dynamic Facial Expression Recognition
3) the local modes on each STM can be instantiated by fitting to UMM, and the corresponding expressionlet is constructed by modeling the variations in each local mode.
Dynamic Facial Expression Recognition
Facial Expression Recognition
+1
Face Video Retrieval With Image Query via Hashing Across Euclidean Space and Riemannian Manifold
Retrieving videos of a specific person given his/her face image as query becomes more and more appealing for applications like smart movie fast-forwards and suspect searching.
Discriminant Analysis on Riemannian Manifold of Gaussian Distributions for Face Recognition With Image Sets
This paper presents a method named Discriminant Analysis on Riemannian manifold of Gaussian distributions (DARG) to solve the problem of face recognition with image sets.
Projection Metric Learning on Grassmann Manifold With Application to Video Based Face Recognition
In video based face recognition, great success has been made by representing videos as linear subspaces, which typically lie in a special type of non-Euclidean space known as Grassmann manifold.
A new hierarchical method for inter-patient heartbeat classification using random projections and RR intervals
This was done in order to improve the classification performance of these two classes of heartbeats by using different features and classification methods.
Ranked #2 on
Heartbeat Classification
on MIT-BIH AR
Learning Expressionlets on Spatio-Temporal Manifold for Dynamic Facial Expression Recognition
In this paper, we attempt to solve both problems via manifold modeling of videos based on a novel mid-level representation, i. e. expressionlet.
Dynamic Facial Expression Recognition
Facial Expression Recognition
+1
Learning Euclidean-to-Riemannian Metric for Point-to-Set Classification
Since the points commonly lie in Euclidean space while the sets are typically modeled as elements on Riemannian manifold, they can be treated as Euclidean points and Riemannian points respectively.
