Search Results for author:
Found 20 papers, 8 papers with code
Optimus-2: Multimodal Minecraft Agent with Goal-Observation-Action Conditioned Policy
GOAP contains (1) an Action-guided Behavior Encoder that models causal relationships between observations and actions at each timestep, then dynamically interacts with the historical observation-action sequence, consolidating it into fixed-length behavior tokens, and (2) an MLLM that aligns behavior tokens with open-ended language instructions to predict actions auto-regressively.
PolaFormer: Polarity-aware Linear Attention for Vision Transformers
Linear attention has emerged as a promising alternative to softmax-based attention, leveraging kernelized feature maps to reduce complexity from quadratic to linear in sequence length.
CatV2TON: Taming Diffusion Transformers for Vision-Based Virtual Try-On with Temporal Concatenation
Comprehensive experiments demonstrate that CatV2TON outperforms existing methods in both image and video try-on tasks, offering a versatile and reliable solution for realistic virtual try-ons across diverse scenarios.
Transferable Adversarial Face Attack with Text Controlled Attribute
Extensive experiments on two high-resolution face recognition datasets validate that our TCA$^2$ method can generate natural text-guided adversarial impersonation faces with high transferability.
AlignMamba: Enhancing Multimodal Mamba with Local and Global Cross-modal Alignment
Extensive experiments on complete and incomplete multimodal fusion tasks demonstrate the effectiveness and efficiency of the proposed method.
CATCH: Complementary Adaptive Token-level Contrastive Decoding to Mitigate Hallucinations in LVLMs
Large Vision-Language Model (LVLM) systems have demonstrated impressive vision-language reasoning capabilities but suffer from pervasive and severe hallucination issues, posing significant risks in critical domains such as healthcare and autonomous systems.
EMMA: Empowering Multi-modal Mamba with Structural and Hierarchical Alignment
In this work, we propose Empowering Multi-modal Mamba with Structural and Hierarchical Alignment (EMMA), which enables the MLLM to extract fine-grained visual information.
ExpLLM: Towards Chain of Thought for Facial Expression Recognition
Specifically, we have designed the CoT mechanism from three key perspectives: key observations, overall emotional interpretation, and conclusion.
Facial Expression Recognition
Facial Expression Recognition (FER)
Optimus-1: Hybrid Multimodal Memory Empowered Agents Excel in Long-Horizon Tasks
On top of the Hybrid Multimodal Memory module, a multimodal agent, Optimus-1, is constructed with dedicated Knowledge-guided Planner and Experience-Driven Reflector, contributing to a better planning and reflection in the face of long-horizon tasks in Minecraft.
OV-DINO: Unified Open-Vocabulary Detection with Language-Aware Selective Fusion
To address these challenges, we propose a novel unified open-vocabulary detection method called OV-DINO, which is pre-trained on diverse large-scale datasets with language-aware selective fusion in a unified framework.
Ranked #5 on
Zero-Shot Object Detection
on MSCOCO
(AP metric, using extra
training data)
Prompt Customization for Continual Learning
In contrast to conventional methods that employ hard prompt selection, PGM assigns different coefficients to prompts from a fixed-sized pool of prompts and generates tailored prompts.
CricaVPR: Cross-image Correlation-aware Representation Learning for Visual Place Recognition
Over the past decade, most methods in visual place recognition (VPR) have used neural networks to produce feature representations.
Deep Homography Estimation for Visual Place Recognition
Moreover, we design a re-projection error of inliers loss to train the DHE network without additional homography labels, which can also be jointly trained with the backbone network to help it extract the features that are more suitable for local matching.
Enhancing Emotional Generation Capability of Large Language Models via Emotional Chain-of-Thought
In this paper, we propose the Emotional Chain-of-Thought (ECoT), a plug-and-play prompting method that enhances the performance of LLMs on various emotional generation tasks by aligning with human emotional intelligence guidelines.
Benign Shortcut for Debiasing: Fair Visual Recognition via Intervention with Shortcut Features
To this end, we propose \emph{Shortcut Debiasing}, to first transfer the target task's learning of bias attributes from bias features to shortcut features, and then employ causal intervention to eliminate shortcut features during inference.
Strip-MLP: Efficient Token Interaction for Vision MLP
Finally, based on the Strip MLP layer, we propose a novel \textbf{L}ocal \textbf{S}trip \textbf{M}ixing \textbf{M}odule (LSMM) to boost the token interaction power in the local region.
Relate auditory speech to EEG by shallow-deep attention-based network
Electroencephalography (EEG) plays a vital role in detecting how brain responses to different stimulus.
Positional-Spectral-Temporal Attention in 3D Convolutional Neural Networks for EEG Emotion Recognition
The proposed module, denoted by PST-Attention, consists of Positional, Spectral and Temporal Attention modules to explore more discriminative EEG features.
Efficient Spatialtemporal Context Modeling for Action Recognition
Inspired from 2D criss-cross attention used in segmentation task, we propose a recurrent 3D criss-cross attention (RCCA-3D) module to model the dense long-range spatiotemporal contextual information in video for action recognition.
Efficient Convolutional Auto-Encoding via Random Convexification and Frequency-Domain Minimization
The omnipresence of deep learning architectures such as deep convolutional neural networks (CNN)s is fueled by the synergistic combination of ever-increasing labeled datasets and specialized hardware.
