Search Results for author:
Found 21 papers, 2 papers with code
SocialGesture: Delving into Multi-person Gesture Understanding
Previous research in human gesture recognition has largely overlooked multi-person interactions, which are crucial for understanding the social context of naturally occurring gestures.
Learning Predictive Visuomotor Coordination
Understanding and predicting human visuomotor coordination is crucial for applications in robotics, human-computer interaction, and assistive technologies.
Towards Online Multi-Modal Social Interaction Understanding
To address the challenges of missing the useful future context, we develop a novel framework, named Online-MMSI-VLM, that leverages two complementary strategies: multi-party conversation forecasting and social-aware visual prompting with multi-modal large language models.
Building a Mind Palace: Structuring Environment-Grounded Semantic Graphs for Effective Long Video Analysis with LLMs
Long-form video understanding with Large Vision Language Models is challenged by the need to analyze temporally dispersed yet spatially concentrated key moments within limited context windows.
Unleashing In-context Learning of Autoregressive Models for Few-shot Image Manipulation
We also introduce a relation regularization method to further disentangle image transformation features from irrelevant contents in exemplar images.
Human Action Anticipation: A Survey
Predicting future human behavior is an increasingly popular topic in computer vision, driven by the interest in applications such as autonomous vehicles, digital assistants and human-robot interactions.
MM-SpuBench: Towards Better Understanding of Spurious Biases in Multimodal LLMs
Our findings illuminate the persistence of the reliance on spurious correlations from these models and underscore the urge for new methodologies to mitigate spurious biases.
What is the Visual Cognition Gap between Humans and Multimodal LLMs?
Recently, Multimodal Large Language Models (MLLMs) have shown great promise in language-guided perceptual tasks such as recognition, segmentation, and object detection.
Modeling Multimodal Social Interactions: New Challenges and Baselines with Densely Aligned Representations
Furthermore, we propose a novel multimodal baseline that leverages densely aligned language-visual representations by synchronizing visual features with their corresponding utterances.
Weakly Supervised Segmentation of Vertebral Bodies with Iterative Slice-propagation
In this paper, we propose a Weakly supervised Iterative Spinal Segmentation (WISS) method leveraging only four corner landmark weak labels on a single sagittal slice to achieve automatic volumetric segmentation from CT images for VBs.
Learning-based Bone Quality Classification Method for Spinal Metastasis
In this paper, we explore a learning-based automatic bone quality classification method for spinal metastasis based on CT images.
LEGO: Learning EGOcentric Action Frame Generation via Visual Instruction Tuning
Additionally, existing diffusion-based image manipulation models are sub-optimal in controlling the state transition of an action in egocentric image pixel space because of the domain gap.
Listen to Look into the Future: Audio-Visual Egocentric Gaze Anticipation
Motivated by this observation, we introduce the first model that leverages both the video and audio modalities for egocentric gaze anticipation.
Werewolf Among Us: A Multimodal Dataset for Modeling Persuasion Behaviors in Social Deduction Games
We also explore the generalization ability of language models for persuasion modeling and the role of persuasion strategies in predicting social deduction game outcomes.
In the Eye of Transformer: Global-Local Correlation for Egocentric Gaze Estimation
To this end, we design the transformer encoder to embed the global context as one additional visual token and further propose a novel Global-Local Correlation (GLC) module to explicitly model the correlation of the global token and each local token.
A deep learning pipeline for localization, differentiation, and uncertainty estimation of liver lesions using multi-phasic and multi-sequence MRI
Results: the proposed CAD solution achieves a mean F1 score of 0. 62, outperforming the abdominal radiologist (0. 47), matching the junior hepatology radiologist (0. 61), and underperforming the senior hepatology radiologist (0. 68).
VeniBot: Towards Autonomous Venipuncture with Semi-supervised Vein Segmentation from Ultrasound Images
In the modern medical care, venipuncture is an indispensable procedure for both diagnosis and treatment.
VeniBot: Towards Autonomous Venipuncture with Automatic Puncture Area and Angle Regression from NIR Images
For the automation, we focus on the positioning part and propose a Dual-In-Dual-Out network based on two-step learning and two-task learning, which can achieve fully automatic regression of the suitable puncture area and angle from near-infrared(NIR) images.
Scalable Semi-supervised Landmark Localization for X-ray Images using Few-shot Deep Adaptive Graph
It first trains a DAG model on the labeled data and then fine-tunes the pre-trained model on the unlabeled data with a teacher-student SSL mechanism.
Hetero-Modal Learning and Expansive Consistency Constraints for Semi-Supervised Detection from Multi-Sequence Data
Lesion detection serves a critical role in early diagnosis and has been well explored in recent years due to methodological advancesand increased data availability.
Fully-Automated Liver Tumor Localization and Characterization from Multi-Phase MR Volumes Using Key-Slice ROI Parsing: A Physician-Inspired Approach
Using radiological scans to identify liver tumors is crucial for proper patient treatment.
