Search Results for author:
Found 35 papers, 18 papers with code
Bootstrap Your Own Views: Masked Ego-Exo Modeling for Fine-grained View-invariant Video Representations
View-invariant representation learning from egocentric (first-person, ego) and exocentric (third-person, exo) videos is a promising approach toward generalizing video understanding systems across multiple viewpoints.
Single Ground Truth Is Not Enough: Add Linguistic Variability to Aspect-based Sentiment Analysis Evaluation
Aspect-based sentiment analysis (ABSA) is the challenging task of extracting sentiment along with its corresponding aspects and opinions from human language.
Aspect-Based Sentiment Analysis
Aspect-Based Sentiment Analysis (ABSA)
+2
Read, Watch and Scream! Sound Generation from Text and Video
Despite the impressive progress of multimodal generative models, video-to-audio generation still suffers from limited performance and limits the flexibility to prioritize sound synthesis for specific objects within the scene.
Ranked #7 on
Video-to-Sound Generation
on VGG-Sound
Bridging Vision and Language Spaces with Assignment Prediction
This paper introduces VLAP, a novel approach that bridges pretrained vision models and large language models (LLMs) to make frozen LLMs understand the visual world.
KorNAT: LLM Alignment Benchmark for Korean Social Values and Common Knowledge
For the common knowledge dataset, we constructed samples based on Korean textbooks and GED reference materials.
Dense Text-to-Image Generation with Attention Modulation
To address this, we propose DenseDiffusion, a training-free method that adapts a pre-trained text-to-image model to handle such dense captions while offering control over the scene layout.
Hierarchical Visual Primitive Experts for Compositional Zero-Shot Learning
Previous works for CZSL often suffer from grasping the contextuality between attribute and object, as well as the discriminability of visual features, and the long-tailed distribution of real-world compositional data.
VisAlign: Dataset for Measuring the Degree of Alignment between AI and Humans in Visual Perception
In this paper, we focus on the models' visual perception alignment with humans, further referred to as AI-human visual alignment.
Panoramic Image-to-Image Translation
This task is difficult due to the geometric distortion of panoramic images and the lack of a panoramic image dataset with diverse conditions, like weather or time.
Three Recipes for Better 3D Pseudo-GTs of 3D Human Mesh Estimation in the Wild
Recovering 3D human mesh in the wild is greatly challenging as in-the-wild (ITW) datasets provide only 2D pose ground truths (GTs).
Ranked #6 on
3D Multi-Person Pose Estimation
on MuPoTS-3D
Dual-path Adaptation from Image to Video Transformers
In this paper, we efficiently transfer the surpassing representation power of the vision foundation models, such as ViT and Swin, for video understanding with only a few trainable parameters.
Ranked #1 on
Action Classification
on Diving-48
Let 2D Diffusion Model Know 3D-Consistency for Robust Text-to-3D Generation
Text-to-3D generation has shown rapid progress in recent days with the advent of score distillation, a methodology of using pretrained text-to-2D diffusion models to optimize neural radiance field (NeRF) in the zero-shot setting.
Imaginary Voice: Face-styled Diffusion Model for Text-to-Speech
This is the first time that face images are used as a condition to train a TTS model.
Robust Camera Pose Refinement for Multi-Resolution Hash Encoding
Multi-resolution hash encoding has recently been proposed to reduce the computational cost of neural renderings, such as NeRF.
Semi-Parametric Video-Grounded Text Generation
Efficient video-language modeling should consider the computational cost because of a large, sometimes intractable, number of video frames.
Ranked #32 on
Video Question Answering
on NExT-QA
Specializing Multi-domain NMT via Penalizing Low Mutual Information
Multi-domain Neural Machine Translation (NMT) trains a single model with multiple domains.
Language-free Training for Zero-shot Video Grounding
Given an untrimmed video and a language query depicting a specific temporal moment in the video, video grounding aims to localize the time interval by understanding the text and video simultaneously.
MIDMs: Matching Interleaved Diffusion Models for Exemplar-based Image Translation
Specifically, we formulate a diffusion-based matching-and-generation framework that interleaves cross-domain matching and diffusion steps in the latent space by iteratively feeding the intermediate warp into the noising process and denoising it to generate a translated image.
Automatic Detection of Noisy Electrocardiogram Signals without Explicit Noise Labels
Automatic deep learning-based examination of ECG signals can lead to inaccurate diagnosis, and manual analysis involves rejection of noisy ECG samples by clinicians, which might cost extra time.
PointFix: Learning to Fix Domain Bias for Robust Online Stereo Adaptation
To mitigate this issue, we propose to incorporate an auxiliary point-selective network into a meta-learning framework, called PointFix, to provide a robust initialization of stereo models for online stereo adaptation.
Mutual Information Divergence: A Unified Metric for Multimodal Generative Models
Based on a recent trend that multimodal generative evaluations exploit a vison-and-language pre-trained model, we propose the negative Gaussian cross-mutual information using the CLIP features as a unified metric, coined by Mutual Information Divergence (MID).
Ranked #1 on
Human Judgment Classification
on Pascal-50S
Hallucination Pair-wise Detection (1-ref)
Hallucination Pair-wise Detection (4-ref)
+5
Probabilistic Representations for Video Contrastive Learning
This paper presents Probabilistic Video Contrastive Learning, a self-supervised representation learning method that bridges contrastive learning with probabilistic representation.
Pin the Memory: Learning to Generalize Semantic Segmentation
The rise of deep neural networks has led to several breakthroughs for semantic segmentation.
Multi-domain Unsupervised Image-to-Image Translation with Appearance Adaptive Convolution
We show that the proposed method produces visually diverse and plausible results in multiple domains compared to the state-of-the-art methods.
Exploration into Translation-Equivariant Image Quantization
This is an exploratory study that discovers the current image quantization (vector quantization) do not satisfy translation equivariance in the quantized space due to aliasing.
Unifying Heterogeneous Electronic Health Records Systems via Text-Based Code Embedding
EHR systems lack a unified code system forrepresenting medical concepts, which acts asa barrier for the deployment of deep learningmodels in large scale to multiple clinics and hos-pitals.
Conditional Generation of Periodic Signals with Fourier-Based Decoder
Periodic signals play an important role in daily lives.
Wide and Narrow: Video Prediction from Context and Motion
Video prediction, forecasting the future frames from a sequence of input frames, is a challenging task since the view changes are influenced by various factors, such as the global context surrounding the scene and local motion dynamics.
Self-balanced Learning For Domain Generalization
Domain generalization aims to learn a prediction model on multi-domain source data such that the model can generalize to a target domain with unknown statistics.
Unifying Heterogeneous Electronic Health Records Systems via Text-Based Code Embedding
To overcome this problem, we introduce Description-based Embedding, DescEmb, a code-agnostic description-based representation learning framework for predictive modeling on EHR.
CausalCity: Complex Simulations with Agency for Causal Discovery and Reasoning
The ability to perform causal and counterfactual reasoning are central properties of human intelligence.
Bridge to Answer: Structure-aware Graph Interaction Network for Video Question Answering
As a result, our method can learn the question conditioned visual representations attributed to appearance and motion that show powerful capability for video question answering.
Looking into Your Speech: Learning Cross-modal Affinity for Audio-visual Speech Separation
In this paper, we address the problem of separating individual speech signals from videos using audio-visual neural processing.
SumGraph: Video Summarization via Recursive Graph Modeling
The goal of video summarization is to select keyframes that are visually diverse and can represent a whole story of an input video.
Context-Aware Emotion Recognition Networks
We present deep networks for context-aware emotion recognition, called CAER-Net, that exploit not only human facial expression but also context information in a joint and boosting manner.
Ranked #1 on
Emotion Recognition in Context
on CAER-Dynamic
