Search Results for author:
Found 114 papers, 53 papers with code
Learning Speaker-Invariant Visual Features for Lipreading
Lipreading is a challenging cross-modal task that aims to convert visual lip movements into spoken text.
Align is not Enough: Multimodal Universal Jailbreak Attack against Multimodal Large Language Models
Large Language Models (LLMs) have evolved into Multimodal Large Language Models (MLLMs), significantly enhancing their capabilities by integrating visual information and other types, thus aligning more closely with the nature of human intelligence, which processes a variety of data forms beyond just text.
Contrastive Alignment with Semantic Gap-Aware Corrections in Text-Video Retrieval
To mitigate this, we propose GARE, a Gap-Aware Retrieval framework that introduces a learnable, pair-specific increment Delta_ij between text t_i and video v_j to offload the tension from the global anchor representation.
VAEmo: Efficient Representation Learning for Visual-Audio Emotion with Knowledge Injection
Audiovisual emotion recognition (AVER) aims to infer human emotions from nonverbal visual-audio (VA) cues, offering modality-complementary and language-agnostic advantages.
Ranked #1 on
Video Emotion Recognition
on CREMA-D
Contrastive Learning
Dynamic Facial Expression Recognition
+3
Invariance Matters: Empowering Social Recommendation via Graph Invariant Learning
Graph-based social recommendation systems have shown significant promise in enhancing recommendation performance, particularly in addressing the issue of data sparsity in user behaviors.
Communication-Efficient and Personalized Federated Foundation Model Fine-Tuning via Tri-Matrix Adaptation
In federated learning, fine-tuning pre-trained foundation models poses significant challenges, particularly regarding high communication cost and suboptimal model performance due to data heterogeneity between the clients.
EgoBlind: Towards Egocentric Visual Assistance for the Blind People
We present EgoBlind, the first egocentric VideoQA dataset collected from blind individuals to evaluate the assistive capabilities of contemporary multimodal large language models (MLLMs).
SwimVG: Step-wise Multimodal Fusion and Adaption for Visual Grounding
Most existing methods transfer visual/linguistic knowledge separately by fully fine-tuning uni-modal pre-trained models, followed by a simple stack of visual-language transformers for multimodal fusion.
MoLoRec: A Generalizable and Efficient Framework for LLM-Based Recommendation
The first paradigm designs multi-domain or multi-task instruction data for generalizable recommendation, so as to align LLMs with general recommendation areas and deal with cold-start recommendation.
Less is More: Information Bottleneck Denoised Multimedia Recommendation
This is achieved by maximizing the mutual information between multimedia representation and recommendation tasks, while concurrently minimizing it between multimedia representation and pre-trained multimedia features.
Biomedical Relation Extraction via Adaptive Document-Relation Cross-Mapping and Concept Unique Identifier
Specifically, we propose a document-level Bio-RE framework via LLM Adaptive Document-Relation Cross-Mapping (ADRCM) Fine-Tuning and Concept Unique Identifier (CUI) Retrieval-Augmented Generation (RAG).
Discrete to Continuous: Generating Smooth Transition Poses from Sign Language Observations
To address this, we propose a novel framework, Sign-D2C, that employs a conditional diffusion model to synthesize contextually smooth transition frames, enabling the seamless construction of continuous sign language sequences.
Revisiting Audio-Visual Segmentation with Vision-Centric Transformer
Audio-Visual Segmentation (AVS) aims to segment sound-producing objects in video frames based on the associated audio signal.
Linguistics-Vision Monotonic Consistent Network for Sign Language Production
Sign Language Production (SLP) aims to generate sign videos corresponding to spoken language sentences, where the conversion of sign Glosses to Poses (G2P) is the key step.
Unveiling Uncertainty: A Deep Dive into Calibration and Performance of Multimodal Large Language Models
This paper investigates representative MLLMs, focusing on their calibration across various scenarios, including before and after visual fine-tuning, as well as before and after multimodal training of the base LLMs.
Sign-IDD: Iconicity Disentangled Diffusion for Sign Language Production
Sign-IDD incorporates a novel Iconicity Disentanglement (ID) module to bridge the gap between relative positions among joints.
Moderating the Generalization of Score-based Generative Model
To fill this gap, we first examine the current 'gold standard' in Machine Unlearning (MU), i. e., re-training the model after removing the undesirable training data, and find it does not work in SGMs.
Discrete to Continuous: Generating Smooth Transition Poses from Sign Language Observation
To address this, we propose a novel framework, Sign-D2C, that employs a conditional diffusion model to synthesize contextually smooth transition frames, enabling the seamless construction of continuous sign language sequences.
CARE Transformer: Mobile-Friendly Linear Visual Transformer via Decoupled Dual Interaction
In this paper, we propose a new de\textbf{C}oupled du\textbf{A}l-interactive linea\textbf{R} att\textbf{E}ntion (CARE) mechanism, revealing that features' decoupling and interaction can fully unleash the power of linear attention.
Multi-Stage Vision Token Dropping: Towards Efficient Multimodal Large Language Model
In this paper, we propose Multi-stage Token Dropping (MustDrop) to measure the importance of each token from the whole lifecycle, including the vision encoding stage, prefilling stage, and decoding stage.
DAT: Dialogue-Aware Transformer with Modality-Group Fusion for Human Engagement Estimation
Engagement estimation plays a crucial role in understanding human social behaviors, attracting increasing research interests in fields such as affective computing and human-computer interaction.
Text Proxy: Decomposing Retrieval from a 1-to-N Relationship into N 1-to-1 Relationships for Text-Video Retrieval
By replacing a single text query with a series of text proxies, TV-ProxyNet not only broadens the query scope but also achieves a more precise expansion.
Towards Unified Multimodal Editing with Enhanced Knowledge Collaboration
In this paper, we propose UniKE, a novel multimodal editing method that establishes a unified perspective and paradigm for intrinsic knowledge editing and external knowledge resorting.
Static for Dynamic: Towards a Deeper Understanding of Dynamic Facial Expressions Using Static Expression Data
Dynamic facial expression recognition (DFER) infers emotions from the temporal evolution of expressions, unlike static facial expression recognition (SFER), which relies solely on a single snapshot.
Ranked #3 on
Dynamic Facial Expression Recognition
on DFEW
Dynamic Facial Expression Recognition
Facial Expression Recognition
+1
Seeing is Believing? Enhancing Vision-Language Navigation using Visual Perturbations
Autonomous navigation for an embodied agent guided by natural language instructions remains a formidable challenge in vision-and-language navigation (VLN).
SDI-Net: Toward Sufficient Dual-View Interaction for Low-light Stereo Image Enhancement
However, these methods still do not fully exploit the interaction between left and right view information.
Exploring Robust Face-Voice Matching in Multilingual Environments
We focus on the impact of different languages in face-voice matching by building upon Fusion and Orthogonal Projection (FOP), introducing four key components: a dual-branch structure, dynamic sample pair weighting, robust data augmentation, and score polarization strategy.
Graph Bottlenecked Social Recommendation
In this paper, we focus on learning the denoised social structure to facilitate recommendation tasks from an information bottleneck perspective.
Path-Specific Causal Reasoning for Fairness-aware Cognitive Diagnosis
Then, we design a novel attribute-oriented predictor to decouple the sensitive attributes, in which fairness-related sensitive features will be eliminated and other useful information will be retained.
Double Correction Framework for Denoising Recommendation
To tackle the above limitations, we propose a Double Correction Framework for Denoising Recommendation (DCF), which contains two correction components from views of more precise sample dropping and avoiding more sparse data.
Multimodality Invariant Learning for Multimedia-Based New Item Recommendation
How to quickly provide recommendations for new items at the inference time is challenging.
Controllable Relation Disentanglement for Few-Shot Class-Incremental Learning
In this paper, we propose to tackle Few-Shot Class-Incremental Learning (FSCIL) from a new perspective, i. e., relation disentanglement, which means enhancing FSCIL via disentangling spurious relation between categories.
Gradient-Aware Logit Adjustment Loss for Long-tailed Classifier
Additionally, We find that most of the solutions to long-tailed problems are still biased towards head classes in the end, and we propose a simple and post hoc prediction re-balancing strategy to further mitigate the basis toward head class.
Doubly Abductive Counterfactual Inference for Text-based Image Editing
Through the lens of the formulation, we find that the crux of TBIE is that existing techniques hardly achieve a good trade-off between editability and fidelity, mainly due to the overfitting of the single-image fine-tuning.
Few-shot Learner Parameterization by Diffusion Time-steps
To this end, we find an inductive bias that the time-steps of a Diffusion Model (DM) can isolate the nuanced class attributes, i. e., as the forward diffusion adds noise to an image at each time-step, nuanced attributes are usually lost at an earlier time-step than the spurious attributes that are visually prominent.
Group Multi-View Transformer for 3D Shape Analysis with Spatial Encoding
The large model GMViT achieves excellent 3D classification and retrieval results on the benchmark datasets ModelNet, ShapeNetCore55, and MCB.
From Static to Dynamic: Adapting Landmark-Aware Image Models for Facial Expression Recognition in Videos
And the TMAs capture and model the relationships of dynamic changes in facial expressions, effectively extending the pre-trained image model for videos.
Ranked #2 on
Dynamic Facial Expression Recognition
on FERV39k
Dynamic Facial Expression Recognition
Facial Expression Recognition
+1
Deep Ensembles Meets Quantile Regression: Uncertainty-aware Imputation for Time Series
Our method not only produces accurate imputations that is robust to high missing rates, but also is computationally efficient due to the fast training of its non-generative model.
Viewport Prediction for Volumetric Video Streaming by Exploring Video Saliency and Trajectory Information
To our knowledge, this is the first comprehensive study of viewport prediction in volumetric video streaming.
One-bit Supervision for Image Classification: Problem, Solution, and Beyond
An intriguing property of the setting is that the burden of annotation largely alleviates in comparison to offering the accurate label.
Clarity ChatGPT: An Interactive and Adaptive Processing System for Image Restoration and Enhancement
The generalization capability of existing image restoration and enhancement (IRE) methods is constrained by the limited pre-trained datasets, making it difficult to handle agnostic inputs such as different degradation levels and scenarios beyond their design scopes.
Grid Jigsaw Representation with CLIP: A New Perspective on Image Clustering
Unsupervised representation learning for image clustering is essential in computer vision.
CropCap: Embedding Visual Cross-Partition Dependency for Image Captioning
Transformer-based approaches to image captioning have shown great success by utilizing long-term dependency for visual embedding.
Boundary Discretization and Reliable Classification Network for Temporal Action Detection
Furthermore, the reliable classification module (RCM) predicts reliable global action categories to reduce false positives.
Embedded Heterogeneous Attention Transformer for Cross-lingual Image Captioning
The crucial issue in this task is to model the global and the local matching between the image and different languages.
Generative Contrastive Graph Learning for Recommendation
Second, feature augmentation imposes the same scale noise augmentation on each node, which neglects the unique characteristics of nodes on the graph.
Advancing Incremental Few-shot Semantic Segmentation via Semantic-guided Relation Alignment and Adaptation
Incremental few-shot semantic segmentation (IFSS) aims to incrementally extend a semantic segmentation model to novel classes according to only a few pixel-level annotated data, while preserving its segmentation capability on previously learned base categories.
Iterative Adversarial Attack on Image-guided Story Ending Generation
Multimodal learning involves developing models that can integrate information from various sources like images and texts.
Unlearnable Examples Give a False Sense of Security: Piercing through Unexploitable Data with Learnable Examples
Unfortunately, we find UEs provide a false sense of security, because they cannot stop unauthorized users from utilizing other unprotected data to remove the protection, by turning unlearnable data into learnable again.
Read, Diagnose and Chat: Towards Explainable and Interactive LLMs-Augmented Depression Detection in Social Media
This paper proposes a new depression detection system based on LLMs that is both interpretable and interactive.
Multimodal Feature Extraction and Fusion for Emotional Reaction Intensity Estimation and Expression Classification in Videos with Transformers
In this paper, we present our advanced solutions to the two sub-challenges of Affective Behavior Analysis in the wild (ABAW) 2023: the Emotional Reaction Intensity (ERI) Estimation Challenge and Expression (Expr) Classification Challenge.
Adaptive Data-Free Quantization
Data-free quantization (DFQ) recovers the performance of quantized network (Q) without the original data, but generates the fake sample via a generator (G) by learning from full-precision network (P), which, however, is totally independent of Q, overlooking the adaptability of the knowledge from generated samples, i. e., informative or not to the learning process of Q, resulting into the overflow of generalization error.
Contrastive Video Question Answering via Video Graph Transformer
CoVGT's uniqueness and superiority are three-fold: 1) It proposes a dynamic graph transformer module which encodes video by explicitly capturing the visual objects, their relations and dynamics, for complex spatio-temporal reasoning.
Ranked #34 on
Video Question Answering
on NExT-QA
(using extra training data)
Rethinking Data-Free Quantization as a Zero-Sum Game
how to generate the samples with desirable adaptability to benefit the quantized network?
Improving Recommendation Fairness via Data Augmentation
We augment imbalanced training data towards balanced data distribution to improve fairness.
LipFormer: Learning to Lipread Unseen Speakers based on Visual-Landmark Transformers
However, generalizing these methods to unseen speakers incurs catastrophic performance degradation due to the limited number of speakers in training bank and the evident visual variations caused by the shape/color of lips for different speakers.
3D Human Pose Estimation With Spatio-Temporal Criss-Cross Attention
On this basis, we devise STCFormer by stacking multiple STC blocks and further integrate a new Structure-enhanced Positional Embedding (SPE) into STCFormer to take the structure of human body into consideration.
Ranked #8 on
3D Human Pose Estimation
on MPI-INF-3DHP
Global Temporal Difference Network for Action Recognition
In the global aggregation module, the global prior knowledge is learned by aggregating the visual feature sequence of video into a global vector.
Ranked #63 on
Action Recognition
on Something-Something V2
Stereo Image Rain Removal via Dual-View Mutual Attention
Stereo images, containing left and right view images with disparity, are utilized in solving low-vision tasks recently, e. g., rain removal and super-resolution.
Decoupled Cross-Scale Cross-View Interaction for Stereo Image Enhancement in The Dark
Specifically, we present a decoupled interaction module (DIM) that aims for sufficient dual-view information interaction.
MEGCF: Multimodal Entity Graph Collaborative Filtering for Personalized Recommendation
This paper aims at solving the mismatch problem between MFE and UIM, so as to generate high-quality embedding representations and better model multimodal user preferences.
Joint Multi-grained Popularity-aware Graph Convolution Collaborative Filtering for Recommendation
In this work, we propose to model multi-grained popularity features and jointly learn them together with high-order connectivity, to match the differentiation of user preferences exhibited in popularity features.
Switchable Online Knowledge Distillation
Instead of focusing on the accuracy gap at test phase by the existing arts, the core idea of SwitOKD is to adaptively calibrate the gap at training phase, namely distillation gap, via a switching strategy between two modes -- expert mode (pause the teacher while keep the student learning) and learning mode (restart the teacher).
Emotion Separation and Recognition from a Facial Expression by Generating the Poker Face with Vision Transformers
PF-ViT aims to separate and recognize the disturbance-agnostic emotion from a static facial image via generating its corresponding poker face, without the need for paired images.
Ranked #7 on
Facial Expression Recognition (FER)
on FER+
Towards Feature Distribution Alignment and Diversity Enhancement for Data-Free Quantization
To obtain high inter-class separability of semantic features, we cluster and align the feature distribution statistics to imitate the distribution of real data, so that the performance degradation is alleviated.
A Review-aware Graph Contrastive Learning Framework for Recommendation
Second, while most current models suffer from limited user behaviors, can we exploit the unique self-supervised signals in the review-aware graph to guide two recommendation components better?
FCL-GAN: A Lightweight and Real-Time Baseline for Unsupervised Blind Image Deblurring
Blind image deblurring (BID) remains a challenging and significant task.
Automatic Depression Detection via Learning and Fusing Features from Visual Cues
Depression is one of the most prevalent mental disorders, which seriously affects one's life.
Deep Color Consistent Network for Low-Light Image Enhancement
Current low-light image enhancement methods can well improve the illumination.
Vibration-based Uncertainty Estimation for Learning from Limited Supervision
We investigate the problem of estimating uncertainty for training data, so that deep neural networks can make use of the results for learning from limited supervision.
Few-shot Learning with Global Relatedness Decoupled-Distillation
To overcome these problems, we propose a new Global Relatedness Decoupled-Distillation (GRDD) method using the global category knowledge and the Relatedness Decoupled-Distillation (RDD) strategy.
Privileged Graph Distillation for Cold Start Recommendation
The teacher model is composed of a heterogeneous graph structure for warm users and items with privileged CF links.
Set2setRank: Collaborative Set to Set Ranking for Implicit Feedback based Recommendation
Despite the performance gain of these implicit feedback based models, the recommendation results are still far from satisfactory due to the sparsity of the observed item set for each user.
Few-shot Partial Multi-view Learning
The challenges of this task are twofold: (i) it is difficult to overcome the impact of data scarcity under the interference of missing views; (ii) the limited number of data exacerbates information scarcity, thus making it harder to address the view-missing issue in turn.
Fine-Grained Fashion Similarity Prediction by Attribute-Specific Embedding Learning
In this similarity paradigm, one should pay more attention to the similarity in terms of a specific design/attribute between fashion items.
Revisiting Local Descriptor for Improved Few-Shot Classification
Few-shot classification studies the problem of quickly adapting a deep learner to understanding novel classes based on few support images.
Learning Fair Representations for Recommendation: A Graph-based Perspective
For each user, this transformation is achieved under the adversarial learning of a user-centric graph, in order to obfuscate each sensitive feature between both the filtered user embedding and the sub graph structures of this user.
One-bit Supervision for Image Classification
Instead of training a model upon the accurate label of each sample, our setting requires the model to query with a predicted label of each sample and learn from the answer whether the guess is correct.
RGCF: Refined Graph Convolution Collaborative Filtering with concise and expressive embedding
In this work, we develop a new GCN-based Collaborative Filtering model, named Refined Graph convolution Collaborative Filtering(RGCF), where the construction of the embeddings of users (items) are delicately redesigned from several aspects during the aggregation on the graph.
Joint Item Recommendation and Attribute Inference: An Adaptive Graph Convolutional Network Approach
Therefore, item recommendation and attribute inference have become two main tasks in these platforms.
Learning to Transfer Graph Embeddings for Inductive Graph based Recommendation
The transfer network is designed to approximate the learned item embeddings from graph neural networks by taking each item's visual content as input, in order to tackle the new segment problem in the test phase.
Real-time Semantic Segmentation via Spatial-detail Guided Context Propagation
In SGCPNet, we propose the strategy of spatial-detail guided context propagation.
Memory-Augmented Relation Network for Few-Shot Learning
Metric-based few-shot learning methods concentrate on learning transferable feature embedding that generalizes well from seen categories to unseen categories under the supervision of limited number of labelled instances.
Real-world Person Re-Identification via Degradation Invariance Learning
Person re-identification (Re-ID) in real-world scenarios usually suffers from various degradation factors, e. g., low-resolution, weak illumination, blurring and adverse weather.
Creating Something from Nothing: Unsupervised Knowledge Distillation for Cross-Modal Hashing
In recent years, cross-modal hashing (CMH) has attracted increasing attentions, mainly because its potential ability of mapping contents from different modalities, especially in vision and language, into the same space, so that it becomes efficient in cross-modal data retrieval.
Estimation-Action-Reflection: Towards Deep Interaction Between Conversational and Recommender Systems
Recommender systems are embracing conversational technologies to obtain user preferences dynamically, and to overcome inherent limitations of their static models.
Revisiting Graph based Collaborative Filtering: A Linear Residual Graph Convolutional Network Approach
Second, we propose a residual network structure that is specifically designed for CF with user-item interaction modeling, which alleviates the over smoothing problem in graph convolution aggregation operation with sparse user-item interaction data.
DiffNet++: A Neural Influence and Interest Diffusion Network for Social Recommendation
Recently, we propose a preliminary work of a neural influence diffusion network (i. e., DiffNet) for social recommendation (Diffnet), which models the recursive social diffusion process to capture the higher-order relationships for each user.
Diversifying Inference Path Selection: Moving-Mobile-Network for Landmark Recognition
We intuitively find that M$^2$Net can essentially promote the diversity of the inference path (selected blocks subset) selection, so as to enhance the recognition accuracy.
MMGCN: Multi-modal Graph Convolution Network for Personalized Recommendation of Micro-video
Existing works on multimedia recommendation largely exploit multi-modal contents to enrich item representations, while less effort is made to leverage information interchange between users and items to enhance user representations and further capture user's fine-grained preferences on different modalities.
Ranked #1 on
Multi-Media Recommendation
on MovieLens 10M
A Coarse-to-Fine Multi-stream Hybrid Deraining Network for Single Image Deraining
To obtain the negative rain streaks during training process more accurately, we present a new module named dual path residual dense block, i. e., Residual path and Dense path.
Robust Subspace Discovery by Block-diagonal Adaptive Locality-constrained Representation
Leveraging on the Frobenius-norm based latent low-rank representation model, rBDLR jointly learns the coding coefficients and salient features, and improves the results by enhancing the robustness to outliers and errors in given data, preserving local information of salient features adaptively and ensuring the block-diagonal structures of the coefficients.
Joint Subspace Recovery and Enhanced Locality Driven Robust Flexible Discriminative Dictionary Learning
We propose a joint subspace recovery and enhanced locality based robust flexible label consistent dictionary learning method called Robust Flexible Discriminative Dictionary Learning (RFDDL).
Learning to Compose and Reason with Language Tree Structures for Visual Grounding
Grounding natural language in images, such as localizing "the black dog on the left of the tree", is one of the core problems in artificial intelligence, as it needs to comprehend the fine-grained and compositional language space.
Personalized Multimedia Item and Key Frame Recommendation
We argue that the key challenge of this problem lies in discovering users' visual profiles for key frame recommendation, as most recommendation models would fail without any users' fine-grained image behavior.
Online Filter Clustering and Pruning for Efficient Convnets
Pruning filters is an effective method for accelerating deep neural networks (DNNs), but most existing approaches prune filters on a pre-trained network directly which limits in acceleration.
A Neural Influence Diffusion Model for Social Recommendation
The key idea of our proposed model is that we design a layer-wise influence propagation structure to model how users' latent embeddings evolve as the social diffusion process continues.
Cross-Entropy Adversarial View Adaptation for Person Re-identification
The main contribution is to learn coupled asymmetric mappings regarding view characteristics which are adversarially trained to address the view discrepancy by optimising the cross-entropy view confusion objective.
Deep Item-based Collaborative Filtering for Top-N Recommendation
In this work, we propose a more expressive ICF solution by accounting for the nonlinear and higher-order relationship among items.
Fast Matrix Factorization with Non-Uniform Weights on Missing Data
This poses an imbalanced learning problem, since the scale of missing entries is usually much larger than that of observed entries, but they cannot be ignored due to the valuable negative signal.
SocialGCN: An Efficient Graph Convolutional Network based Model for Social Recommendation
Based on a classical CF model, the key idea of our proposed model is that we borrow the strengths of GCNs to capture how users' preferences are influenced by the social diffusion process in social networks.
A Hierarchical Attention Model for Social Contextual Image Recommendation
After that, we design a hierarchical attention network that naturally mirrors the hierarchical relationship (elements in each aspects level, and the aspect level) of users' latent interests with the identified key aspects.
Interleaved Structured Sparse Convolutional Neural Networks
In this paper, we study the problem of designing efficient convolutional neural network architectures with the interest in eliminating the redundancy in convolution kernels.
Multi-Cue Correlation Filters for Robust Visual Tracking
By combining different types of features, our approach constructs multiple experts through Discriminative Correlation Filter (DCF) and each of them tracks the target independently.
Movie Question Answering: Remembering the Textual Cues for Layered Visual Contents
Movies provide us with a mass of visual content as well as attracting stories.
IGCV$2$: Interleaved Structured Sparse Convolutional Neural Networks
In this paper, we study the problem of designing efficient convolutional neural network architectures with the interest in eliminating the redundancy in convolution kernels.
Self-Supervised Video Hashing with Hierarchical Binary Auto-encoder
Existing video hash functions are built on three isolated stages: frame pooling, relaxed learning, and binarization, which have not adequately explored the temporal order of video frames in a joint binary optimization model, resulting in severe information loss.
Enhancing Person Re-identification in a Self-trained Subspace
To address this problem, in this paper, we propose a self-trained subspace learning paradigm for person re-ID which effectively utilizes both labeled and unlabeled data to learn a discriminative subspace where person images across disjoint camera views can be easily matched.
Point-of-Interest Recommendations: Learning Potential Check-ins from Friends
The emergence of Location-based Social Network (LBSN) services provides a wonderful opportunity to build personalized Point-of-Interest (POI) recommender systems.
Cascaded Interactional Targeting Network for Egocentric Video Analysis
Firstly, a novel EM-like learning framework is proposed to train the pixel-level deep convolutional neural network (DCNN) by seamlessly integrating weakly supervised data (i. e., massive bounding box annotations) with a small set of strongly supervised data (i. e., fully annotated hand segmentation maps) to achieve state-of-the-art hand segmentation performance.
Pooling the Convolutional Layers in Deep ConvNets for Action Recognition
It achieves the accuracy of 93. 78\% on UCF101 which is the state-of-the-art and the accuracy of 65. 62\% on HMDB51 which is comparable to the state-of-the-art.
Interaction Part Mining: A Mid-Level Approach for Fine-Grained Action Recognition
Secondly, these object regions are matched and tracked across frames to form a large spatio-temporal graph based on the appearance matching and the dense motion trajectories through them.
Fine-grained Action Recognition
Human-Object Interaction Detection
+2
Crowded Scene Analysis: A Survey
Then, existing models, popular algorithms, evaluation protocols, as well as system performance are provided corresponding to different aspects of crowded scene analysis.
