Search Results for author:
Found 143 papers, 51 papers with code
Optimizing Preference Alignment with Differentiable NDCG Ranking
DRPO leverages NDCG, a widely used LTR metric, to optimize the ranking of responses within lists based on preference data, thereby enhancing ranking accuracies.
Multi-granularity Contrastive Cross-modal Collaborative Generation for End-to-End Long-term Video Question Answering
Long-term Video Question Answering (VideoQA) is a challenging vision-and-language bridging task focusing on semantic understanding of untrimmed long-term videos and diverse free-form questions, simultaneously emphasizing comprehensive cross-modal reasoning to yield precise answers.
Prompting Video-Language Foundation Models with Domain-specific Fine-grained Heuristics for Video Question Answering
Video Question Answering (VideoQA) represents a crucial intersection between video understanding and language processing, requiring both discriminative unimodal comprehension and sophisticated cross-modal interaction for accurate inference.
Search Prominence with Costly Product Returns
Search prominence may have a detrimental impact on a firm's profits in the presence of costly product returns.
Learning the Generalizable Manipulation Skills on Soft-body Tasks via Guided Self-attention Behavior Cloning Policy
Embodied AI represents a paradigm in AI research where artificial agents are situated within and interact with physical or virtual environments.
NaVIP: An Image-Centric Indoor Navigation Solution for Visually Impaired People
We benchmark on two main aspects: 1) positioning system and 2) exploration support, prioritizing training scalability and real-time inference, to validate the prospect of image-based solution towards indoor navigation.
Guided Self-attention: Find the Generalized Necessarily Distinct Vectors for Grain Size Grading
With the development of steel materials, metallographic analysis has become increasingly important.
A General Framework for Producing Interpretable Semantic Text Embeddings
Semantic text embedding is essential to many tasks in Natural Language Processing (NLP).
DDNet: Deformable Convolution and Dense FPN for Surface Defect Detection in Recycled Books
In addition to the model, we present a comprehensive dataset specifically curated for surface defect detection in recycled and recirculated books.
LMT-GP: Combined Latent Mean-Teacher and Gaussian Process for Semi-supervised Low-light Image Enhancement
Meanwhile, we use a mean-teacher-assisted Gaussian process learning strategy to establish a connection between the latent and pseudo-latent vectors obtained from the labeled and unlabeled data.
MVPbev: Multi-view Perspective Image Generation from BEV with Test-time Controllability and Generalizability
Unlike prior methods that neglect layout consistency, lack the ability to handle detailed text prompts, or are incapable of generalizing to unseen view points, MVPbev simultaneously generates cross-view consistent images of different perspective views with a two-stage design, allowing object-level control and novel view generation at test-time.
Facial Identity Anonymization via Intrinsic and Extrinsic Attention Distraction
On the other, we anonymize the visual clues (i. e. appearance and geometry structure) by distracting the extrinsic identity attention.
Learning to Discover Knowledge: A Weakly-Supervised Partial Domain Adaptation Approach
As such, the key issues of WS-PDA are: 1) how to sufficiently discover the knowledge from the noisy labeled source domain and the unlabeled target domain, and 2) how to successfully adapt the knowledge across domains.
Solution for CVPR 2024 UG2+ Challenge Track on All Weather Semantic Segmentation
In this report, we present our solution for the semantic segmentation in adverse weather, in UG2+ Challenge at CVPR 2024.
Large Language Model Assisted Adversarial Robustness Neural Architecture Search
Motivated by the potential of large language models (LLMs) as optimizers for solving combinatorial optimization problems, this paper proposes a novel LLM-assisted optimizer (LLMO) to address adversarial robustness neural architecture search (ARNAS), a specific application of combinatorial optimization.
Enhancing Emotion Recognition in Conversation through Emotional Cross-Modal Fusion and Inter-class Contrastive Learning
Previous ERC methods relied on simple connections for cross-modal fusion and ignored the information differences between modalities, resulting in the model being unable to focus on modality-specific emotional information.
Incremental Pseudo-Labeling for Black-Box Unsupervised Domain Adaptation
Black-Box unsupervised domain adaptation (BBUDA) learns knowledge only with the prediction of target data from the source model without access to the source data and source model, which attempts to alleviate concerns about the privacy and security of data.
Imp: Highly Capable Large Multimodal Models for Mobile Devices
By harnessing the capabilities of large language models (LLMs), recent large multimodal models (LMMs) have shown remarkable versatility in open-world multimodal understanding.
Ranked #62 on
Visual Question Answering
on MM-Vet
Testing for an Explosive Bubble using High-Frequency Volatility
A real-time date-stamping strategy based on the devolatized sample is proposed for the origination and conclusion dates of the explosive regime.
Unleashing the Power of Multi-Task Learning: A Comprehensive Survey Spanning Traditional, Deep, and Pretrained Foundation Model Eras
Overall, we hope this survey provides the research community with a comprehensive overview of the advancements in MTL from its inception in 1997 to the present in 2023.
Frequency-Guided Multi-Level Human Action Anomaly Detection with Normalizing Flows
We introduce the task of human action anomaly detection (HAAD), which aims to identify anomalous motions in an unsupervised manner given only the pre-determined normal category of training action samples.
Leveraging Large Language Model to Generate a Novel Metaheuristic Algorithm with CRISPE Framework
In this paper, we borrow the large language model (LLM) ChatGPT-3. 5 to automatically and quickly design a new metaheuristic algorithm (MA) with only a small amount of input.
Tackling Noisy Labels with Network Parameter Additive Decomposition
Given data with noisy labels, over-parameterized deep networks suffer overfitting mislabeled data, resulting in poor generalization.
AUD-TGN: Advancing Action Unit Detection with Temporal Convolution and GPT-2 in Wild Audiovisual Contexts
Leveraging the synergy of both audio data and visual data is essential for understanding human emotions and behaviors, especially in in-the-wild setting.
Multimodal Fusion Method with Spatiotemporal Sequences and Relationship Learning for Valence-Arousal Estimation
This paper presents our approach for the VA (Valence-Arousal) estimation task in the ABAW6 competition.
Compound Expression Recognition via Multi Model Ensemble
Compound Expression Recognition (CER) plays a crucial role in interpersonal interactions.
Efficient Feature Extraction and Late Fusion Strategy for Audiovisual Emotional Mimicry Intensity Estimation
In this paper, we present the solution to the Emotional Mimicry Intensity (EMI) Estimation challenge, which is part of 6th Affective Behavior Analysis in-the-wild (ABAW) Competition. The EMI Estimation challenge task aims to evaluate the emotional intensity of seed videos by assessing them from a set of predefined emotion categories (i. e., "Admiration", "Amusement", "Determination", "Empathic Pain", "Excitement" and "Joy").
Exploring Facial Expression Recognition through Semi-Supervised Pretraining and Temporal Modeling
Facial Expression Recognition (FER) plays a crucial role in computer vision and finds extensive applications across various fields.
Facial Expression Recognition
Facial Expression Recognition (FER)
Efficient Multiplayer Battle Game Optimizer for Adversarial Robust Neural Architecture Search
This paper introduces a novel metaheuristic algorithm, known as the efficient multiplayer battle game optimizer (EMBGO), specifically designed for addressing complex numerical optimization tasks.
A Comprehensive Survey of 3D Dense Captioning: Localizing and Describing Objects in 3D Scenes
Three-Dimensional (3D) dense captioning is an emerging vision-language bridging task that aims to generate multiple detailed and accurate descriptions for 3D scenes.
Latent Semantic Consensus For Deterministic Geometric Model Fitting
The principle of LSC is to preserve the latent semantic consensus in both data points and model hypotheses.
Mitigating Label Noise on Graph via Topological Sample Selection
Despite the success of the carefully-annotated benchmarks, the effectiveness of existing graph neural networks (GNNs) can be considerably impaired in practice when the real-world graph data is noisily labeled.
Neural auto-designer for enhanced quantum kernels
Quantum kernels hold great promise for offering computational advantages over classical learners, with the effectiveness of these kernels closely tied to the design of the quantum feature map.
EmoTalker: Emotionally Editable Talking Face Generation via Diffusion Model
In recent years, the field of talking faces generation has attracted considerable attention, with certain methods adept at generating virtual faces that convincingly imitate human expressions.
BCLNet: Bilateral Consensus Learning for Two-View Correspondence Pruning
In our approach, we design a distinctive self-attention block to capture global context and parallel process it with the established local context learning module, which enables us to simultaneously capture both local and global consensuses.
Multiplayer Battle Game-Inspired Optimizer for Complex Optimization Problems
Various popular multiplayer battle royale games share a lot of common elements.
Graph Context Transformation Learning for Progressive Correspondence Pruning
To further apply the recalibrated graph contexts to the global domain, we propose the Graph Context Guidance Transformer.
ZS-SRT: An Efficient Zero-Shot Super-Resolution Training Method for Neural Radiance Fields
This framework aims to guide the NeRF model to synthesize high-resolution novel views via single-scene internal learning rather than requiring any external high-resolution training data.
Robust Computer Vision in an Ever-Changing World: A Survey of Techniques for Tackling Distribution Shifts
As a result, it raises concerns about the overall robustness of the machine learning techniques for computer vision applications that are deployed publicly for consumers.
Multilevel Saliency-Guided Self-Supervised Learning for Image Anomaly Detection
To fully exploit saliency guidance, on each map, we select a pixel pair from the cluster with the highest centroid saliency to form a patch pair.
Image-Pointcloud Fusion based Anomaly Detection using PD-REAL Dataset
We present PD-REAL, a novel large-scale dataset for unsupervised anomaly detection (AD) in the 3D domain.
Winning Prize Comes from Losing Tickets: Improve Invariant Learning by Exploring Variant Parameters for Out-of-Distribution Generalization
By fully exploring both variant and invariant parameters, our EVIL can effectively identify a robust subnetwork to improve OOD generalization.
FlatMatch: Bridging Labeled Data and Unlabeled Data with Cross-Sharpness for Semi-Supervised Learning
Therefore, the label guidance on labeled data is hard to be propagated to unlabeled data.
Boundary Discretization and Reliable Classification Network for Temporal Action Detection
Furthermore, the reliable classification module (RCM) predicts reliable global action categories to reduce false positives.
Understanding Robust Overfitting from the Feature Generalization Perspective
Given that the only difference between adversarial and natural training lies in the inclusion of adversarial perturbations, we further hypothesize that adversarial perturbations degrade the generalization of features in natural data and verify this hypothesis through extensive experiments.
A Multi-In and Multi-Out Dendritic Neuron Model and its Optimization
Recently, a novel mathematical ANN model, known as the dendritic neuron model (DNM), has been proposed to address nonlinear problems by more accurately reflecting the structure of real neurons.
Regularly Truncated M-estimators for Learning with Noisy Labels
However, such a procedure is arguably debatable from two folds: (a) it does not consider the bad influence of noisy labels in selected small-loss examples; (b) it does not make good use of the discarded large-loss examples, which may be clean or have meaningful information for generalization.
The RoboDepth Challenge: Methods and Advancements Towards Robust Depth Estimation
In this paper, we summarize the winning solutions from the RoboDepth Challenge -- an academic competition designed to facilitate and advance robust OoD depth estimation.
A Two-stage Fine-tuning Strategy for Generalizable Manipulation Skill of Embodied AI
The advent of Chat-GPT has led to a surge of interest in Embodied AI.
Unleashing the Potential of Regularization Strategies in Learning with Noisy Labels
In recent years, research on learning with noisy labels has focused on devising novel algorithms that can achieve robustness to noisy training labels while generalizing to clean data.
Making Binary Classification from Multiple Unlabeled Datasets Almost Free of Supervision
Training a classifier exploiting a huge amount of supervised data is expensive or even prohibited in a situation, where the labeling cost is high.
Importance Sparsification for Sinkhorn Algorithm
Sinkhorn algorithm has been used pervasively to approximate the solution to optimal transport (OT) and unbalanced optimal transport (UOT) problems.
SAGE-NDVI: A Stereotype-Breaking Evaluation Metric for Remote Sensing Image Dehazing Using Satellite-to-Ground NDVI Knowledge
Image dehazing is a meaningful low-level computer vision task and can be applied to a variety of contexts.
BiomedGPT: A Generalist Vision-Language Foundation Model for Diverse Biomedical Tasks
Traditional biomedical artificial intelligence (AI) models, designed for specific tasks or modalities, often exhibit limited flexibility in real-world deployment and struggle to utilize holistic information.
Ranked #1 on
Text Summarization
on MeQSum
Multi-task Paired Masking with Alignment Modeling for Medical Vision-Language Pre-training
To address this limitation, we propose a unified Med-VLP framework based on Multi-task Paired Masking with Alignment (MPMA) to integrate the cross-modal alignment task into the joint image-text reconstruction framework to achieve more comprehensive cross-modal interaction, while a Global and Local Alignment (GLA) module is designed to assist self-supervised paradigm in obtaining semantic representations with rich domain knowledge.
ANetQA: A Large-scale Benchmark for Fine-grained Compositional Reasoning over Untrimmed Videos
A recent benchmark AGQA poses a promising paradigm to generate QA pairs automatically from pre-annotated scene graphs, enabling it to measure diverse reasoning abilities with granular control.
A Comparison of Image Denoising Methods
In this paper, to investigate the applicability of existing denoising techniques, we compare a variety of denoising methods on both synthetic and real-world datasets for different applications.
The Second Monocular Depth Estimation Challenge
This paper discusses the results for the second edition of the Monocular Depth Estimation Challenge (MDEC).
SAR2EO: A High-resolution Image Translation Framework with Denoising Enhancement
Synthetic Aperture Radar (SAR) to electro-optical (EO) image translation is a fundamental task in remote sensing that can enrich the dataset by fusing information from different sources.
Robust Generalization against Photon-Limited Corruptions via Worst-Case Sharpness Minimization
Experimentally, we simulate photon-limited corruptions using CIFAR10/100 and ImageNet30 datasets and show that SharpDRO exhibits a strong generalization ability against severe corruptions and exceeds well-known baseline methods with large performance gains.
A Dual Branch Network for Emotional Reaction Intensity Estimation
Emotional Reaction Intensity(ERI) estimation is an important task in multimodal scenarios, and has fundamental applications in medicine, safe driving and other fields.
Local Region Perception and Relationship Learning Combined with Feature Fusion for Facial Action Unit Detection
Human affective behavior analysis plays a vital role in human-computer interaction (HCI) systems.
Exploring Large-scale Unlabeled Faces to Enhance Facial Expression Recognition
Facial Expression Recognition (FER) is an important task in computer vision and has wide applications in human-computer interaction, intelligent security, emotion analysis, and other fields.
Prophet: Prompting Large Language Models with Complementary Answer Heuristics for Knowledge-based Visual Question Answering
Knowledge-based visual question answering (VQA) requires external knowledge beyond the image to answer the question.
Ranked #3 on
Visual Question Answering (VQA)
on A-OKVQA
Mosaic Representation Learning for Self-supervised Visual Pre-training
To address this problem, we propose a mosaic representation learning framework (MosRep), consisting of a new data augmentation strategy that enriches the backgrounds of each small crop and improves the quality of visual representations.
GLOW: Global Layout Aware Attacks on Object Detection
More desired attacks, to this end, should be able to fool defenses with such consistency checks.
A Comprehensive Survey on Pretrained Foundation Models: A History from BERT to ChatGPT
This study provides a comprehensive review of recent research advancements, challenges, and opportunities for PFMs in text, image, graph, as well as other data modalities.
Robust Generalization against Corruptions via Worst-Case Sharp ness Minimization
Robust generalization aims to deal with the most challenging data distributions which are rarely presented in training set and contain severe noise corruptions.
ShiftDDPMs: Exploring Conditional Diffusion Models by Shifting Diffusion Trajectories
In this paper, we propose a novel and flexible conditional diffusion model by introducing conditions into the forward process.
Combating Noisy Labels with Sample Selection by Mining High-Discrepancy Examples
As selected data have high discrepancies in probabilities, the divergence of two networks can be maintained by training on such data.
Graph Matching with Bi-level Noisy Correspondence
In this paper, we study a novel and widely existing problem in graph matching (GM), namely, Bi-level Noisy Correspondence (BNC), which refers to node-level noisy correspondence (NNC) and edge-level noisy correspondence (ENC).
Ranked #1 on
Graph Matching
on Willow Object Class
Teacher-Student Network for 3D Point Cloud Anomaly Detection with Few Normal Samples
Moreover, our method only requires very few normal samples to train the student network due to the teacher-student distillation mechanism.
Facial Expression Spotting Based on Optical Flow Features
The purpose of micro expression (ME) and macro expression (MaE) spotting task is to locate the onset and offset frames of MaE and ME clips.
Micro Expression Generation with Thin-plate Spline Motion Model and Face Parsing
We (Team: USTC-IAT-United) also compare our method with other competitors' in MEGC2022, and the expert evaluation results show that our method performs best, which verifies the effectiveness of our method.
Strength-Adaptive Adversarial Training
Firstly, applying a pre-specified perturbation budget on networks of various model capacities will yield divergent degree of robustness disparity between natural and robust accuracies, which deviates from robust network's desideratum.
Pseudo-Label Generation and Various Data Augmentation for Semi-Supervised Hyperspectral Object Detection
To overcome these difficulties, this paper first select fewer but suitable data augmentation methods to improve the accuracy of the supervised model based on the labeled training set, which is suitable for the characteristics of hyperspectral images.
Tensor-Based Multi-Modality Feature Selection and Regression for Alzheimer's Disease Diagnosis
The assessment of Alzheimer's Disease (AD) and Mild Cognitive Impairment (MCI) associated with brain changes remains a challenging task.
Bag of Tricks and a Strong Baseline for FGVC
This motivates us to explore the impact of different methods and components in fine-grained classification on FungiCLEF 2022.
High-Performance Transmission Mechanism Design of Multi-Stream Carrier Aggregation for 5G Non-Standalone Network
Multi-stream carrier aggregation is a key technology to expand bandwidth and improve the throughput of the fifth-generation wireless communication systems.
A viable framework for semi-supervised learning on realistic dataset
Semi-supervised Fine-Grained Recognition is a challenging task due to the difficulty of data imbalance, high inter-class similarity and domain mismatch.
Learning Disentangled Representations for Controllable Human Motion Prediction
Specifically, the inductive bias imposed by the extra CVAE path encourages two latent variables in two paths to respectively govern separate representations for each partial-body motion.
Understanding Robust Overfitting of Adversarial Training and Beyond
Here, we explore the causes of robust overfitting by comparing the data distribution of \emph{non-overfit} (weak adversary) and \emph{overfitted} (strong adversary) adversarial training, and observe that the distribution of the adversarial data generated by weak adversary mainly contain small-loss data.
An optimal transport approach for selecting a representative subsample with application in efficient kernel density estimation
Theoretically, we show the selected subsample can be used for efficient density estimation by deriving the convergence rate for the proposed subsample kernel density estimator.
Hilbert Curve Projection Distance for Distribution Comparison
Distribution comparison plays a central role in many machine learning tasks like data classification and generative modeling.
Efficient Approximation of Gromov-Wasserstein Distance Using Importance Sparsification
As a valid metric of metric-measure spaces, Gromov-Wasserstein (GW) distance has shown the potential for matching problems of structured data like point clouds and graphs.
Scene Clustering Based Pseudo-labeling Strategy for Multi-modal Aerial View Object Classification
Multi-modal aerial view object classification (MAVOC) in Automatic target recognition (ATR), although an important and challenging problem, has been under studied.
Multi-model Ensemble Learning Method for Human Expression Recognition
Moreover, we introduce the multi-fold ensemble method to train and ensemble several models with the same architecture but different data distributions to enhance the performance of our solution.
Bilaterally Slimmable Transformer for Elastic and Efficient Visual Question Answering
Recent advances in Transformer architectures [1] have brought remarkable improvements to visual question answering (VQA).
Hyper-relationship Learning Network for Scene Graph Generation
As a result, HLN significantly improves the performance of scene graph generation by integrating and reasoning from object interactions, relationship interactions, and transitive inference of hyper-relationships.
Do We Need to Penalize Variance of Losses for Learning with Label Noise?
Algorithms which minimize the averaged loss have been widely designed for dealing with noisy labels.
Wnet: Audio-Guided Video Object Segmentation via Wavelet-Based Cross-Modal Denoising Networks
Audio-Guided video semantic segmentation is a challenging problem in visual analysis and editing, which automatically separates foreground objects from background in a video sequence according to the referring audio expressions.
ESCNet: Gaze Target Detection With the Understanding of 3D Scenes
This paper aims to address the single image gaze target detection problem.
Mining limited data for more robust and generalized ML models
It’s more efficient to look for ways improving the data based a fixed neural network architecture.
Structure-Preserving Graph Kernel for Brain Network Classification
This paper presents a novel graph-based kernel learning approach for connectome analysis.
Uncertainty Set Prediction of Aggregated Wind Power Generation based on Bayesian LSTM and Spatio-Temporal Analysis
This paper focuses on the uncertainty set prediction of the aggregated generation of geographically distributed wind farms.
Co-variance: Tackling Noisy Labels with Sample Selection by Emphasizing High-variance Examples
The sample selection approach is popular in learning with noisy labels, which tends to select potentially clean data out of noisy data for robust training.
ROSITA: Enhancing Vision-and-Language Semantic Alignments via Cross- and Intra-modal Knowledge Integration
Nevertheless, most existing VLP approaches have not fully utilized the intrinsic knowledge within the image-text pairs, which limits the effectiveness of the learned alignments and further restricts the performance of their models.
BiSTF: Bilateral-Branch Self-Training Framework for Semi-Supervised Large-scale Fine-Grained Recognition
Semi-supervised Fine-Grained Recognition is a challenge task due to the difficulty of data imbalance, high inter-class similarity and domain mismatch.
A Weakly-Supervised Depth Estimation Network Using Attention Mechanism
It consists of a dense residual network structure, an adaptive weight channel attention (AWCA) module, a patch second non-local (PSNL) module and a soft label generation method.
The Story in Your Eyes: An Individual-difference-aware Model for Cross-person Gaze Estimation
We propose a novel method on refining cross-person gaze prediction task with eye/face images only by explicitly modelling the person-specific differences.
Improving White-box Robustness of Pre-processing Defenses via Joint Adversarial Training
However, pre-processing methods may suffer from the robustness degradation effect, in which the defense reduces rather than improving the adversarial robustness of a target model in a white-box setting.
Instance Correction for Learning with Open-set Noisy Labels
Lots of approaches, e. g., loss correction and label correction, cannot handle such open-set noisy labels well, since they need training data and test data to share the same label space, which does not hold for learning with open-set noisy labels.
Sample Selection with Uncertainty of Losses for Learning with Noisy Labels
In this way, we also give large-loss but less selected data a try; then, we can better distinguish between the cases (a) and (b) by seeing if the losses effectively decrease with the uncertainty after the try.
Ranked #26 on
Image Classification
on mini WebVision 1.0
Weakly Supervised Dense Video Captioning via Jointly Usage of Knowledge Distillation and Cross-modal Matching
This paper proposes an approach to Dense Video Captioning (DVC) without pairwise event-sentence annotation.
Removing Adversarial Noise in Class Activation Feature Space
Then, we train a denoising model to minimize the distances between the adversarial examples and the natural examples in the class activation feature space.
Comprehensive Graph-conditional Similarity Preserving Network for Unsupervised Cross-modal Hashing
Essentially, DGCPN addresses the inaccurate similarity problem by exploring and exploiting the data's intrinsic relationships in a graph.
Sufficient dimension reduction for classification using principal optimal transport direction
The proposed method, named principal optimal transport direction (POTD), estimates the basis of the SDR subspace using the principal directions of the optimal transport coupling between the data respecting different response categories.
Learning Domain-invariant Graph for Adaptive Semi-supervised Domain Adaptation with Few Labeled Source Samples
Traditional domain adaptation algorithms assume that enough labeled data, which are treated as the prior knowledge are available in the source domain.
Deep Fusion Siamese Network for Automatic Kinship Verification
Recognizing Families In the Wild (RFIW) is a challenging kinship recognition task with multiple tracks, which is based on Families in the Wild (FIW), a large-scale and comprehensive image database for automatic kinship recognition.
Retrieval of Family Members Using Siamese Neural Network
While in inference procedure, we try another similarity computing method by dropping the followed several fully connected layers and directly computing the cosine similarity of the two feature vectors.
Optimal Distributed Subsampling for Maximum Quasi-Likelihood Estimators with Massive Data
We first derive optimal Poisson subsampling probabilities in the context of quasi-likelihood estimation under the A- and L-optimality criteria.
Deep Multimodal Neural Architecture Search
Most existing works focus on a single task and design neural architectures manually, which are highly task-specific and hard to generalize to different tasks.
Ranked #19 on
Visual Question Answering (VQA)
on VQA v2 test-std
Repulsive Mixture Models of Exponential Family PCA for Clustering
To alleviate this problem, in this paper, a repulsiveness-encouraging prior is introduced among mixing components and a diversified EPCA mixture (DEPCAM) model is developed in the Bayesian framework.
Detecting Communities in Heterogeneous Multi-Relational Networks:A Message Passing based Approach
Specifically, an HMRNet is reorganized into a hierarchical structure with homogeneous networks as its layers and heterogeneous links connecting them.
Weakly-Supervised Multi-Level Attentional Reconstruction Network for Grounding Textual Queries in Videos
The task of temporally grounding textual queries in videos is to localize one video segment that semantically corresponds to the given query.
Multimodal Unified Attention Networks for Vision-and-Language Interactions
Learning an effective attention mechanism for multimodal data is important in many vision-and-language tasks that require a synergic understanding of both the visual and textual contents.
Deep Modular Co-Attention Networks for Visual Question Answering
In this paper, we propose a deep Modular Co-Attention Network (MCAN) that consists of Modular Co-Attention (MCA) layers cascaded in depth.
Ranked #7 on
Question Answering
on SQA3D
Effective degrees of freedom for surface finish defect detection and classification
One of the primary concerns of product quality control in the automotive industry is an automated detection of defects of small sizes on specular car body surfaces.
ActivityNet-QA: A Dataset for Understanding Complex Web Videos via Question Answering
It is both crucial and natural to extend this research direction to the video domain for video question answering (VideoQA).
Ranked #31 on
Video Question Answering
on ActivityNet-QA
Visual Question Answering (VQA)
Zero-Shot Video Question Answer
Multimodal Transformer with Multi-View Visual Representation for Image Captioning
Despite the success of existing studies, current methods only model the co-attention that characterizes the inter-modal interactions while neglecting the self-attention that characterizes the intra-modal interactions.
Grand Challenge of 106-Point Facial Landmark Localization
However, previous competitions on facial landmark localization (i. e., the 300-W, 300-VW and Menpo challenges) aim to predict 68-point landmarks, which are incompetent to depict the structure of facial components.
On Exploring Undetermined Relationships for Visual Relationship Detection
In this paper, we explore the beneficial effect of undetermined relationships on visual relationship detection.
Local Deep-Feature Alignment for Unsupervised Dimension Reduction
Next, we exploit an affine transformation to align the local deep features of each neighbourhood with the global features.
Single Pixel Reconstruction for One-stage Instance Segmentation
In this paper, we propose a one-stage framework, SPRNet, which performs efficient instance segmentation by introducing a single pixel reconstruction (SPR) branch to off-the-shelf one-stage detectors.
Adapting Stochastic Block Models to Power-Law Degree Distributions
Stochastic block models (SBMs) have been playing an important role in modeling clusters or community structures of network data.
Unsupervised Multi-modal Hashing for Cross-modal retrieval
With the advantage of low storage cost and high efficiency, hashing learning has received much attention in the domain of Big Data.
Cross-modal Subspace Learning via Kernel Correlation Maximization and Discriminative Structure Preserving
Our model not only considers the inter-modality correlation by maximizing the kernel correlation but also preserves the semantically structural information within each modality.
Discriminative Supervised Hashing for Cross-Modal similarity Search
With the advantage of low storage cost and high retrieval efficiency, hashing techniques have recently been an emerging topic in cross-modal similarity search.
Textually Guided Ranking Network for Attentional Image Retweet Modeling
In this paper, we study the problem of image retweet prediction in social media, which predicts the image sharing behavior that the user reposts the image tweets from their followees.
Learning Discriminative Hashing Codes for Cross-Modal Retrieval based on Multi-view Features
Many hashing methods based on a single view have been extensively studied for information retrieval.
Semi-supervised Hashing for Semi-Paired Cross-View Retrieval
Recently, hashing techniques have gained importance in large-scale retrieval tasks because of their retrieval speed.
Rethinking Diversified and Discriminative Proposal Generation for Visual Grounding
Visual grounding aims to localize an object in an image referred to by a textual query phrase.
Ranked #9 on
Phrase Grounding
on Flickr30k Entities Test
FishEyeRecNet: A Multi-Context Collaborative Deep Network for Fisheye Image Rectification
Images captured by fisheye lenses violate the pinhole camera assumption and suffer from distortions.
Deep Boosting of Diverse Experts
In this paper, a deep boosting algorithm is developed to learn more discriminative ensemble classifier by seamlessly combining a set of base deep CNNs (base experts) with diverse capabilities, e. g., these base deep CNNs are sequentially trained to recognize a set of object classes in an easy-to-hard way according to their learning complexities.
Multi-modal Face Pose Estimation with Multi-task Manifold Deep Learning
In the proposed deep learning based framework, Manifold Regularized Convolutional Layers (MRCL) improve traditional convolutional layers by learning the relationship among outputs of neurons.
Towards Realistic Face Photo-Sketch Synthesis via Composition-Aided GANs
Experimental results show that our method is capable of generating both visually comfortable and identity-preserving face sketches/photos over a wide range of challenging data.
Ranked #1 on
Face Sketch Synthesis
on CUFS
(FID metric)
Beyond Bilinear: Generalized Multimodal Factorized High-order Pooling for Visual Question Answering
For fine-grained image and question representations, a `co-attention' mechanism is developed by using a deep neural network architecture to jointly learn the attentions for both the image and the question, which can allow us to reduce the irrelevant features effectively and obtain more discriminative features for image and question representations.
Multi-modal Factorized Bilinear Pooling with Co-Attention Learning for Visual Question Answering
For multi-modal feature fusion, here we develop a Multi-modal Factorized Bilinear (MFB) pooling approach to efficiently and effectively combine multi-modal features, which results in superior performance for VQA compared with other bilinear pooling approaches.
Embedding Visual Hierarchy with Deep Networks for Large-Scale Visual Recognition
Our LMM model can provide an end-to-end approach for jointly learning: (a) the deep networks to extract more discriminative deep features for image and object class representation; (b) the tree classifier for recognizing large numbers of object classes hierarchically; and (c) the visual hierarchy adaptation for achieving more accurate indexing of large numbers of object classes hierarchically.
Deep Mixture of Diverse Experts for Large-Scale Visual Recognition
In this paper, a deep mixture of diverse experts algorithm is developed for seamlessly combining a set of base deep CNNs (convolutional neural networks) with diverse outputs (task spaces), e. g., such base deep CNNs are trained to recognize different subsets of tens of thousands of atomic object classes.
Constrained Low-Rank Learning Using Least Squares-Based Regularization
To achieve this goal, we cast the problem into a constrained rank minimization framework by adopting the least squares regularization.
Whole-brain substitute CT generation using Markov random field mixture models
The amount of data needed to train a model for s-CT generation is of the order of 100 million voxels.
Bayesian and empirical Bayesian forests
We derive ensembles of decision trees through a nonparametric Bayesian model, allowing us to view random forests as samples from a posterior distribution.
Applications
