Search Results for author:
Found 118 papers, 43 papers with code
Compound Rank-k Projections for Bilinear Analysis
In many real-world applications, data are represented by matrices or high-order tensors.
Semi-supervised Feature Analysis by Mining Correlations among Multiple Tasks
In this paper, we propose a novel semi-supervised feature selection framework by mining correlations among multiple tasks and apply it to different multimedia applications.
Balanced k-Means and Min-Cut Clustering
Clustering is an effective technique in data mining to generate groups that are the matter of interest.
Improved Spectral Clustering via Embedded Label Propagation
Our algorithm is built upon two advancements of the state of the art:1) label propagation, which propagates a node\'s labels to neighboring nodes according to their proximity; and 2) manifold learning, which has been widely used in its capacity to leverage the manifold structure of data points.
A Convex Formulation for Spectral Shrunk Clustering
Thus, applying manifold information obtained from the original space to the clustering process in a low-dimensional subspace is prone to inferior performance.
A Convex Sparse PCA for Feature Analysis
In addition, based on the sparse model used in CSPCA, an optimal weight is assigned to each of the original feature, which in turn provides the output with good interpretability.
Unsupervised Feature Analysis with Class Margin Optimization
In this paper, we propose an unsupervised feature selection method seeking a feature coefficient matrix to select the most distinctive features.
Dynamic Concept Composition for Zero-Example Event Detection
In this paper, we focus on automatically detecting events in unconstrained videos without the use of any visual training exemplars.
They Are Not Equally Reliable: Semantic Event Search Using Differentiated Concept Classifiers
Complex event detection on unconstrained Internet videos has seen much progress in recent years.
Strategies for Searching Video Content with Text Queries or Video Examples
The large number of user-generated videos uploaded on to the Internet everyday has led to many commercial video search engines, which mainly rely on text metadata for search.
Uncovering Locally Discriminative Structure for Feature Analysis
We propose a method that utilizes both the manifold structure of data and local discriminant information.
Simple to Complex Cross-modal Learning to Rank
The heterogeneity-gap between different modalities brings a significant challenge to multimedia information retrieval.
Complex Event Detection by Identifying Reliable Shots From Untrimmed Videos
relevant) to the given event class, we formulate this task as a multi-instance learning (MIL) problem by taking each video as a bag and the video shots in each video as instances.
Reinforcement Cutting-Agent Learning for Video Object Segmentation
In this paper, we formulate this problem as a Markov Decision Process, where agents are learned to segment object regions under a deep reinforcement learning framework.
Multi-shot Person Re-identification through Set Distance with Visual Distributional Representation
In this work, we propose the idea of visual distributional representation, which interprets an image set as samples drawn from an unknown distribution in appearance feature space.
RCAA: Relational Context-Aware Agents for Person Search
In this paper, we address this problem by training relational context-aware agents which learn the actions to localize the target person from the gallery of whole scene images.
MMALFM: Explainable Recommendation by Leveraging Reviews and Images
Then the aspect importance is integrated into a novel aspect-aware latent factor model (ALFM), which learns user's and item's latent factors based on ratings.
Distributionally Robust Semi-Supervised Learning for People-Centric Sensing
Semi-supervised learning is crucial for alleviating labelling burdens in people-centric sensing.
Ensemble Teaching for Hybrid Label Propagation
More importantly, HyDEnT conducts propagation under the guidance of an ensemble of teachers.
Continual Reinforcement Learning with Diversity Exploration and Adversarial Self-Correction
We first develop an unsupervised diversity exploration method to learn task-specific skills using an unsupervised objective.
Multi-Head Attention with Diversity for Learning Grounded Multilingual Multimodal Representations
With the aim of promoting and understanding the multilingual version of image search, we leverage visual object detection and propose a model with diverse multi-head attention to learn grounded multilingual multimodal representations.
Vision-Language Navigation with Self-Supervised Auxiliary Reasoning Tasks
In this paper, we introduce Auxiliary Reasoning Navigation (AuxRN), a framework with four self-supervised auxiliary reasoning tasks to take advantage of the additional training signals derived from the semantic information.
Ranked #13 on
Vision and Language Navigation
on VLN Challenge
Blockwisely Supervised Neural Architecture Search with Knowledge Distillation
Moreover, we find that the knowledge of a network model lies not only in the network parameters but also in the network architecture.
Ranked #1 on
Neural Architecture Search
on CIFAR-100
Argus: Efficient Activity Detection System for Extended Video Analysis
We propose an Efficient Activity Detection System, Argus, for Extended Video Analysis in the surveillance scenario.
Unity Style Transfer for Person Re-Identification
To solve this problem, we propose a UnityStyle adaption method, which can smooth the style disparities within the same camera and across different cameras.
ZSTAD: Zero-Shot Temporal Activity Detection
An integral part of video analysis and surveillance is temporal activity detection, which means to simultaneously recognize and localize activities in long untrimmed videos.
Vision-Dialog Navigation by Exploring Cross-modal Memory
Benefiting from the collaborative learning of the L-mem and the V-mem, our CMN is able to explore the memory about the decision making of historical navigation actions which is for the current step.
Unsupervised Multimodal Neural Machine Translation with Pseudo Visual Pivoting
In this paper, we investigate how to utilize visual content for disambiguation and promoting latent space alignment in unsupervised MMT.
Connecting the Dots: Multivariate Time Series Forecasting with Graph Neural Networks
Modeling multivariate time series has long been a subject that has attracted researchers from a diverse range of fields including economics, finance, and traffic.
Ranked #1 on
Univariate Time Series Forecasting
on Electricity
A Comprehensive Survey of Neural Architecture Search: Challenges and Solutions
Neural Architecture Search (NAS) is just such a revolutionary algorithm, and the related research work is complicated and rich.
Auxiliary Signal-Guided Knowledge Encoder-Decoder for Medical Report Generation
Firstly, the regions of primary interest to radiologists are usually located in a small area of the global image, meaning that the remainder parts of the image could be considered as irrelevant noise in the training procedure.
Melanoma Diagnosis with Spatio-Temporal Feature Learning on Sequential Dermoscopic Images
Existing studies for automated melanoma diagnosis are based on single-time point images of lesions.
Multi-view Drone-based Geo-localization via Style and Spatial Alignment
In this paper, we focus on the task of multi-view multi-source geo-localization, which serves as an important auxiliary method of GPS positioning by matching drone-view image and satellite-view image with pre-annotated GPS tag.
Accurate Bounding-box Regression with Distance-IoU Loss for Visual Tracking
Moreover, we introduce a classification part that is trained online and optimized with a Conjugate-Gradient-based strategy to guarantee real-time tracking speed.
Self-Weighted Robust LDA for Multiclass Classification with Edge Classes
In this regard, a novel self-weighted robust LDA with l21-norm based pairwise between-class distance criterion, called SWRLDA, is proposed for multi-class classification especially with edge classes.
Hierarchical Neural Architecture Search for Deep Stereo Matching
To reduce the human efforts in neural network design, Neural Architecture Search (NAS) has been applied with remarkable success to various high-level vision tasks such as classification and semantic segmentation.
Ranked #2 on
Stereo Disparity Estimation
on Scene Flow
Differentiable Neural Architecture Search in Equivalent Space with Exploration Enhancement
A probabilistic exploration enhancement method is accordingly devised to encourage intelligent exploration during the architecture search in the latent space, to avoid local optimal in architecture search.
Exploring Inter-Channel Correlation for Diversity-Preserved Knowledge Distillation
Extensive experiments on two vision tasks, including ImageNet classification and Pascal VOC segmentation, demonstrate the superiority of our ICKD, which consistently outperforms many existing methods, advancing the state-of-the-art in the fields of Knowledge Distillation.
Ranked #21 on
Knowledge Distillation
on ImageNet
UPDeT: Universal Multi-agent RL via Policy Decoupling with Transformers
Recent advances in multi-agent reinforcement learning have been largely limited in training one model from scratch for every new task.
EXPLORING VULNERABILITIES OF BERT-BASED APIS
We then demonstrate how the extracted model can be exploited to develop effective attribute inference attack to expose sensitive information of the training data.
UPDeT: Universal Multi-agent Reinforcement Learning via Policy Decoupling with Transformers
Recent advances in multi-agent reinforcement learning have been largely limited in training one model from scratch for every new task.
A Comprehensive Survey of Scene Graphs: Generation and Application
For example, given an image, we want to not only detect and recognize objects in the image, but also know the relationship between objects (visual relationship detection), and generate a text description (image captioning) based on the image content.
NAS-TC: Neural Architecture Search on Temporal Convolutions for Complex Action Recognition
Accordingly, because of the automated design of its network structure, Neural architecture search (NAS) has achieved great success in the image processing field and attracted substantial research attention in recent years.
BossNAS: Exploring Hybrid CNN-transformers with Block-wisely Self-supervised Neural Architecture Search
In this work, we present Block-wisely Self-supervised Neural Architecture Search (BossNAS), an unsupervised NAS method that addresses the problem of inaccurate architecture rating caused by large weight-sharing space and biased supervision in previous methods.
Dynamic Slimmable Network
Here, we explore a dynamic network slimming regime, named Dynamic Slimmable Network (DS-Net), which aims to achieve good hardware-efficiency via dynamically adjusting filter numbers of networks at test time with respect to different inputs, while keeping filters stored statically and contiguously in hardware to prevent the extra burden.
SOON: Scenario Oriented Object Navigation with Graph-based Exploration
In this task, an agent is required to navigate from an arbitrary position in a 3D embodied environment to localize a target following a scene description.
Ranked #5 on
Visual Navigation
on SOON Test
Attribute-Modulated Generative Meta Learning for Zero-Shot Classification
The promising strategies for ZSL are to synthesize visual features of unseen classes conditioned on semantic side information and to incorporate meta-learning to eliminate the model's inherent bias towards seen classes.
Person Search Challenges and Solutions: A Survey
This paper surveyed the recent works on image-based and text-based person search from the perspective of challenges and solutions.
Vision-Language Navigation with Random Environmental Mixup
Then, we cross-connect the key views of different scenes to construct augmented scenes.
Ranked #38 on
Vision and Language Navigation
on VLN Challenge
iDARTS: Differentiable Architecture Search with Stochastic Implicit Gradients
A key challenge to the scalability and quality of the learned architectures is the need for differentiating through the inner-loop optimisation.
Ranked #22 on
Neural Architecture Search
on NAS-Bench-201, CIFAR-10
Deep Learning for Embodied Vision Navigation: A Survey
A navigation agent is supposed to have various intelligent skills, such as visual perceiving, mapping, planning, exploring and reasoning, etc.
Legislator Representation Learning with Social Context and Expert Knowledge
Modeling the ideological perspectives of political actors is an essential task in computational political science with applications in many downstream tasks.
KGAP: Knowledge Graph Augmented Political Perspective Detection in News Media
Specifically, we construct a political knowledge graph to serve as domain-specific external knowledge.
FFA-IR: Towards an Explainable and Reliable Medical Report Generation Benchmark
Researchers have explored advanced methods from computer vision and natural language processing to incorporate medical domain knowledge for the generation of readable medical reports.
Unsupervised Person Re-Identification: A Systematic Survey of Challenges and Solutions
Therefore, comprehensive surveys on this topic are essential to summarise challenges and solutions to foster future research.
Semantics-Guided Contrastive Network for Zero-Shot Object detection
To address these issues, we develop a novel Semantics-Guided Contrastive Network for ZSD, named ContrastZSD, a detection framework that first brings contrastive learning mechanism into the realm of zero-shot detection.
Ranked #4 on
Zero-Shot Object Detection
on MS-COCO
Contrastive Learning
Generalized Zero-Shot Object Detection
+3
DS-Net++: Dynamic Weight Slicing for Efficient Inference in CNNs and Transformers
Dynamic networks have shown their promising capability in reducing theoretical computation complexity by adapting their architectures to the input during inference.
Reliable Shot Identification for Complex Event Detection via Visual-Semantic Embedding
In this paper, we decompose the video into several segments and intuitively model the task of complex event detection as a multiple instance learning problem by representing each video as a "bag" of segments in which each segment is referred to as an instance.
Active Learning for Deep Visual Tracking
In this paper, we propose an active learning method for deep visual tracking, which selects and annotates the unlabeled samples to train the deep CNNs model.
Dynamic Slimmable Denoising Network
Here, we present dynamic slimmable denoising network (DDS-Net), a general method to achieve good denoising quality with less computational complexity, via dynamically adjusting the channel configurations of networks at test time with respect to different noisy images.
Signature-Graph Networks
SGN constructs unique undirected graphs for each image based on the CNN feature maps.
An Entropy-guided Reinforced Partial Convolutional Network for Zero-Shot Learning
Zero-Shot Learning (ZSL) aims to transfer learned knowledge from observed classes to unseen classes via semantic correlations.
BaLeNAS: Differentiable Architecture Search via the Bayesian Learning Rule
Differentiable Architecture Search (DARTS) has received massive attention in recent years, mainly because it significantly reduces the computational cost through weight sharing and continuous relaxation.
Self-Supervised Global-Local Structure Modeling for Point Cloud Domain Adaptation With Reliable Voted Pseudo Labels
In this paper, we propose an unsupervised domain adaptation method for deep point cloud representation learning.
Diversity-boosted Generalization-Specialization Balancing for Zero-shot Learning
We further propose a differentiable dataset-level balance and update the weights in a linear annealing schedule to simulate network pruning and thus obtain the optimal structure for BSNet with dataset-level balance achieved.
Exploring Inter-Channel Correlation for Diversity-preserved KnowledgeDistillation
Extensive experiments on two vision tasks, includ-ing ImageNet classification and Pascal VOC segmentation, demonstrate the superiority of our ICKD, which consis-tently outperforms many existing methods, advancing thestate-of-the-art in the fields of Knowledge Distillation.
Voice-Face Homogeneity Tells Deepfake
To this end, a voice-face matching method is devised to measure the matching degree of these two.
Beyond Fixation: Dynamic Window Visual Transformer
Recently, a surge of interest in visual transformers is to reduce the computational cost by limiting the calculation of self-attention to a local window.
CGUA: Context-Guided and Unpaired-Assisted Weakly Supervised Person Search
On the other hand, there are numerous unpaired persons in real-world scene images.
Automated Progressive Learning for Efficient Training of Vision Transformers
First, we develop a strong manual baseline for progressive learning of ViTs, by introducing momentum growth (MoGrow) to bridge the gap brought by model growth.
Dual-AI: Dual-path Actor Interaction Learning for Group Activity Recognition
Learning spatial-temporal relation among multiple actors is crucial for group activity recognition.
PRE-NAS: Predictor-assisted Evolutionary Neural Architecture Search
This often requires a high computational overhead to evaluate a number of candidate networks from the set of all possible networks in the search space during the search.
Ranked #16 on
Neural Architecture Search
on NAS-Bench-201, CIFAR-10
Towards Explanation for Unsupervised Graph-Level Representation Learning
Due to the superior performance of Graph Neural Networks (GNNs) in various domains, there is an increasing interest in the GNN explanation problem "\emph{which fraction of the input graph is the most crucial to decide the model's decision?}"
Knowledge Distillation via the Target-aware Transformer
To this end, we propose a novel one-to-all spatial matching knowledge distillation approach.
Policy Diagnosis via Measuring Role Diversity in Cooperative Multi-agent RL
In this study, we quantify the agent's behavior difference and build its relationship with the policy performance via {\bf Role Diversity}, a metric to measure the characteristics of MARL tasks.
Cross-modal Clinical Graph Transformer for Ophthalmic Report Generation
To endow models with the capability of incorporating expert knowledge, we propose a Cross-modal clinical Graph Transformer (CGT) for ophthalmic report generation (ORG), in which clinical relation triples are injected into the visual features as prior knowledge to drive the decoding procedure.
Domain Adaptive Nuclei Instance Segmentation and Classification via Category-aware Feature Alignment and Pseudo-labelling
In this work, we propose a novel deep neural network, namely Category-Aware feature alignment and Pseudo-Labelling Network (CAPL-Net) for UDA nuclei instance segmentation and classification.
CLMFormer: Mitigating Data Redundancy to Revitalize Transformer-based Long-Term Time Series Forecasting System
To further enhance forecasting accuracy, we introduce a memory-driven decoder.
An Efficient Spatio-Temporal Pyramid Transformer for Action Detection
The task of action detection aims at deducing both the action category and localization of the start and end moment for each action instance in a long, untrimmed video.
Spatial-temporal Analysis for Automated Concrete Workability Estimation
Concrete workability measure is mostly determined based on subjective assessment of a certified assessor with visual inspections.
Prompt-driven efficient Open-set Semi-supervised Learning
In this paper, we propose a prompt-driven efficient OSSL framework, called OpenPrompt, which can propagate class information from labeled to unlabeled data with only a small number of trainable parameters.
MARLlib: A Scalable and Efficient Multi-agent Reinforcement Learning Library
A significant challenge facing researchers in the area of multi-agent reinforcement learning (MARL) pertains to the identification of a library that can offer fast and compatible development for multi-agent tasks and algorithm combinations, while obviating the need to consider compatibility issues.
Multi-agent Reinforcement Learning
reinforcement-learning
+1
ViLPAct: A Benchmark for Compositional Generalization on Multimodal Human Activities
We introduce ViLPAct, a novel vision-language benchmark for human activity planning.
PAR: Political Actor Representation Learning with Social Context and Expert Knowledge
Extensive experiments demonstrate that PAR is better at augmenting political text understanding and successfully advances the state-of-the-art in political perspective detection and roll call vote prediction.
Learning Self-Regularized Adversarial Views for Self-Supervised Vision Transformers
Despite the success, its development and application on self-supervised vision transformers have been hindered by several barriers, including the high search cost, the lack of supervision, and the unsuitable search space.
Simple Primitives with Feasibility- and Contextuality-Dependence for Open-World Compositional Zero-shot Learning
We design Semantic Attention (SA) and generative Knowledge Disentanglement (KD) to learn the dependence of feasibility and contextuality, respectively.
3D-TOGO: Towards Text-Guided Cross-Category 3D Object Generation
Here, we make the first attempt to achieve generic text-guided cross-category 3D object generation via a new 3D-TOGO model, which integrates a text-to-views generation module and a views-to-3D generation module.
HTML: Hybrid Temporal-scale Multimodal Learning Framework for Referring Video Object Segmentation
To tackle this problem, we propose a concise Hybrid Temporal-scale Multimodal Learning (HTML) framework, which can effectively align lingual and visual features to discover core object semantics in the video, by learning multimodal interaction hierarchically from different temporal scales.
Ranked #6 on
Referring Video Object Segmentation
on Refer-YouTube-VOS
(using extra training data)
ViewCo: Discovering Text-Supervised Segmentation Masks via Multi-View Semantic Consistency
Specifically, we first propose text-to-views consistency modeling to learn correspondence for multiple views of the same input image.
Guided Image-to-Image Translation by Discriminator-Generator Communication
The goal of Image-to-image (I2I) translation is to transfer an image from a source domain to a target domain, which has recently drawn increasing attention.
Generative Adversarial Network
Image-to-Image Translation
+1
Dynamic Graph Enhanced Contrastive Learning for Chest X-ray Report Generation
To address the limitation, we propose a knowledge graph with Dynamic structure and nodes to facilitate medical report generation with Contrastive Learning, named DCL.
A Benchmark for Cycling Close Pass Near Miss Event Detection from Video Streams
Instance-level detection aims to detect which vehicle in the scene gives rise to a close pass near miss.
Towards Medical Artificial General Intelligence via Knowledge-Enhanced Multimodal Pretraining
In MOTOR, we combine two kinds of basic medical knowledge, i. e., general and specific knowledge, in a complementary manner to boost the general pretraining process.
Toward the Automated Construction of Probabilistic Knowledge Graphs for the Maritime Domain
In order to exploit the potentially useful and rich information from such sources, it is necessary to extract not only the relevant entities and concepts but also their semantic relations, together with the uncertainty associated with the extracted knowledge (i. e., in the form of probabilistic knowledge graphs).
Maximum Entropy Heterogeneous-Agent Reinforcement Learning
We embed cooperative MARL problems into probabilistic graphical models, from which we derive the maximum entropy (MaxEnt) objective for MARL.
Multi-agent Reinforcement Learning
reinforcement-learning
+1
Two-stream Multi-level Dynamic Point Transformer for Two-person Interaction Recognition
Subsequently, a frame features learning module and a two-stream multi-level feature aggregation module extract global and partial features from the sampled frames, effectively representing the local-region spatial information, appearance information, and motion information related to the interactions.
FULLER: Unified Multi-modality Multi-task 3D Perception via Multi-level Gradient Calibration
Specifically, the gradients, produced by the task heads and used to update the shared backbone, will be calibrated at the backbone's last layer to alleviate the task conflict.
SSMG: Spatial-Semantic Map Guided Diffusion Model for Free-form Layout-to-Image Generation
Despite significant progress in Text-to-Image (T2I) generative models, even lengthy and complex text descriptions still struggle to convey detailed controls.
ProAgent: Building Proactive Cooperative Agents with Large Language Models
Building agents with adaptive behavior in cooperative tasks stands as a paramount goal in the realm of multi-agent systems.
PSDiff: Diffusion Model for Person Search with Iterative and Collaborative Refinement
Dominant Person Search methods aim to localize and recognize query persons in a unified network, which jointly optimizes two sub-tasks, \ie, pedestrian detection and Re-IDentification (ReID).
No Token Left Behind: Efficient Vision Transformer via Dynamic Token Idling
By allowing the idle tokens to be re-selected in the following layers, IdleViT mitigates the negative impact of improper pruning in the early stages.
Mask Propagation for Efficient Video Semantic Segmentation
By reusing predictions from key frames, we circumvent the need to process a large volume of video frames individually with resource-intensive segmentors, alleviating temporal redundancy and significantly reducing computational costs.
Disentangled Representation Learning with Transmitted Information Bottleneck
Encoding only the task-related information from the raw data, \ie, disentangled representation learning, can greatly contribute to the robustness and generalizability of models.
Generating Action-conditioned Prompts for Open-vocabulary Video Action Recognition
To realize this, we innovatively blend video models with Large Language Models (LLMs) to devise Action-conditioned Prompts.
Shot2Story20K: A New Benchmark for Comprehensive Understanding of Multi-shot Videos
A human need to capture both the event in every shot and associate them together to understand the story behind it.
Ranked #1 on
video narration captioning
on Shot2Story20K
Video Recognition in Portrait Mode
While existing datasets mainly comprise landscape mode videos, our paper seeks to introduce portrait mode videos to the research community and highlight the unique challenges associated with this video format.
Noisy Correspondence Learning with Self-Reinforcing Errors Mitigation
Cross-modal retrieval relies on well-matched large-scale datasets that are laborious in practice.
MatchNAS: Optimizing Edge AI in Sparse-Label Data Contexts via Automating Deep Neural Network Porting for Mobile Deployment
In this paper, we propose MatchNAS, a novel scheme for porting DNNs to mobile devices.
DNA Family: Boosting Weight-Sharing NAS with Block-Wise Supervisions
Addressing this problem, we modularize a large search space into blocks with small search spaces and develop a family of models with the distilling neural architecture (DNA) techniques.
SWAP-NAS: Sample-Wise Activation Patterns for Ultra-fast NAS
The SWAP-Score is strongly correlated with ground-truth performance across various search spaces and tasks, outperforming 15 existing training-free metrics on NAS-Bench-101/201/301 and TransNAS-Bench-101.
NavCoT: Boosting LLM-Based Vision-and-Language Navigation via Learning Disentangled Reasoning
Vision-and-Language Navigation (VLN), as a crucial research problem of Embodied AI, requires an embodied agent to navigate through complex 3D environments following natural language instructions.
Unified Static and Dynamic Network: Efficient Temporal Filtering for Video Grounding
Inspired by the activity-silent and persistent activity mechanisms in human visual perception biology, we design a Unified Static and Dynamic Network (UniSDNet), to learn the semantic association between the video and text/audio queries in a cross-modal environment for efficient video grounding.
Self-Supervised Multi-Frame Neural Scene Flow
Neural Scene Flow Prior (NSFP) and Fast Neural Scene Flow (FNSF) have shown remarkable adaptability in the context of large out-of-distribution autonomous driving.
LongVLM: Efficient Long Video Understanding via Large Language Models
In this way, we encode video representations that incorporate both local and global information, enabling the LLM to generate comprehensive responses for long-term videos.
MLP Can Be A Good Transformer Learner
We identify that regarding the attention layer in bottom blocks, their subsequent MLP layers, i. e. two feed-forward layers, can elicit the same entropy quantity.
Mining Inter-Video Proposal Relations for Video Object Detection
Recent studies have shown that, context aggregating information from proposals in different frames can clearly enhance the performance of video object detection.
Ranked #11 on
Video Object Detection
on ImageNet VID
