Search Results for author:
Found 136 papers, 46 papers with code
RESIN-11: Schema-guided Event Prediction for 11 Newsworthy Scenarios
We introduce RESIN-11, a new schema-guided event extraction&prediction framework that can be applied to a large variety of newsworthy scenarios.
Coreference by Appearance: Visually Grounded Event Coreference Resolution
Event coreference resolution is critical to understand events in the growing number of online news with multiple modalities including text, video, speech, etc.
Bridging the Gap between Recognition-level Pre-training and Commonsensical Vision-language Tasks
Large-scale visual-linguistic pre-training aims to capture the generic representations from multimodal features, which are essential for downstream vision-language tasks.
Supervised Masked Knowledge Distillation for Few-Shot Transformers
Vision Transformers (ViTs) emerge to achieve impressive performance on many data-abundant computer vision tasks by capturing long-range dependencies among local features.
DiGeo: Discriminative Geometry-Aware Learning for Generalized Few-Shot Object Detection
Generalized few-shot object detection aims to achieve precise detection on both base classes with abundant annotations and novel classes with limited training data.
In Defense of Structural Symbolic Representation for Video Event-Relation Prediction
Understanding event relationships in videos requires a model to understand the underlying structures of events, i. e., the event type, the associated argument roles, and corresponding entities) along with factual knowledge needed for reasoning.
TempCLR: Temporal Alignment Representation with Contrastive Learning
For long videos, given a paragraph of description where the sentences describe different segments of the video, by matching all sentence-clip pairs, the paragraph and the full video are aligned implicitly.
Find Someone Who: Visual Commonsense Understanding in Human-Centric Grounding
We present a new commonsense task, Human-centric Commonsense Grounding, that tests the models' ability to ground individuals given the context descriptions about what happened before, and their mental/physical states or intentions.
Understanding ME? Multimodal Evaluation for Fine-grained Visual Commonsense
Visual commonsense understanding requires Vision Language (VL) models to not only understand image and text but also cross-reference in-between to fully integrate and achieve comprehension of the visual scene described.
Video Event Extraction via Tracking Visual States of Arguments
Video event extraction aims to detect salient events from a video and identify the arguments for each event as well as their semantic roles.
Weakly-Supervised Temporal Article Grounding
Specifically, given an article and a relevant video, WSAG aims to localize all ``groundable'' sentences to the video, and these sentences are possibly at different semantic scales.
Video in 10 Bits: Few-Bit VideoQA for Efficiency and Privacy
Specifically, we insert a lightweight Feature Compression Module (FeatComp) into a VideoQA model which learns to extract task-specific tiny features as little as 10 bits, which are optimal for answering certain types of questions.
Learning Visual Representation from Modality-Shared Contrastive Language-Image Pre-training
Large-scale multi-modal contrastive pre-training has demonstrated great utility to learn transferable features for a range of downstream tasks by mapping multiple modalities into a shared embedding space.
Multimodal Event Graphs: Towards Event Centric Understanding of Multimodal World
Recognizing that the visual `arrest' event is a subevent of the broader `protest' event is a challenging, yet important problem that prior work has not explored.
Towards Fast Adaptation of Pretrained Contrastive Models for Multi-channel Video-Language Retrieval
We surprisingly find that discrete text tokens coupled with a pretrained contrastive text model yields the best performance, which can even outperform state-of-the-art on the iVQA and How2QA datasets without the additional training on millions of video-language data.
Ranked #1 on
Video Question Answering
on How2QA
Language Models with Image Descriptors are Strong Few-Shot Video-Language Learners
The goal of this work is to build flexible video-language models that can generalize to various video-to-text tasks from few examples, such as domain-specific captioning, question answering, and future event prediction.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+5
Multimodal Adaptive Distillation for Leveraging Unimodal Encoders for Vision-Language Tasks
Experiments demonstrate that MAD leads to consistent gains in the low-shot, domain-shifted, and fully-supervised conditions on VCR, SNLI-VE, and VQA, achieving SOTA performance on VCR compared to other single models pretrained with image-text data.
Ranked #3 on
Visual Question Answering (VQA)
on VCR (Q-A) test
Multi-Modal Few-Shot Object Detection with Meta-Learning-Based Cross-Modal Prompting
Our approach is motivated by the high-level conceptual similarity of (metric-based) meta-learning and prompt-based learning to learn generalizable few-shot and zero-shot object detection models respectively without fine-tuning.
Fine-Grained Visual Entailment
In this paper, we propose an extension of this task, where the goal is to predict the logical relationship of fine-grained knowledge elements within a piece of text to an image.
Few-Shot Object Detection with Fully Cross-Transformer
Inspired by the recent work on vision transformers and vision-language transformers, we propose a novel Fully Cross-Transformer based model (FCT) for FSOD by incorporating cross-transformer into both the feature backbone and detection head.
Learning To Recognize Procedural Activities with Distant Supervision
In this paper we consider the problem of classifying fine-grained, multi-step activities (e. g., cooking different recipes, making disparate home improvements, creating various forms of arts and crafts) from long videos spanning up to several minutes.
CLIP-TD: CLIP Targeted Distillation for Vision-Language Tasks
Experiments demonstrate that our proposed CLIP-TD leads to exceptional gains in the low-shot (up to 51. 9%) and domain-shifted (up to 71. 3%) conditions of VCR, while simultaneously improving performance under standard fully-supervised conditions (up to 2%), achieving state-of-art performance on VCR compared to other single models that are pretrained with image-text data only.
CLIP-Event: Connecting Text and Images with Event Structures
Vision-language (V+L) pretraining models have achieved great success in supporting multimedia applications by understanding the alignments between images and text.
MuMuQA: Multimedia Multi-Hop News Question Answering via Cross-Media Knowledge Extraction and Grounding
Specifically, the task involves multi-hop questions that require reasoning over image-caption pairs to identify the grounded visual object being referred to and then predicting a span from the news body text to answer the question.
Query Adaptive Few-Shot Object Detection with Heterogeneous Graph Convolutional Networks
Few-shot object detection (FSOD) aims to detect never-seen objects using few examples.
SGEITL: Scene Graph Enhanced Image-Text Learning for Visual Commonsense Reasoning
As for pre-training, a scene-graph-aware pre-training method is proposed to leverage structure knowledge extracted in the visual scene graph.
PreViTS: Contrastive Pretraining with Video Tracking Supervision
In this work, we propose PreViTS, an SSL framework that utilizes an unsupervised tracking signal for selecting clips containing the same object, which helps better utilize temporal transformations of objects.
MA-CLIP: Towards Modality-Agnostic Contrastive Language-Image Pre-training
Large-scale multimodal contrastive pretraining has demonstrated great utility to support high performance in a range of downstream tasks by mapping multiple modalities into a shared embedding space.
Joint Multimedia Event Extraction from Video and Article
We introduce the new task of Video MultiMedia Event Extraction (Video M2E2) and propose two novel components to build the first system towards this task.
Partner-Assisted Learning for Few-Shot Image Classification
In this paper, we focus on the design of training strategy to obtain an elemental representation such that the prototype of each novel class can be estimated from a few labeled samples.
InfoSurgeon: Cross-Media Fine-grained Information Consistency Checking for Fake News Detection
To defend against machine-generated fake news, an effective mechanism is urgently needed.
RESIN: A Dockerized Schema-Guided Cross-document Cross-lingual Cross-media Information Extraction and Event Tracking System
We present a new information extraction system that can automatically construct temporal event graphs from a collection of news documents from multiple sources, multiple languages (English and Spanish for our experiment), and multiple data modalities (speech, text, image and video).
Multimodal Clustering Networks for Self-supervised Learning from Unlabeled Videos
Multimodal self-supervised learning is getting more and more attention as it allows not only to train large networks without human supervision but also to search and retrieve data across various modalities.
VATT: Transformers for Multimodal Self-Supervised Learning from Raw Video, Audio and Text
We train VATT end-to-end from scratch using multimodal contrastive losses and evaluate its performance by the downstream tasks of video action recognition, audio event classification, image classification, and text-to-video retrieval.
Ranked #7 on
Action Classification
on Moments in Time
(using extra training data)
Meta Faster R-CNN: Towards Accurate Few-Shot Object Detection with Attentive Feature Alignment
To improve the fine-grained few-shot proposal classification, we propose a novel attentive feature alignment method to address the spatial misalignment between the noisy proposals and few-shot classes, thus improving the performance of few-shot object detection.
Co-Grounding Networks with Semantic Attention for Referring Expression Comprehension in Videos
In this paper, we address the problem of referring expression comprehension in videos, which is challenging due to complex expression and scene dynamics.
VX2TEXT: End-to-End Learning of Video-Based Text Generation From Multimodal Inputs
We present \textsc{Vx2Text}, a framework for text generation from multimodal inputs consisting of video plus text, speech, or audio.
Analogical Reasoning for Visually Grounded Compositional Generalization
Children acquire language subconsciously by observing the surrounding world and listening to descriptions.
Task-Adaptive Negative Envision for Few-Shot Open-Set Recognition
In this paper, we instead propose task-adaptive negative class envision for FSOR to integrate threshold tuning into the learning process.
Open-Vocabulary Object Detection Using Captions
Weakly supervised and zero-shot learning techniques have been explored to scale object detectors to more categories with less supervision, but they have not been as successful and widely adopted as supervised models.
Neuro-Symbolic Representations for Video Captioning: A Case for Leveraging Inductive Biases for Vision and Language
In this paper, we propose a new model architecture for learning multi-modal neuro-symbolic representations for video captioning.
Unsupervised Vision-and-Language Pre-training Without Parallel Images and Captions
Pre-trained contextual vision-and-language (V&L) models have achieved impressive performance on various benchmarks.
Uncertainty-Aware Few-Shot Image Classification
In this work, we propose Uncertainty-Aware Few-Shot framework for image classification by modeling uncertainty of the similarities of query-support pairs and performing uncertainty-aware optimization.
Ref-NMS: Breaking Proposal Bottlenecks in Two-Stage Referring Expression Grounding
The prevailing framework for solving referring expression grounding is based on a two-stage process: 1) detecting proposals with an object detector and 2) grounding the referent to one of the proposals.
Analogical Reasoning for Visually Grounded Language Acquisition
Children acquire language subconsciously by observing the surrounding world and listening to descriptions.
GAIA: A Fine-grained Multimedia Knowledge Extraction System
We present the first comprehensive, open source multimedia knowledge extraction system that takes a massive stream of unstructured, heterogeneous multimedia data from various sources and languages as input, and creates a coherent, structured knowledge base, indexing entities, relations, and events, following a rich, fine-grained ontology.
COVID-19 Literature Knowledge Graph Construction and Drug Repurposing Report Generation
To combat COVID-19, both clinicians and scientists need to digest vast amounts of relevant biomedical knowledge in scientific literature to understand the disease mechanism and related biological functions.
Learning Visual Commonsense for Robust Scene Graph Generation
Scene graph generation models understand the scene through object and predicate recognition, but are prone to mistakes due to the challenges of perception in the wild.
Beyond Triplet Loss: Meta Prototypical N-tuple Loss for Person Re-identification
There is a lack of loss design which enables the joint optimization of multiple instances (of multiple classes) within per-query optimization for person ReID.
Deep Learning Guided Building Reconstruction from Satellite Imagery-derived Point Clouds
For points that belong to the same roof shape, a multi-cue, hierarchical RANSAC approach is proposed for efficient and reliable segmenting and reconstructing the building point cloud.
Cross-media Structured Common Space for Multimedia Event Extraction
We introduce a new task, MultiMedia Event Extraction (M2E2), which aims to extract events and their arguments from multimedia documents.
Cross-lingual Structure Transfer for Zero-resource Event Extraction
Most of the current cross-lingual transfer learning methods for Information Extraction (IE) have been only applied to name tagging.
Unifying Specialist Image Embedding into Universal Image Embedding
Therefore, we propose to distill the knowledge in multiple specialists into a universal embedding to solve this problem.
Training with Streaming Annotation
In this paper, we address a practical scenario where training data is released in a sequence of small-scale batches and annotation in earlier phases has lower quality than the later counterparts.
Weakly Supervised Visual Semantic Parsing
Scene Graph Generation (SGG) aims to extract entities, predicates and their semantic structure from images, enabling deep understanding of visual content, with many applications such as visual reasoning and image retrieval.
Bridging Knowledge Graphs to Generate Scene Graphs
Scene graphs are powerful representations that parse images into their abstract semantic elements, i. e., objects and their interactions, which facilitates visual comprehension and explainable reasoning.
General Partial Label Learning via Dual Bipartite Graph Autoencoder
Compared to the traditional Partial Label Learning (PLL) problem, GPLL relaxes the supervision assumption from instance-level -- a label set partially labels an instance -- to group-level: 1) a label set partially labels a group of instances, where the within-group instance-label link annotations are missing, and 2) cross-group links are allowed -- instances in a group may be partially linked to the label set from another group.
Ranked #1 on
Partial Label Learning
on MPII Movie Description
Cross-Dimensional Self-Attention for Multivariate, Geo-tagged Time Series Imputation
In order to impute the missing values, state-of-the-art methods are built on Recurrent Neural Networks (RNN), which process each time stamp sequentially, prohibiting the direct modeling of the relationship between distant time stamps.
Flow-Distilled IP Two-Stream Networks for Compressed Video Action Recognition
Recent works instead use modern compressed video modalities as an alternative to the RGB spatial stream and improve the inference speed by orders of magnitudes.
Learning to Learn Words from Visual Scenes
Language acquisition is the process of learning words from the surrounding scene.
Cross-lingual Structure Transfer for Relation and Event Extraction
The identification of complex semantic structures such as events and entity relations, already a challenging Information Extraction task, is doubly difficult from sources written in under-resourced and under-annotated languages.
Towards Train-Test Consistency for Semi-supervised Temporal Action Localization
The inconsistent strategy makes it hard to explicitly supervise the action localization model with temporal boundary annotations at training time.
Context-Gated Convolution
As such, being aware of the global context, the modulated convolution kernel of our proposed CGC can better extract representative local patterns and compose discriminative features.
Ranked #58 on
Image Classification
on ObjectNet
(using extra training data)
Detecting and Simulating Artifacts in GAN Fake Images
By using the simulated images to train a spectrum based classifier, even without seeing the fake images produced by the targeted GAN model during training, our approach achieves state-of-the-art performances on detecting fake images generated by popular GAN models such as CycleGAN.
Variational Context: Exploiting Visual and Textual Context for Grounding Referring Expressions
Specifically, our framework exploits the reciprocal relation between the referent and context, i. e., either of them influences estimation of the posterior distribution of the other, and thereby the search space of context can be greatly reduced.
CDSA: Cross-Dimensional Self-Attention for Multivariate, Geo-tagged Time Series Imputation
In order to jointly capture the self-attention across multiple dimensions, including time, location and the sensor measurements, while maintain low computational complexity, we propose a novel approach called Cross-Dimensional Self-Attention (CDSA) to process each dimension sequentially, yet in an order-independent manner.
Unsupervised Embedding Learning via Invariant and Spreading Instance Feature
This paper studies the unsupervised embedding learning problem, which requires an effective similarity measurement between samples in low-dimensional embedding space.
Unsupervised Rank-Preserving Hashing for Large-Scale Image Retrieval
We propose an unsupervised hashing method which aims to produce binary codes that preserve the ranking induced by a real-valued representation.
DMC-Net: Generating Discriminative Motion Cues for Fast Compressed Video Action Recognition
Motion has shown to be useful for video understanding, where motion is typically represented by optical flow.
Ranked #1 on
Action Recognition
on UCF-101
Counterfactual Critic Multi-Agent Training for Scene Graph Generation
CMAT is a multi-agent policy gradient method that frames objects as cooperative agents, and then directly maximizes a graph-level metric as the reward.
Multi-granularity Generator for Temporal Action Proposal
In this paper, we propose a multi-granularity generator (MGG) to perform the temporal action proposal from different granularity perspectives, relying on the video visual features equipped with the position embedding information.
Ranked #2 on
Action Recognition
on THUMOS’14
Multi-level Multimodal Common Semantic Space for Image-Phrase Grounding
Following dedicated non-linear mappings for visual features at each level, word, and sentence embeddings, we obtain multiple instantiations of our common semantic space in which comparisons between any target text and the visual content is performed with cosine similarity.
Ranked #1 on
Phrase Grounding
on Flickr30k
Low-shot Learning via Covariance-Preserving Adversarial Augmentation Networks
Deep neural networks suffer from over-fitting and catastrophic forgetting when trained with small data.
Incorporating Background Knowledge into Video Description Generation
We develop an approach that uses video meta-data to retrieve topically related news documents for a video and extracts the events and named entities from these documents.
Heated-Up Softmax Embedding
Metric learning aims at learning a distance which is consistent with the semantic meaning of the samples.
AutoLoc: Weakly-supervised Temporal Action Localization in Untrimmed Videos
In this paper, we first develop a novel weakly-supervised TAL framework called AutoLoc to directly predict the temporal boundary of each action instance.
Ranked #14 on
Weakly Supervised Action Localization
on ActivityNet-1.2
Weakly Supervised Action Localization
Weakly-supervised Temporal Action Localization
+1
Multimodal Social Media Analysis for Gang Violence Prevention
In this paper we partnered computer scientists with social work researchers, who have domain expertise in gang violence, to analyze how public tweets with images posted by youth who mention gang associations on Twitter can be leveraged to automatically detect psychosocial factors and conditions that could potentially assist social workers and violence outreach workers in prevention and early intervention programs.
AutoLoc: Weakly-supervised Temporal Action Localization
In this paper, we first develop a novel weakly-supervised TAL framework called AutoLoc to directly predict the temporal boundary of each action instance.
Weakly-supervised Temporal Action Localization
Weakly Supervised Temporal Action Localization
Entity-aware Image Caption Generation
Current image captioning approaches generate descriptions which lack specific information, such as named entities that are involved in the images.
Online Detection of Action Start in Untrimmed, Streaming Videos
We aim to tackle a novel task in action detection - Online Detection of Action Start (ODAS) in untrimmed, streaming videos.
Grounding Referring Expressions in Images by Variational Context
This is a general yet challenging vision-language task since it does not only require the localization of objects, but also the multimodal comprehension of context --- visual attributes (e. g., "largest", "baby") and relationships (e. g., "behind") that help to distinguish the referent from other objects, especially those of the same category.
Zero-Shot Visual Recognition using Semantics-Preserving Adversarial Embedding Networks
We propose a novel framework called Semantics-Preserving Adversarial Embedding Network (SP-AEN) for zero-shot visual recognition (ZSL), where test images and their classes are both unseen during training.
Multi-Modal Multi-Scale Deep Learning for Large-Scale Image Annotation
In this paper, we address two main issues in large-scale image annotation: 1) how to learn a rich feature representation suitable for predicting a diverse set of visual concepts ranging from object, scene to abstract concept; 2) how to annotate an image with the optimal number of class labels.
Skip RNN: Learning to Skip State Updates in Recurrent Neural Networks
We introduce the Skip RNN model which extends existing RNN models by learning to skip state updates and shortens the effective size of the computational graph.
Learning Spread-out Local Feature Descriptors
We propose a simple, yet powerful regularization technique that can be used to significantly improve both the pairwise and triplet losses in learning local feature descriptors.
More cat than cute? Interpretable Prediction of Adjective-Noun Pairs
This work aims at disentangling the contributions of the `adjectives' and `nouns' in the visual prediction of ANPs.
ConvNet Architecture Search for Spatiotemporal Feature Learning
Learning image representations with ConvNets by pre-training on ImageNet has proven useful across many visual understanding tasks including object detection, semantic segmentation, and image captioning.
Ranked #68 on
Action Recognition
on HMDB-51
PPR-FCN: Weakly Supervised Visual Relation Detection via Parallel Pairwise R-FCN
We aim to tackle a novel vision task called Weakly Supervised Visual Relation Detection (WSVRD) to detect "subject-predicate-object" relations in an image with object relation groundtruths available only at the image level.
Localizing Actions from Video Labels and Pseudo-Annotations
The goal of this paper is to determine the spatio-temporal location of actions in video.
Learning discriminative and transformation covariant local feature detectors.
Specifically, we extend the covariant constraint proposed by Lenc and Vedaldi [8] by defining the concepts of “standard patch” and “canonical feature” and leverage these to train a novel robust covariant detector.
Learning Discriminative and Transformation Covariant Local Feature Detectors
Specifically, we extend the covariant constraint proposed by Lenc and Vedaldi by defining the concepts of "standard patch" and "canonical feature" and leverage these to train a novel robust covariant detector.
Modeling Multimodal Clues in a Hybrid Deep Learning Framework for Video Classification
More specifically, we utilize three Convolutional Neural Networks (CNNs) operating on appearance, motion and audio signals to extract their corresponding features.
PatternNet: Visual Pattern Mining with Deep Neural Network
In this paper, we study the problem of visual pattern mining and propose a novel deep neural network architecture called PatternNet for discovering these patterns that are both discriminative and representative.
CDC: Convolutional-De-Convolutional Networks for Precise Temporal Action Localization in Untrimmed Videos
Temporal action localization is an important yet challenging problem.
Ranked #20 on
Temporal Action Localization
on THUMOS’14
(mAP IOU@0.6 metric)
Visual Translation Embedding Network for Visual Relation Detection
To the best of our knowledge, VTransE is the first end-to-end relation detection network.
Model-Driven Feed-Forward Prediction for Manipulation of Deformable Objects
A fully featured 3D model of the garment is constructed in real-time and volumetric features are then used to obtain the most similar model in the database to predict the object category and pose.
Deep Image Set Hashing
Learning-based hashing is often used in large scale image retrieval as they provide a compact representation of each sample and the Hamming distance can be used to efficiently compare two samples.
Multilingual Visual Sentiment Concept Matching
The impact of culture in visual emotion perception has recently captured the attention of multimedia research.
Interactive Segmentation on RGBD Images via Cue Selection
While these methods achieve better results than color-based methods, they are still limited in either using depth as an additional color channel or simply combining depth with color in a linear way.
Going Deeper for Multilingual Visual Sentiment Detection
In the original MVSO release, adjective-noun pair (ANP) detectors were trained for the six languages using an AlexNet-styled architecture by fine-tuning from DeepSentiBank.
EventNet Version 1.1 Technical Report
Finally, EventNet version 1. 1 contains 67, 641 videos, 500 events, and 5, 028 event-specific concepts.
Generic Instance Search and Re-identification from One Example via Attributes and Categories
Searching among instances from the same category as the query, the category-specific attributes outperform existing approaches by a large margin on shoes and cars and perform on par with the state-of-the-art on buildings.
Deep Cross Residual Learning for Multitask Visual Recognition
We propose a novel extension of residual learning for deep networks that enables intuitive learning across multiple related tasks using cross-connections called cross-residuals.
Temporal Action Localization in Untrimmed Videos via Multi-stage CNNs
To address this challenging issue, we exploit the effectiveness of deep networks in temporal action localization via three segment-based 3D ConvNets: (1) a proposal network identifies candidate segments in a long video that may contain actions; (2) a classification network learns one-vs-all action classification model to serve as initialization for the localization network; and (3) a localization network fine-tunes on the learned classification network to localize each action instance.
Ranked #1 on
Temporal Action Localization
on MEXaction2
Event Specific Multimodal Pattern Mining with Image-Caption Pairs
In this paper we describe a novel framework and algorithms for discovering image patch patterns from a large corpus of weakly supervised image-caption pairs generated from news events.
On Binary Embedding using Circulant Matrices
To address this problem, we propose Circulant Binary Embedding (CBE) which generates binary codes by projecting the data with a circulant matrix.
Learning to Hash for Indexing Big Data - A Survey
Such learning to hash methods exploit information such as data distributions or class labels when optimizing the hash codes or functions.
Visual Affect Around the World: A Large-scale Multilingual Visual Sentiment Ontology
Our work expressly focuses on the uniqueness of culture and language in relation to human affect, specifically sentiment and emotion semantics, and how they manifest in social multimedia.
EventNet: A Large Scale Structured Concept Library for Complex Event Detection in Video
Extensive experiments over the zero-shot event retrieval task when no training samples are available show that the EventNet concept library consistently and significantly outperforms the state-of-the-art (such as the 20K ImageNet concepts trained with CNN) by a large margin up to 207%.
Attributes and Categories for Generic Instance Search From One Example
This paper aims for generic instance search from one example where the instance can be an arbitrary 3D object like shoes, not just near-planar and one-sided instances like buildings and logos.
New Insights Into Laplacian Similarity Search
Graph-based computer vision applications rely critically on similarity metrics which compute the pairwise similarity between any pair of vertices on graphs.
Compact Nonlinear Maps and Circulant Extensions
This leads to much more compact maps without hurting the performance.
Deep Transfer Network: Unsupervised Domain Adaptation
In this paper, we propose a new domain adaptation framework named Deep Transfer Network (DTN), where the highly flexible deep neural networks are used to implement such a distribution matching process.
Exploiting Feature and Class Relationships in Video Categorization with Regularized Deep Neural Networks
In this paper, we study the challenging problem of categorizing videos according to high-level semantics such as the existence of a particular human action or a complex event.
An exploration of parameter redundancy in deep networks with circulant projections
We explore the redundancy of parameters in deep neural networks by replacing the conventional linear projection in fully-connected layers with the circulant projection.
Discrete Graph Hashing
Hashing has emerged as a popular technique for fast nearest neighbor search in gigantic databases.
DeepSentiBank: Visual Sentiment Concept Classification with Deep Convolutional Neural Networks
Nearly one million Flickr images tagged with these ANPs are downloaded to train the classifiers of the concepts.
Hash-SVM: Scalable Kernel Machines for Large-Scale Visual Classification
This paper presents a novel algorithm which uses compact hash bits to greatly improve the efficiency of non-linear kernel SVM in very large scale visual classification problems.
Locally Linear Hashing for Extracting Non-Linear Manifolds
Previous efforts in hashing intend to preserve data variance or pairwise affinity, but neither is adequate in capturing the manifold structures hidden in most visual data.
Video Event Detection by Inferring Temporal Instance Labels
To solve this problem, we propose a large-margin formulation which treats the instance labels as hidden latent variables, and simultaneously infers the instance labels as well as the instance-level classification model.
Circulant Binary Embedding
To address this problem, we propose Circulant Binary Embedding (CBE) which generates binary codes by projecting the data with a circulant matrix.
Building A Large Concept Bank for Representing Events in Video
In this paper, we propose to build Concept Bank, the largest concept library consisting of 4, 876 concepts specifically designed to cover 631 real-world events.
On Learning from Label Proportions
Learning from Label Proportions (LLP) is a learning setting, where the training data is provided in groups, or "bags", and only the proportion of each class in each bag is known.
Analyzing the Harmonic Structure in Graph-Based Learning
We show that either explicitly or implicitly, various well-known graph-based models exhibit a common significant \emph{harmonic} structure in its target function -- the value of a vertex is approximately the weighted average of the values of its adjacent neighbors.
$\propto$SVM for learning with label proportions
We study the problem of learning with label proportions in which the training data is provided in groups and only the proportion of each class in each group is known.
Designing Category-Level Attributes for Discriminative Visual Recognition
In this paper, we propose a novel formulation to automatically design discriminative "category-level attributes", which can be efficiently encoded by a compact category-attribute matrix.
Sample-Specific Late Fusion for Visual Category Recognition
However, the existing methods generally use a fixed fusion weight for all the scores of a classifier, and thus fail to optimally determine the fusion weight for the individual samples.
A Bayesian Approach to Multimodal Visual Dictionary Learning
nary learning methods rely on image descriptors alone or together with class labels.
Hash Bit Selection: A Unified Solution for Selection Problems in Hashing
We represent the bit pool as a vertx- and edge-weighted graph with the candidate bits as vertices.
Label Propagation from ImageNet to 3D Point Clouds
Our approach shows further major gains in accuracy when the training data from the target scenes is used, outperforming state-ofthe-art approaches with far better efficiency.
Robust Object Co-detection
Object co-detection aims at simultaneous detection of objects of the same category from a pool of related images by exploiting consistent visual patterns present in candidate objects in the images.
Distributed Low-rank Subspace Segmentation
Vision problems ranging from image clustering to motion segmentation to semi-supervised learning can naturally be framed as subspace segmentation problems, in which one aims to recover multiple low-dimensional subspaces from noisy and corrupted input data.
Learning with Partially Absorbing Random Walks
We prove that under proper absorption rates, a random walk starting from a set $\mathcal{S}$ of low conductance will be mostly absorbed in $\mathcal{S}$.
