Search Results for author:
Found 62 papers, 34 papers with code
Self-Paced Learning with Diversity
Self-paced learning (SPL) is a recently proposed learning regime inspired by the learning process of humans and animals that gradually incorporates easy to more complex samples into training.
What Objective Does Self-paced Learning Indeed Optimize?
Self-paced learning (SPL) is a recently raised methodology designed through simulating the learning principle of humans/animals.
A Self-Paced Multiple-Instance Learning Framework for Co-Saliency Detection
As an interesting and emerging topic, co-saliency detection aims at simultaneously extracting common salient objects in a group of images.
Strategies for Searching Video Content with Text Queries or Video Examples
The large number of user-generated videos uploaded on to the Internet everyday has led to many commercial video search engines, which mainly rely on text metadata for search.
Exploiting Multi-modal Curriculum in Noisy Web Data for Large-scale Concept Learning
Learning video concept detectors automatically from the big but noisy web data with no additional manual annotations is a novel but challenging area in the multimedia and the machine learning community.
Self-paced Learning for Weakly Supervised Evidence Discovery in Multimedia Event Search
Multimedia event detection has been receiving increasing attention in recent years.
Video Representation Learning and Latent Concept Mining for Large-scale Multi-label Video Classification
We report on CMU Informedia Lab's system used in Google's YouTube 8 Million Video Understanding Challenge.
MemexQA: Visual Memex Question Answering
This paper proposes a new task, MemexQA: given a collection of photos or videos from a user, the goal is to automatically answer questions that help users recover their memory about events captured in the collection.
Graph Distillation for Action Detection with Privileged Modalities
We propose a technique that tackles action detection in multimodal videos under a realistic and challenging condition in which only limited training data and partially observed modalities are available.
MentorNet: Learning Data-Driven Curriculum for Very Deep Neural Networks on Corrupted Labels
Recent deep networks are capable of memorizing the entire data even when the labels are completely random.
Ranked #16 on
Image Classification
on WebVision-1000
Decoupled Novel Object Captioner
Thus, the sequence model can be decoupled from the novel object descriptions.
Focal Visual-Text Attention for Visual Question Answering
Recent insights on language and vision with neural networks have been successfully applied to simple single-image visual question answering.
Ranked #1 on
Memex Question Answering
on MemexQA
Focal Visual-Text Attention for Memex Question Answering
In addition to a text answer, a few grounding photos are also given to justify the answer.
Ranked #1 on
Memex Question Answering
on MemexQA
Composing Text and Image for Image Retrieval - An Empirical Odyssey
In this paper, we study the task of image retrieval, where the input query is specified in the form of an image plus some text that describes desired modifications to the input image.
Ranked #2 on
Image Retrieval with Multi-Modal Query
on MIT-States
Contrastive Adaptation Network for Unsupervised Domain Adaptation
Unsupervised Domain Adaptation (UDA) makes predictions for the target domain data while manual annotations are only available in the source domain.
Ranked #7 on
Domain Adaptation
on Office-31
Peeking into the Future: Predicting Future Person Activities and Locations in Videos
To facilitate the training, the network is learned with an auxiliary task of predicting future location in which the activity will happen.
Ranked #1 on
Activity Prediction
on ActEV
Let's Transfer Transformations of Shared Semantic Representations
In this work we show how one can learn transformations with no training examples by learning them on another domain and then transfer to the target domain.
Revisiting EmbodiedQA: A Simple Baseline and Beyond
In this paper, we empirically study this problem and introduce 1) a simple yet effective baseline that achieves promising performance; 2) an easier and practical setting for EmbodiedQA where an agent has a chance to adapt the trained model to a new environment before it actually answers users questions.
Eidetic 3D LSTM: A Model for Video Prediction and Beyond
We first evaluate the E3D-LSTM network on widely-used future video prediction datasets and achieve the state-of-the-art performance.
Ranked #1 on
Video Prediction
on KTH
(Cond metric)
State-aware Re-identification Feature for Multi-target Multi-camera Tracking
Multi-target Multi-camera Tracking (MTMCT) aims to extract the trajectories from videos captured by a set of cameras.
Robust Neural Machine Translation with Doubly Adversarial Inputs
Neural machine translation (NMT) often suffers from the vulnerability to noisy perturbations in the input.
Feature Partitioning for Efficient Multi-Task Architectures
Multi-task learning holds the promise of less data, parameters, and time than training of separate models.
Synthetic vs Real: Deep Learning on Controlled Noise
Performing controlled experiments on noisy data is essential in thoroughly understanding deep learning across a spectrum of noise levels.
Confident Learning: Estimating Uncertainty in Dataset Labels
Confident learning (CL) is an alternative approach which focuses instead on label quality by characterizing and identifying label errors in datasets, based on the principles of pruning noisy data, counting with probabilistic thresholds to estimate noise, and ranking examples to train with confidence.
Beyond Synthetic Noise: Deep Learning on Controlled Noisy Labels
Due to the lack of suitable datasets, previous research has only examined deep learning on controlled synthetic label noise, and real-world label noise has never been studied in a controlled setting.
Ranked #12 on
Image Classification
on WebVision-1000
The Garden of Forking Paths: Towards Multi-Future Trajectory Prediction
The first contribution is a new dataset, created in a realistic 3D simulator, which is based on real world trajectory data, and then extrapolated by human annotators to achieve different latent goals.
Ranked #1 on
Multi-future Trajectory Prediction
on ForkingPaths
Neural Design Network: Graphic Layout Generation with Constraints
The first module predicts a graph with complete relations from a graph with user-specified relations.
Controllable and Progressive Image Extrapolation
Image extrapolation aims at expanding the narrow field of view of a given image patch.
SimAug: Learning Robust Representations from 3D Simulation for Pedestrian Trajectory Prediction in Unseen Cameras
We refer to our method as SimAug.
Ranked #2 on
Trajectory Prediction
on ActEV
AdvAug: Robust Adversarial Augmentation for Neural Machine Translation
In this paper, we propose a new adversarial augmentation method for Neural Machine Translation (NMT).
Ranked #22 on
Machine Translation
on WMT2014 English-German
RetrieveGAN: Image Synthesis via Differentiable Patch Retrieval
Image generation from scene description is a cornerstone technique for the controlled generation, which is beneficial to applications such as content creation and image editing.
Text as Neural Operator: Image Manipulation by Text Instruction
In recent years, text-guided image manipulation has gained increasing attention in the multimedia and computer vision community.
Simplifying Reinforced Feature Selection via Restructured Choice Strategy of Single Agent
To address the question, we develop a single-agent reinforced feature selection approach integrated with restructured choice strategy.
Regularizing Generative Adversarial Networks under Limited Data
Recent years have witnessed the rapid progress of generative adversarial networks (GANs).
Ranked #1 on
Image Generation
on CIFAR-100
Faster Meta Update Strategy for Noise-Robust Deep Learning
It has been shown that deep neural networks are prone to overfitting on biased training data.
Self-supervised and Supervised Joint Training for Resource-rich Machine Translation
Self-supervised pre-training of text representations has been successfully applied to low-resource Neural Machine Translation (NMT).
ViTGAN: Training GANs with Vision Transformers
Recently, Vision Transformers (ViTs) have shown competitive performance on image recognition while requiring less vision-specific inductive biases.
Ranked #68 on
Image Generation
on CIFAR-10
Discrete Representations Strengthen Vision Transformer Robustness
Vision Transformer (ViT) is emerging as the state-of-the-art architecture for image recognition.
Ranked #3 on
Domain Generalization
on Stylized-ImageNet
Pyramid Adversarial Training Improves ViT Performance
In this work, we present pyramid adversarial training (PyramidAT), a simple and effective technique to improve ViT's overall performance.
Ranked #9 on
Domain Generalization
on ImageNet-C
(using extra training data)
BLT: Bidirectional Layout Transformer for Controllable Layout Generation
During inference, BLT first generates a draft layout from the input and then iteratively refines it into a high-quality layout by masking out low-confident attributes.
MaskGIT: Masked Generative Image Transformer
At inference time, the model begins with generating all tokens of an image simultaneously, and then refines the image iteratively conditioned on the previous generation.
Ranked #2 on
Text-to-Image Generation
on LHQC
Improved Masked Image Generation with Token-Critic
Given a masked-and-reconstructed real image, the Token-Critic model is trained to distinguish which visual tokens belong to the original image and which were sampled by the generative transformer.
Visual Prompt Tuning for Generative Transfer Learning
We base our framework on state-of-the-art generative vision transformers that represent an image as a sequence of visual tokens to the autoregressive or non-autoregressive transformers.
MAGVIT: Masked Generative Video Transformer
We introduce the MAsked Generative VIdeo Transformer, MAGVIT, to tackle various video synthesis tasks with a single model.
Ranked #1 on
Video Prediction
on Something-Something V2
Streaming Traffic Flow Prediction Based on Continuous Reinforcement Learning
We propose to formulate the problem as a continuous reinforcement learning task, where the agent is the next flow value predictor, the action is the next time-series flow value in the sensor, and the environment state is a dynamically fused representation of the sensor and transportation network.
Multi-View MOOC Quality Evaluation via Information-Aware Graph Representation Learning
To tackle the challenges, we formulate the problem as a course representation learning task-based and develop an Information-aware Graph Representation Learning(IaGRL) for multi-view MOOC quality evaluation.
A Multi-Source Information Learning Framework for Airbnb Price Prediction
Thirdly, we uses the points of interest(POI) around the rental house information generates a variety of spatial network graphs, and learns the embedding of the network to obtain the spatial feature embedding.
Muse: Text-To-Image Generation via Masked Generative Transformers
Compared to pixel-space diffusion models, such as Imagen and DALL-E 2, Muse is significantly more efficient due to the use of discrete tokens and requiring fewer sampling iterations; compared to autoregressive models, such as Parti, Muse is more efficient due to the use of parallel decoding.
Ranked #1 on
Text-to-Image Generation
on MS-COCO
(FID metric)
Auditing Gender Presentation Differences in Text-to-Image Models
Text-to-image models, which can generate high-quality images based on textual input, have recently enabled various content-creation tools.
StyleDrop: Text-to-Image Generation in Any Style
Pre-trained large text-to-image models synthesize impressive images with an appropriate use of text prompts.
Learning Disentangled Prompts for Compositional Image Synthesis
We study domain-adaptive image synthesis, the problem of teaching pretrained image generative models a new style or concept from as few as one image to synthesize novel images, to better understand the compositional image synthesis.
SPAE: Semantic Pyramid AutoEncoder for Multimodal Generation with Frozen LLMs
In this work, we introduce Semantic Pyramid AutoEncoder (SPAE) for enabling frozen LLMs to perform both understanding and generation tasks involving non-linguistic modalities such as images or videos.
VideoGLUE: Video General Understanding Evaluation of Foundation Models
We evaluate existing foundation models video understanding capabilities using a carefully designed experiment protocol consisting of three hallmark tasks (action recognition, temporal localization, and spatiotemporal localization), eight datasets well received by the community, and four adaptation methods tailoring a foundation model (FM) for a downstream task.
Language Model Beats Diffusion -- Tokenizer is Key to Visual Generation
While Large Language Models (LLMs) are the dominant models for generative tasks in language, they do not perform as well as diffusion models on image and video generation.
Ranked #2 on
Video Prediction
on Kinetics-600 12 frames, 64x64
Text-Driven Image Editing via Learnable Regions
Language has emerged as a natural interface for image editing.
Fine-grained Controllable Video Generation via Object Appearance and Context
To achieve detailed control, we propose a unified framework to jointly inject control signals into the existing text-to-video model.
Photorealistic Video Generation with Diffusion Models
We present W. A. L. T, a transformer-based approach for photorealistic video generation via diffusion modeling.
Ranked #1 on
Video Prediction
on Kinetics-600 12 frames, 64x64
VideoPoet: A Large Language Model for Zero-Shot Video Generation
We present VideoPoet, a language model capable of synthesizing high-quality video, with matching audio, from a large variety of conditioning signals.
Ranked #3 on
Text-to-Video Generation
on MSR-VTT
Spatial-Temporal Interplay in Human Mobility: A Hierarchical Reinforcement Learning Approach with Hypergraph Representation
In the realm of human mobility, the decision-making process for selecting the next-visit location is intricately influenced by a trade-off between spatial and temporal constraints, which are reflective of individual needs and preferences.
BRAU-Net++: U-Shaped Hybrid CNN-Transformer Network for Medical Image Segmentation
Specifically, BRAU-Net++ uses bi-level routing attention as the core building block to design our u-shaped encoder-decoder structure, in which both encoder and decoder are hierarchically constructed, so as to learn global semantic information while reducing computational complexity.
A Generalized Framework with Adaptive Weighted Soft-Margin for Imbalanced SVM Classification
In this paper, we present a new generalized framework with Adaptive Weight function for soft-margin Weighted SVM (AW-WSVM), which aims to enhance the issue of imbalance and outlier sensitivity in standard support vector machine (SVM) for classifying two-class data.
SimAug: Learning Robust Representations from Simulation for Trajectory Prediction
We approach this problem through the real-data-free setting in which the model is trained only on 3D simulation data and applied out-of-the-box to a wide variety of real cameras.
Ranked #1 on
Trajectory Forecasting
on ActEV
