Search Results for author:
Found 23 papers, 9 papers with code
Heterogeneous Memory Enhanced Multimodal Attention Model for Video Question Answering
In this paper, we propose a novel end-to-end trainable Video Question Answering (VideoQA) framework with three major components: 1) a new heterogeneous memory which can effectively learn global context information from appearance and motion features; 2) a redesigned question memory which helps understand the complex semantics of question and highlights queried subjects; and 3) a new multimodal fusion layer which performs multi-step reasoning by attending to relevant visual and textual hints with self-updated attention.
Ranked #27 on
Visual Question Answering (VQA)
on MSRVTT-QA
Boosting Light-Weight Depth Estimation Via Knowledge Distillation
However, this KD process can be challenging and insufficient due to the large model capacity gap between the teacher and the student.
Learning Latent Sub-events in Activity Videos Using Temporal Attention Filters
In this paper, we newly introduce the concept of temporal attention filters, and describe how they can be used for human activity recognition from videos.
Ranked #1 on
Activity Recognition In Videos
on DogCentric
Progressive Self-Distillation for Ground-to-Aerial Perception Knowledge Transfer
In this paper, we introduce the ground-to-aerial perception knowledge transfer and propose a progressive semi-supervised learning framework that enables drone perception using only labeled data of ground viewpoint and unlabeled data of flying viewpoints.
Lifelong-MonoDepth: Lifelong Learning for Multi-Domain Monocular Metric Depth Estimation
With the rapid advancements in autonomous driving and robot navigation, there is a growing demand for lifelong learning models capable of estimating metric (absolute) depth.
DeepDiary: Automatic Caption Generation for Lifelogging Image Streams
Lifelogging cameras capture everyday life from a first-person perspective, but generate so much data that it is hard for users to browse and organize their image collections effectively.
Improved Sample Complexity for Stochastic Compositional Variance Reduced Gradient
Convex composition optimization is an emerging topic that covers a wide range of applications arising from stochastic optimal control, reinforcement learning and multi-stage stochastic programming.
Where to Attack: A Dynamic Locator Model for Backdoor Attack in Text Classifications
Based on the predicted P2P, four effective strategies are introduced to show the BDA performance.
Improved Oracle Complexity of Variance Reduced Methods for Nonsmooth Convex Stochastic Composition Optimization
We consider the nonsmooth convex composition optimization problem where the objective is a composition of two finite-sum functions and analyze stochastic compositional variance reduced gradient (SCVRG) methods for them.
Joint Person Segmentation and Identification in Synchronized First- and Third-person Videos
In this paper, we wish to solve two specific problems: (1) given two or more synchronized third-person videos of a scene, produce a pixel-level segmentation of each visible person and identify corresponding people across different views (i. e., determine who in camera A corresponds with whom in camera B), and (2) given one or more synchronized third-person videos as well as a first-person video taken by a mobile or wearable camera, segment and identify the camera wearer in the third-person videos.
Multi-Task Spatiotemporal Neural Networks for Structured Surface Reconstruction
Deep learning methods have surpassed the performance of traditional techniques on a wide range of problems in computer vision, but nearly all of this work has studied consumer photos, where precisely correct output is often not critical.
Forecasting Hands and Objects in Future Frames
The key idea is that (1) an intermediate representation of a convolutional object recognition model abstracts scene information in its frame and that (2) we can predict (i. e., regress) such representations corresponding to the future frames based on that of the current frame.
Identifying First-person Camera Wearers in Third-person Videos
We consider scenarios in which we wish to perform joint scene understanding, object tracking, activity recognition, and other tasks in environments in which multiple people are wearing body-worn cameras while a third-person static camera also captures the scene.
Federated Generative Adversarial Learning
This work studies training generative adversarial networks under the federated learning setting.
Projection Robust Wasserstein Distance and Riemannian Optimization
Projection robust Wasserstein (PRW) distance, or Wasserstein projection pursuit (WPP), is a robust variant of the Wasserstein distance.
Federated Few-Shot Learning with Adversarial Learning
In this paper, we propose a federated few-shot learning (FedFSL) framework to learn a few-shot classification model that can classify unseen data classes with only a few labeled samples.
Learn2Agree: Fitting with Multiple Annotators without Objective Ground Truth
For such issues, we propose a novel Learning to Agreement (Learn2Agree) framework to tackle the challenge of learning from multiple annotators without objective ground truth.
Deep Depth Completion from Extremely Sparse Data: A Survey
Depth completion aims at predicting dense pixel-wise depth from an extremely sparse map captured from a depth sensor, e. g., LiDARs.
Dense Depth Distillation with Out-of-Distribution Simulated Images
We study data-free knowledge distillation (KD) for monocular depth estimation (MDE), which learns a lightweight model for real-world depth perception tasks by compressing it from a trained teacher model while lacking training data in the target domain.
Federated Prompting and Chain-of-Thought Reasoning for Improving LLMs Answering
Due to the unsatisfactory accuracy of LLMs' zero-shot prompting with standalone questions, we propose to improve the distributed synonymous questions using Self-Consistency (SC) and Chain-of-Thought (CoT) techniques.
Towards Applying Powerful Large AI Models in Classroom Teaching: Opportunities, Challenges and Prospects
This perspective paper proposes a series of interactive scenarios that utilize Artificial Intelligence (AI) to enhance classroom teaching, such as dialogue auto-completion, knowledge and style transfer, and assessment of AI-generated content.
Carbon Price Forecasting with Quantile Regression and Feature Selection
Carbon futures has recently emerged as a novel financial asset in the trading markets such as the European Union and China.
Sci-CoT: Leveraging Large Language Models for Enhanced Knowledge Distillation in Small Models for Scientific QA
Large Language Models (LLMs) have shown outstanding performance across wide range of downstream tasks.
