Search Results for author:
Found 83 papers, 18 papers with code
FLASH: Federated Learning Across Simultaneous Heterogeneities
The key premise of federated learning (FL) is to train ML models across a diverse set of data-owners (clients), without exchanging local data.
Plug-and-Play Transformer Modules for Test-Time Adaptation
Parameter-efficient tuning (PET) methods such as LoRA, Adapter, and Visual Prompt Tuning (VPT) have found success in enabling adaptation to new domains by tuning small modules within a transformer model.
MeTA: Multi-source Test Time Adaptation
Test time adaptation is the process of adapting, in an unsupervised manner, a pre-trained source model to each incoming batch of the test data (i. e., without requiring a substantial portion of the test data to be available, as in traditional domain adaptation) and without access to the source data.
STRIDE: Single-video based Temporally Continuous Occlusion Robust 3D Pose Estimation
This challenge arises because these models struggle to generalize beyond their training datasets, and the variety of occlusions is hard to capture in the training data.
Active Learning Guided Federated Online Adaptation: Applications in Medical Image Segmentation
Motivated by these, we propose a method for medical image segmentation that adapts to each incoming data batch (online adaptation), incorporates physician feedback through active learning, and assimilates knowledge across facilities in a federated setup.
Towards Granularity-adjusted Pixel-level Semantic Annotation
Specifically, we propose an approach to enable the Segment Anything Model (SAM) with semantic recognition capability to generate pixel-level annotations for images without any manual supervision.
POISE: Pose Guided Human Silhouette Extraction under Occlusions
Human silhouette extraction is a fundamental task in computer vision with applications in various downstream tasks.
Effective Restoration of Source Knowledge in Continual Test Time Adaptation
By restoring the knowledge from the source, it effectively corrects the negative consequences arising from the gradual deterioration of model parameters caused by ongoing shifts in the domain.
Learning Deformable 3D Graph Similarity to Track Plant Cells in Unregistered Time Lapse Images
Tracking of plant cells in images obtained by microscope is a challenging problem due to biological phenomena such as large number of cells, non-uniform growth of different layers of the tightly packed plant cells and cell division.
Prior-guided Source-free Domain Adaptation for Human Pose Estimation
To address this limitation, we focus on the task of source-free domain adaptation for pose estimation, where a source model must adapt to a new target domain using only unlabeled target data.
SUMMIT: Source-Free Adaptation of Uni-Modal Models to Multi-Modal Targets
In this work, we relax both of these assumptions by addressing the problem of adapting a set of models trained independently on uni-modal data to a target domain consisting of unlabeled multi-modal data, without having access to the original source dataset.
Efficient Controllable Multi-Task Architectures
Further, we present a simple but effective search algorithm that translates user constraints to runtime width configurations of both the shared encoder and task decoders, for sampling the sub-architectures.
FedYolo: Augmenting Federated Learning with Pretrained Transformers
These insights on scale and modularity motivate a new federated learning approach we call "You Only Load Once" (FedYolo): The clients load a full PTF model once and all future updates are accomplished through communication-efficient modules with limited catastrophic-forgetting, where each task is assigned to its own module.
Collaborative Multi-Agent Video Fast-Forwarding
We show that compared with other approaches in the literature, our frameworks achieve better coverage of important frames, while significantly reducing the number of frames processed at each agent.
Cross-Domain Video Anomaly Detection without Target Domain Adaptation
Most cross-domain unsupervised Video Anomaly Detection (VAD) works assume that at least few task-relevant target domain training data are available for adaptation from the source to the target domain.
AVLEN: Audio-Visual-Language Embodied Navigation in 3D Environments
Similar to audio-visual navigation tasks, the goal of our embodied agent is to localize an audio event via navigating the 3D visual world; however, the agent may also seek help from a human (oracle), where the assistance is provided in free-form natural language.
Centroid Distance Keypoint Detector for Colored Point Clouds
Results demonstrate that our proposed CED keypoint detector requires minimal computational time while attaining high repeatability.
Leveraging Local Patch Differences in Multi-Object Scenes for Generative Adversarial Attacks
Our goal is to design an attack strategy that can learn from such natural scenes by leveraging the local patch differences that occur inherently in such images (e. g. difference between the local patch on the object `person' and the object `bike' in a traffic scene).
GAMA: Generative Adversarial Multi-Object Scene Attacks
Using the joint image-text features to train the generator, we show that GAMA can craft potent transferable perturbations in order to fool victim classifiers in various attack settings.
Cross-Modal Knowledge Transfer Without Task-Relevant Source Data
In such cases, transferring knowledge from a neural network trained on a well-labeled large dataset in the source modality (RGB) to a neural network that works on a target modality (depth, infrared, etc.)
Poisson2Sparse: Self-Supervised Poisson Denoising From a Single Image
In this work, we explore a sparsity and dictionary learning-based approach and present a novel self-supervised learning method for single-image denoising where the noise is approximated as a Poisson process, requiring no clean ground-truth data.
A-ACT: Action Anticipation through Cycle Transformations
To incorporate this ability in intelligent systems a question worth pondering upon is how exactly do we anticipate?
Zero-Query Transfer Attacks on Context-Aware Object Detectors
We compare our zero-query attack against a few-query scheme that repeatedly checks if the victim system is fooled.
Controllable Dynamic Multi-Task Architectures
In contrast to the existing dynamic multi-task approaches that adjust only the weights within a fixed architecture, our approach affords the flexibility to dynamically control the total computational cost and match the user-preferred task importance better.
Context-Aware Transfer Attacks for Object Detection
In this paper, we present a new approach to generate context-aware attacks for object detectors.
ADC: Adversarial attacks against object Detection that evade Context consistency checks
A very recent defense strategy for detecting adversarial examples, that has been shown to be robust to current attacks, is to check for intrinsic context consistencies in the input data, where context refers to various relationships (e. g., object-to-object co-occurrence relationships) in images.
Adversarial Attacks on Black Box Video Classifiers: Leveraging the Power of Geometric Transformations
When compared to the image classification models, black-box adversarial attacks against video classification models have been largely understudied.
Multi-Expert Adversarial Attack Detection in Person Re-identification Using Context Inconsistency
The success of deep neural networks (DNNs) has promoted the widespread applications of person re-identification (ReID).
Exploiting Multi-Object Relationships for Detecting Adversarial Attacks in Complex Scenes
Vision systems that deploy Deep Neural Networks (DNNs) are known to be vulnerable to adversarial examples.
Ada-VSR: Adaptive Video Super-Resolution with Meta-Learning
Specifically, meta-learning is employed to obtain adaptive parameters, using a large-scale external dataset, that can adapt quickly to the novel condition (degradation model) of the given test video during the internal learning task, thereby exploiting external and internal information of a video for super-resolution.
Reconstruction guided Meta-learning for Few Shot Open Set Recognition
However, it is a critical task in many applications like environmental monitoring, where the number of labeled examples for each class is limited.
Deep Quantized Representation for Enhanced Reconstruction
In this paper, we propose a data-driven Deep Quantized Latent Representation (DQLR) methodology for high-quality image reconstruction in the Shoot Apical Meristem (SAM) of Arabidopsis thaliana.
Spatio-Temporal Representation Factorization for Video-based Person Re-Identification
To alleviate these problems, we propose Spatio-Temporal Representation Factorization (STRF), a flexible new computational unit that can be used in conjunction with most existing 3D convolutional neural network architectures for re-ID.
Ranked #2 on
Person Re-Identification
on DukeMTMC-VideoReID
Cross-domain Imitation from Observations
Once this correspondence is found, we can directly transfer the demonstrations on one domain to the other and use it for imitation.
Unsupervised Multi-source Domain Adaptation Without Access to Source Data
A recent line of work addressed this problem and proposed an algorithm that transfers knowledge to the unlabeled target domain from a single source model without requiring access to the source data.
Detection and Localization of Facial Expression Manipulations
Thus, it is important to develop methods that can detect manipulations in facial expressions, and localize the manipulated regions.
Facial Expression Recognition
Facial Expression Recognition (FER)
+1
Learning to identify image manipulations in scientific publications
We show that our method leads to a 90% accuracy rate of detecting duplicated images, a ~ 13% improvement in detection accuracy in comparison to other manipulation detection methods.
Exploiting Context for Robustness to Label Noise in Active Learning
Several works in computer vision have demonstrated the effectiveness of active learning for adapting the recognition model when new unlabeled data becomes available.
ALANET: Adaptive Latent Attention Network forJoint Video Deblurring and Interpolation
Different from these works, we address a more realistic problem of high frame-rate sharp video synthesis with no prior assumption that input is always blurry.
Measurement-driven Security Analysis of Imperceptible Impersonation Attacks
The emergence of Internet of Things (IoT) brings about new security challenges at the intersection of cyber and physical spaces.
Text-based Localization of Moments in a Video Corpus
This task poses a unique challenge as the system is required to perform: (i) retrieval of the relevant video where only a segment of the video corresponds with the queried sentence, and (ii) temporal localization of moment in the relevant video based on sentence query.
Adversarial Knowledge Transfer from Unlabeled Data
While machine learning approaches to visual recognition offer great promise, most of the existing methods rely heavily on the availability of large quantities of labeled training data.
Distributed Multi-agent Video Fast-forwarding
In many intelligent systems, a network of agents collaboratively perceives the environment for better and more efficient situation awareness.
Domain Adaptive Semantic Segmentation Using Weak Labels
In this work, we propose a novel framework for domain adaptation in semantic segmentation with image-level weak labels in the target domain.
Camera On-boarding for Person Re-identification using Hypothesis Transfer Learning
Most of the existing approaches for person re-identification consider a static setting where the number of cameras in the network is fixed.
Learning Person Re-identification Models from Videos with Weak Supervision
In order to cope with this issue, we introduce the problem of learning person re-identification models from videos with weak supervision.
Multiple Instance Learning
Video-Based Person Re-Identification
Exploiting Temporal Coherence for Self-Supervised One-shot Video Re-identification
While supervised techniques in re-identification are extremely effective, the need for large amounts of annotations makes them impractical for large camera networks.
Non-Adversarial Video Synthesis with Learned Priors
Different from these methods, we focus on the problem of generating videos from latent noise vectors, without any reference input frames.
Learning from Trajectories via Subgoal Discovery
Learning to solve complex goal-oriented tasks with sparse terminal-only rewards often requires an enormous number of samples.
Exploiting Global Camera Network Constraints for Unsupervised Video Person Re-identification
Additionally, a cross-view matching strategy followed by global camera network constraints is proposed to explore the matching relationships across the entire camera network.
Prediction and Description of Near-Future Activities in Video
Most of the existing works on human activity analysis focus on recognition or early recognition of the activity labels from complete or partial observations.
A Skip Connection Architecture for Localization of Image Manipulations
In order to exploit these traces in localizing the tampered regions, we propose an encoder-decoder based network where we fuse representations from early layers in the encoder (which are richer in low-level spatial cues, like edges) by skip pooling with representations of the last layer of the decoder and use for manipulation detection.
Context-Aware Query Selection for Active Learning in Event Recognition
We formulate a conditional random field model that encodes the context and devise an information-theoretic approach that utilizes entropy and mutual information of the nodes to compute the set of most informative queries, which are labeled by a human.
Weakly Supervised Video Moment Retrieval From Text Queries
The weak nature of the supervision is because, during training, we only have access to the video-text pairs rather than the temporal extent of the video to which different text descriptions relate.
Detecting GAN generated Fake Images using Co-occurrence Matrices
The advent of Generative Adversarial Networks (GANs) has brought about completely novel ways of transforming and manipulating pixels in digital images.
Hybrid LSTM and Encoder-Decoder Architecture for Detection of Image Forgeries
This paper proposes a high-confidence manipulation localization architecture which utilizes resampling features, Long-Short Term Memory (LSTM) cells, and encoder-decoder network to segment out manipulated regions from non-manipulated ones.
Webly Supervised Joint Embedding for Cross-Modal lmage-Text Retrieval
Inspired by the recent success of web-supervised learning in deep neural networks, we capitalize on readily-available web images with noisy annotations to learn robust image-text joint representation.
Multi-View Frame Reconstruction with Conditional GAN
Multi-view frame reconstruction is an important problem particularly when multiple frames are missing and past and future frames within the camera are far apart from the missing ones.
Webly Supervised Joint Embedding for Cross-Modal Image-Text Retrieval
Inspired by the recent success of webly supervised learning in deep neural networks, we capitalize on readily-available web images with noisy annotations to learn robust image-text joint representation.
Contemplating Visual Emotions: Understanding and Overcoming Dataset Bias
While machine learning approaches to visual emotion recognition offer great promise, current methods consider training and testing models on small scale datasets covering limited visual emotion concepts.
Incorporating Scalability in Unsupervised Spatio-Temporal Feature Learning
This necessitates learning of visual features from videos in an unsupervised setting.
W-TALC: Weakly-supervised Temporal Activity Localization and Classification
Most activity localization methods in the literature suffer from the burden of frame-wise annotation requirement.
Ranked #1 on
Action Classification
on ActivityNet-1.2
Learning Joint Embedding with Multimodal Cues for Cross-Modal Video-Text Retrieval
Constructing a joint representation invariant across different modalities (e. g., video, language) is of significant importance in many multimedia applications.
Ranked #37 on
Video Retrieval
on MSR-VTT
Exploiting Transitivity for Learning Person Re-Identification Models on a Budget
Minimization of labeling effort for person re-identification in camera networks is an important problem as most of the existing popular methods are supervised and they require large amount of manual annotations, acquiring which is a tedious job.
FFNet: Video Fast-Forwarding via Reinforcement Learning
The first group is supported by video summarization techniques, which require processing of the entire video to select an important subset for showing to users.
Boosting Image Forgery Detection using Resampling Features and Copy-move analysis
Realistic image forgeries involve a combination of splicing, resampling, cloning, region removal and other methods.
Joint Prediction of Activity Labels and Starting Times in Untrimmed Videos
We propose a network similar to a hybrid Siamese network with three branches to jointly learn both the future label and the starting time.
Exploiting Spatial Structure for Localizing Manipulated Image Regions
In order to formulate the framework, we employ a hybrid CNN-LSTM model to capture discriminative features between manipulated and non-manipulated regions.
Weakly Supervised Summarization of Web Videos
Casting the problem as a weakly supervised learning problem, we propose a flexible deep 3D CNN architecture to learn the notion of importance using only video-level annotation, and without any human-crafted training data.
Detection and Localization of Image Forgeries using Resampling Features and Deep Learning
In this paper, we propose two methods to detect and localize image manipulations based on a combination of resampling features and deep learning.
The Impact of Typicality for Informative Representative Selection
In computer vision, selection of the most informative samples from a huge pool of training data in order to learn a good recognition model is an active research problem.
Unsupervised Adaptive Re-identification in Open World Dynamic Camera Networks
Most approaches have neglected the dynamic and open world nature of the re-identification problem, where a new camera may be temporarily inserted into an existing system to get additional information.
Collaborative Summarization of Topic-Related Videos
Large collections of videos are grouped into clusters by a topic keyword, such as Eiffel Tower or Surfing, with many important visual concepts repeating across them.
Multi-View Surveillance Video Summarization via Joint Embedding and Sparse Optimization
In this paper, with the aim of summarizing multi-view videos, we introduce a novel unsupervised framework via joint embedding and sparse representative selection.
Diversity-aware Multi-Video Summarization
Most video summarization approaches have focused on extracting a summary from a single video; we propose an unsupervised framework for summarizing a collection of videos.
Video Summarization in a Multi-View Camera Network
While most existing video summarization approaches aim to extract an informative summary of a single video, we propose a novel framework for summarizing multi-view videos by exploiting both intra- and inter-view content correlations in a joint embedding space.
Temporal Model Adaptation for Person Re-Identification
Person re-identification is an open and challenging problem in computer vision.
Continuous Adaptation of Multi-Camera Person Identification Models through Sparse Non-redundant Representative Selection
We demonstrate the effectiveness of our approach on multi-camera person re-identification datasets, to demonstrate the feasibility of learning online classification models in multi-camera big data applications.
Learning Temporal Regularity in Video Sequences
Perceiving meaningful activities in a long video sequence is a challenging problem due to ambiguous definition of 'meaningfulness' as well as clutters in the scene.
Ranked #2 on
Traffic Accident Detection
on A3D
Context Aware Active Learning of Activity Recognition Models
We formulate a conditional random field (CRF) model that encodes the context and devise an information theoretic approach that utilizes entropy and mutual information of the nodes to compute the set of most informative query instances, which need to be labeled by a human.
Incremental Activity Modeling and Recognition in Streaming Videos
Most of the state-of-the-art approaches to human activity recognition in video need an intensive training stage and assume that all of the training examples are labeled and available beforehand.
Context-Aware Modeling and Recognition of Activities in Video
This is motivated from the observations that the activities related in space and time rarely occur independently and can serve as the context for each other.
Information Consensus for Distributed Multi-target Tracking
The estimation errors in tracking and data association, as well as the effect of naivety, are jointly addressed leading to the development of an informationweighted consensus algorithm, which we term as the Multitarget Information Consensus (MTIC) algorithm.
