Search Results for author: Rameswar Panda

Found 67 papers, 34 papers with code

Continuous Adaptation of Multi-Camera Person Identification Models through Sparse Non-redundant Representative Selection

no code implementations1 Jul 2016 Abir Das, Rameswar Panda, Amit K. Roy-Chowdhury

We demonstrate the effectiveness of our approach on multi-camera person re-identification datasets, to demonstrate the feasibility of learning online classification models in multi-camera big data applications.

Person Identification Person Re-Identification

Video Summarization in a Multi-View Camera Network

no code implementations1 Aug 2016 Rameswar Panda, Abir Das, Amit K. Roy-Chowdhury

While most existing video summarization approaches aim to extract an informative summary of a single video, we propose a novel framework for summarizing multi-view videos by exploiting both intra- and inter-view content correlations in a joint embedding space.

Video Summarization

Unsupervised Adaptive Re-identification in Open World Dynamic Camera Networks

no code implementations CVPR 2017 Rameswar Panda, Amran Bhuiyan, Vittorio Murino, Amit K. Roy-Chowdhury

Most approaches have neglected the dynamic and open world nature of the re-identification problem, where a new camera may be temporarily inserted into an existing system to get additional information.

Person Re-Identification

Multi-View Surveillance Video Summarization via Joint Embedding and Sparse Optimization

no code implementations9 Jun 2017 Rameswar Panda, Amit K. Roy-Chowdhury

In this paper, with the aim of summarizing multi-view videos, we introduce a novel unsupervised framework via joint embedding and sparse representative selection.

Video Summarization

Diversity-aware Multi-Video Summarization

no code implementations9 Jun 2017 Rameswar Panda, Niluthpol Chowdhury Mithun, Amit K. Roy-Chowdhury

Most video summarization approaches have focused on extracting a summary from a single video; we propose an unsupervised framework for summarizing a collection of videos.

Video Summarization

Collaborative Summarization of Topic-Related Videos

no code implementations CVPR 2017 Rameswar Panda, Amit K. Roy-Chowdhury

Large collections of videos are grouped into clusters by a topic keyword, such as Eiffel Tower or Surfing, with many important visual concepts repeating across them.

Attribute Information Retrieval +1

Weakly Supervised Summarization of Web Videos

no code implementations ICCV 2017 Rameswar Panda, Abir Das, Ziyan Wu, Jan Ernst, Amit K. Roy-Chowdhury

Casting the problem as a weakly supervised learning problem, we propose a flexible deep 3D CNN architecture to learn the notion of importance using only video-level annotation, and without any human-crafted training data.

Weakly-supervised Learning

FFNet: Video Fast-Forwarding via Reinforcement Learning

1 code implementation CVPR 2018 Shuyue Lan, Rameswar Panda, Qi Zhu, Amit K. Roy-Chowdhury

The first group is supported by video summarization techniques, which require processing of the entire video to select an important subset for showing to users.

reinforcement-learning Reinforcement Learning (RL) +1

Contemplating Visual Emotions: Understanding and Overcoming Dataset Bias

no code implementations ECCV 2018 Rameswar Panda, Jianming Zhang, Haoxiang Li, Joon-Young Lee, Xin Lu, Amit K. Roy-Chowdhury

While machine learning approaches to visual emotion recognition offer great promise, current methods consider training and testing models on small scale datasets covering limited visual emotion concepts.

Emotion Recognition

Webly Supervised Joint Embedding for Cross-Modal Image-Text Retrieval

no code implementations23 Aug 2018 Niluthpol Chowdhury Mithun, Rameswar Panda, Evangelos E. Papalexakis, Amit K. Roy-Chowdhury

Inspired by the recent success of webly supervised learning in deep neural networks, we capitalize on readily-available web images with noisy annotations to learn robust image-text joint representation.

Cross-Modal Retrieval Retrieval +1

Webly Supervised Joint Embedding for Cross-Modal lmage-Text Retrieval

no code implementations Proceedings of the 26th ACM international conference on Multimedia·October 2018 2018 Niluthpol Chowdhury Mithun, Rameswar Panda, Vagelis Papalexakis, Amit K. Roy-Chowdhury

Inspired by the recent success of web-supervised learning in deep neural networks, we capitalize on readily-available web images with noisy annotations to learn robust image-text joint representation.

Cross-Modal Retrieval Retrieval +1

Exploiting Global Camera Network Constraints for Unsupervised Video Person Re-identification

no code implementations27 Aug 2019 Xueping Wang, Rameswar Panda, Min Liu, Yaonan Wang, Amit K. Roy-Chowdhury

Additionally, a cross-view matching strategy followed by global camera network constraints is proposed to explore the matching relationships across the entire camera network.

Graph Matching Metric Learning +2

Estimating Skin Tone and Effects on Classification Performance in Dermatology Datasets

no code implementations29 Oct 2019 Newton M. Kinyanjui, Timothy Odonga, Celia Cintas, Noel C. F. Codella, Rameswar Panda, Prasanna Sattigeri, Kush R. Varshney

We find that the majority of the data in the the two datasets have ITA values between 34. 5{\deg} and 48{\deg}, which are associated with lighter skin, and is consistent with under-representation of darker skinned populations in these datasets.

BIG-bench Machine Learning General Classification +1

Non-Adversarial Video Synthesis with Learned Priors

1 code implementation CVPR 2020 Abhishek Aich, Akash Gupta, Rameswar Panda, Rakib Hyder, M. Salman Asif, Amit K. Roy-Chowdhury

Different from these methods, we focus on the problem of generating videos from latent noise vectors, without any reference input frames.

NASTransfer: Analyzing Architecture Transferability in Large Scale Neural Architecture Search

no code implementations23 Jun 2020 Rameswar Panda, Michele Merler, Mayoore Jaiswal, Hui Wu, Kandan Ramakrishnan, Ulrich Finkler, Chun-Fu Chen, Minsik Cho, David Kung, Rogerio Feris, Bishwaranjan Bhattacharjee

The typical way of conducting large scale NAS is to search for an architectural building block on a small dataset (either using a proxy set from the large dataset or a completely different small scale dataset) and then transfer the block to a larger dataset.

Neural Architecture Search

AR-Net: Adaptive Frame Resolution for Efficient Action Recognition

1 code implementation ECCV 2020 Yue Meng, Chung-Ching Lin, Rameswar Panda, Prasanna Sattigeri, Leonid Karlinsky, Aude Oliva, Kate Saenko, Rogerio Feris

Specifically, given a video frame, a policy network is used to decide what input resolution should be used for processing by the action recognition model, with the goal of improving both accuracy and efficiency.

Action Recognition

Mitigating Dataset Imbalance via Joint Generation and Classification

1 code implementation12 Aug 2020 Aadarsh Sahoo, Ankit Singh, Rameswar Panda, Rogerio Feris, Abir Das

In this work we address these questions from the perspective of dataset imbalance resulting out of severe under-representation of annotated training data for certain classes and its effect on both deep classification and generation methods.

Classification General Classification

Adversarial Knowledge Transfer from Unlabeled Data

1 code implementation13 Aug 2020 Akash Gupta, Rameswar Panda, Sujoy Paul, Jianming Zhang, Amit K. Roy-Chowdhury

While machine learning approaches to visual recognition offer great promise, most of the existing methods rely heavily on the availability of large quantities of labeled training data.

Transfer Learning

Measurement-driven Security Analysis of Imperceptible Impersonation Attacks

no code implementations26 Aug 2020 Shasha Li, Karim Khalil, Rameswar Panda, Chengyu Song, Srikanth V. Krishnamurthy, Amit K. Roy-Chowdhury, Ananthram Swami

The emergence of Internet of Things (IoT) brings about new security challenges at the intersection of cyber and physical spaces.

Face Recognition

Deep Analysis of CNN-based Spatio-temporal Representations for Action Recognition

1 code implementation CVPR 2021 Chun-Fu Chen, Rameswar Panda, Kandan Ramakrishnan, Rogerio Feris, John Cohn, Aude Oliva, Quanfu Fan

In recent years, a number of approaches based on 2D or 3D convolutional neural networks (CNN) have emerged for video action recognition, achieving state-of-the-art results on several large-scale benchmark datasets.

Action Recognition Temporal Action Localization

Select, Label, and Mix: Learning Discriminative Invariant Feature Representations for Partial Domain Adaptation

no code implementations6 Dec 2020 Aadarsh Sahoo, Rameswar Panda, Rogerio Feris, Kate Saenko, Abir Das

Partial domain adaptation which assumes that the unknown target label space is a subset of the source label space has attracted much attention in computer vision.

Partial Domain Adaptation

A Maximal Correlation Approach to Imposing Fairness in Machine Learning

no code implementations30 Dec 2020 Joshua Lee, Yuheng Bu, Prasanna Sattigeri, Rameswar Panda, Gregory Wornell, Leonid Karlinsky, Rogerio Feris

As machine learning algorithms grow in popularity and diversify to many industries, ethical and legal concerns regarding their fairness have become increasingly relevant.

BIG-bench Machine Learning Fairness

Semi-Supervised Action Recognition with Temporal Contrastive Learning

1 code implementation CVPR 2021 Ankit Singh, Omprakash Chakraborty, Ashutosh Varshney, Rameswar Panda, Rogerio Feris, Kate Saenko, Abir Das

We approach this problem by learning a two-pathway temporal contrastive model using unlabeled videos at two different speeds leveraging the fact that changing video speed does not change an action.

Action Recognition Contrastive Learning

VA-RED$^2$: Video Adaptive Redundancy Reduction

no code implementations ICLR 2021 Bowen Pan, Rameswar Panda, Camilo Fosco, Chung-Ching Lin, Alex Andonian, Yue Meng, Kate Saenko, Aude Oliva, Rogerio Feris

An inherent property of real-world videos is the high correlation of information across frames which can translate into redundancy in either temporal or spatial feature maps of the models, or both.

Improved Techniques for Quantizing Deep Networks with Adaptive Bit-Widths

no code implementations2 Mar 2021 Ximeng Sun, Rameswar Panda, Chun-Fu Chen, Naigang Wang, Bowen Pan, Kailash Gopalakrishnan, Aude Oliva, Rogerio Feris, Kate Saenko

Second, to effectively transfer knowledge, we develop a dynamic block swapping method by randomly replacing the blocks in the lower-precision student network with the corresponding blocks in the higher-precision teacher network.

Image Classification Quantization +2

CrossViT: Cross-Attention Multi-Scale Vision Transformer for Image Classification

14 code implementations ICCV 2021 Chun-Fu Chen, Quanfu Fan, Rameswar Panda

To this end, we propose a dual-branch transformer to combine image patches (i. e., tokens in a transformer) of different sizes to produce stronger image features.

General Classification Image Classification

AdaMML: Adaptive Multi-Modal Learning for Efficient Video Recognition

1 code implementation ICCV 2021 Rameswar Panda, Chun-Fu Chen, Quanfu Fan, Ximeng Sun, Kate Saenko, Aude Oliva, Rogerio Feris

Specifically, given a video segment, a multi-modal policy network is used to decide what modalities should be used for processing by the recognition model, with the goal of improving both accuracy and efficiency.

Video Recognition

RegionViT: Regional-to-Local Attention for Vision Transformers

3 code implementations ICLR 2022 Chun-Fu Chen, Rameswar Panda, Quanfu Fan

The regional-to-local attention includes two steps: first, the regional self-attention extract global information among all regional tokens and then the local self-attention exchanges the information among one regional token and the associated local tokens via self-attention.

Action Recognition Image Classification +2

Dynamic Distillation Network for Cross-Domain Few-Shot Recognition with Unlabeled Data

1 code implementation NeurIPS 2021 Ashraful Islam, Chun-Fu Chen, Rameswar Panda, Leonid Karlinsky, Rogerio Feris, Richard J. Radke

As the base dataset and unlabeled dataset are from different domains, projecting the target images in the class-domain of the base dataset with a fixed pretrained model might be sub-optimal.

cross-domain few-shot learning

IA-RED$^2$: Interpretability-Aware Redundancy Reduction for Vision Transformers

no code implementations NeurIPS 2021 Bowen Pan, Rameswar Panda, Yifan Jiang, Zhangyang Wang, Rogerio Feris, Aude Oliva

The self-attention-based model, transformer, is recently becoming the leading backbone in the field of computer vision.

Can An Image Classifier Suffice For Action Recognition?

1 code implementation ICLR 2022 Quanfu Fan, Chun-Fu, Chen, Rameswar Panda

We explore a new perspective on video understanding by casting the video recognition problem as an image recognition task.

Action Recognition Image Classification +2

Dynamic Network Quantization for Efficient Video Inference

1 code implementation ICCV 2021 Ximeng Sun, Rameswar Panda, Chun-Fu Chen, Aude Oliva, Rogerio Feris, Kate Saenko

Deep convolutional networks have recently achieved great success in video recognition, yet their practical realization remains a challenge due to the large amount of computational resources required to achieve robust recognition.

Quantization Video Recognition

Selective Regression Under Fairness Criteria

1 code implementation28 Oct 2021 Abhin Shah, Yuheng Bu, Joshua Ka-Wing Lee, Subhro Das, Rameswar Panda, Prasanna Sattigeri, Gregory W. Wornell

Selective regression allows abstention from prediction if the confidence to make an accurate prediction is not sufficient.

Fairness regression

Contrast and Mix: Temporal Contrastive Video Domain Adaptation with Background Mixing

no code implementations NeurIPS 2021 Aadarsh Sahoo, Rutav Shah, Rameswar Panda, Kate Saenko, Abir Das

Unsupervised domain adaptation which aims to adapt models trained on a labeled source domain to a completely unlabeled target domain has attracted much attention in recent years.

Contrastive Learning Unsupervised Domain Adaptation +1

A Maximal Correlation Framework for Fair Machine Learning

no code implementations Entropy 2022 Joshua Lee, Yuheng Bu, Prasanna Sattigeri, Rameswar Panda, Gregory W. Wornell, Leonid Karlinsky and Rogerio Schmidt Feris

As machine learning algorithms grow in popularity and diversify to many industries, ethical and legal concerns regarding their fairness have become increasingly relevant.

Fairness

VALHALLA: Visual Hallucination for Machine Translation

1 code implementation CVPR 2022 Yi Li, Rameswar Panda, Yoon Kim, Chun-Fu, Chen, Rogerio Feris, David Cox, Nuno Vasconcelos

In particular, given a source sentence an autoregressive hallucination transformer is used to predict a discrete visual representation from the input text, and the combined text and hallucinated representations are utilized to obtain the target translation.

Hallucination Multimodal Machine Translation +2

FETA: Towards Specializing Foundation Models for Expert Task Applications

1 code implementation8 Sep 2022 Amit Alfassy, Assaf Arbelle, Oshri Halimi, Sivan Harary, Roei Herzig, Eli Schwartz, Rameswar Panda, Michele Dolfi, Christoph Auer, Kate Saenko, PeterW. J. Staar, Rogerio Feris, Leonid Karlinsky

However, as we show in this paper, FMs still have poor out-of-the-box performance on expert tasks (e. g. retrieval of car manuals technical illustrations from language queries), data for which is either unseen or belonging to a long-tail part of the data distribution of the huge datasets used for FM pre-training.

Domain Generalization Image Retrieval +6

Semi-Supervised Domain Adaptation with Auto-Encoder via Simultaneous Learning

no code implementations18 Oct 2022 Md Mahmudur Rahman, Rameswar Panda, Mohammad Arif Ul Alam

We present a new semi-supervised domain adaptation framework that combines a novel auto-encoder-based domain adaptation model with a simultaneous learning scheme providing stable improvements over state-of-the-art domain adaptation models.

Domain Adaptation Semi-supervised Domain Adaptation

ConStruct-VL: Data-Free Continual Structured VL Concepts Learning

1 code implementation CVPR 2023 James Seale Smith, Paola Cascante-Bonilla, Assaf Arbelle, Donghyun Kim, Rameswar Panda, David Cox, Diyi Yang, Zsolt Kira, Rogerio Feris, Leonid Karlinsky

This leads to reasoning mistakes, which need to be corrected as they occur by teaching VL models the missing SVLC skills; often this must be done using private data where the issue was found, which naturally leads to a data-free continual (no task-id) VL learning setting.

Synthetic Pre-Training Tasks for Neural Machine Translation

no code implementations19 Dec 2022 Zexue He, Graeme Blackwood, Rameswar Panda, Julian McAuley, Rogerio Feris

Pre-training models with large crawled corpora can lead to issues such as toxicity and bias, as well as copyright and privacy concerns.

Machine Translation NMT +1

Energy Transformer

4 code implementations NeurIPS 2023 Benjamin Hoover, Yuchen Liang, Bao Pham, Rameswar Panda, Hendrik Strobelt, Duen Horng Chau, Mohammed J. Zaki, Dmitry Krotov

Our work combines aspects of three promising paradigms in machine learning, namely, attention mechanism, energy-based models, and associative memory.

Graph Anomaly Detection Graph Classification

Learning to Grow Pretrained Models for Efficient Transformer Training

no code implementations2 Mar 2023 Peihao Wang, Rameswar Panda, Lucas Torroba Hennigen, Philip Greengard, Leonid Karlinsky, Rogerio Feris, David Daniel Cox, Zhangyang Wang, Yoon Kim

Scaling transformers has led to significant breakthroughs in many domains, leading to a paradigm in which larger versions of existing models are trained and released on a periodic basis.

Multitask Prompt Tuning Enables Parameter-Efficient Transfer Learning

no code implementations6 Mar 2023 Zhen Wang, Rameswar Panda, Leonid Karlinsky, Rogerio Feris, Huan Sun, Yoon Kim

Prompt tuning, in which a base pretrained model is adapted to each task via conditioning on learned prompt vectors, has emerged as a promising approach for efficiently adapting large language models to multiple downstream tasks.

Transfer Learning

Going Beyond Nouns With Vision & Language Models Using Synthetic Data

1 code implementation ICCV 2023 Paola Cascante-Bonilla, Khaled Shehada, James Seale Smith, Sivan Doveh, Donghyun Kim, Rameswar Panda, Gül Varol, Aude Oliva, Vicente Ordonez, Rogerio Feris, Leonid Karlinsky

We contribute Synthetic Visual Concepts (SyViC) - a million-scale synthetic dataset and data generation codebase allowing to generate additional suitable data to improve VLC understanding and compositional reasoning of VL models.

Sentence Visual Reasoning

Learning Human Action Recognition Representations Without Real Humans

1 code implementation NeurIPS 2023 Howard Zhong, Samarth Mishra, Donghyun Kim, SouYoung Jin, Rameswar Panda, Hilde Kuehne, Leonid Karlinsky, Venkatesh Saligrama, Aude Oliva, Rogerio Feris

To this end, we present, for the first time, a benchmark that leverages real-world videos with humans removed and synthetic data containing virtual humans to pre-train a model.

Action Recognition Ethics +2

Gated Linear Attention Transformers with Hardware-Efficient Training

2 code implementations11 Dec 2023 Songlin Yang, Bailin Wang, Yikang Shen, Rameswar Panda, Yoon Kim

When used as a replacement for the standard attention layer in Transformers, the resulting gated linear attention (GLA) Transformer is found to perform competitively against the LLaMA-architecture Transformer (Touvron et al., 2023) as well recent linear-time-inference baselines such as RetNet(Sun et al., 2023a) and Mamba (Gu & Dao, 2023) on moderate-scale language modeling experiments.

Language Modelling

Diversity Measurement and Subset Selection for Instruction Tuning Datasets

no code implementations4 Feb 2024 Peiqi Wang, Yikang Shen, Zhen Guo, Matthew Stallone, Yoon Kim, Polina Golland, Rameswar Panda

Our experiments demonstrate that the proposed diversity measure in the normalized weight gradient space is correlated with downstream instruction-following performance.

Instruction Following Point Processes

API Pack: A Massive Multilingual Dataset for API Call Generation

1 code implementation14 Feb 2024 Zhen Guo, Adriana Meza Soria, Wei Sun, Yikang Shen, Rameswar Panda

We introduce API Pack, a multilingual dataset featuring over one million instruction-API call pairs aimed at advancing large language models' API call generation capabilities.

Data Engineering for Scaling Language Models to 128K Context

2 code implementations15 Feb 2024 Yao Fu, Rameswar Panda, Xinyao Niu, Xiang Yue, Hannaneh Hajishirzi, Yoon Kim, Hao Peng

We demonstrate that continual pretraining of the full model on 1B-5B tokens of such data is an effective and affordable strategy for scaling the context length of language models to 128K.

Continual Pretraining

Scattered Mixture-of-Experts Implementation

1 code implementation13 Mar 2024 Shawn Tan, Yikang Shen, Rameswar Panda, Aaron Courville

We present ScatterMoE, an implementation of Sparse Mixture-of-Experts (SMoE) on GPUs.

Cannot find the paper you are looking for? You can Submit a new open access paper.