Search Results for author:
Found 45 papers, 19 papers with code
KPConvX: Modernizing Kernel Point Convolution with Kernel Attention
In the field of deep point cloud understanding, KPConv is a unique architecture that uses kernel points to locate convolutional weights in space, instead of relying on Multi-Layer Perceptron (MLP) encodings.
Ranked #5 on
Semantic Segmentation
on S3DIS Area5
Efficient Non-Parametric Uncertainty Quantification for Black-Box Large Language Models and Decision Planning
This paper focuses on decision planning with uncertainty estimation to address the hallucination problem in language models.
An Empirical Study of Self-supervised Learning with Wasserstein Distance
We find that the model performance depends on the combination of TWD and probability model, and that the Jeffrey divergence regularization helps in model training.
Multimodal Large Language Model for Visual Navigation
Recent efforts to enable visual navigation using large language models have mainly focused on developing complex prompt systems.
Self-Supervised Object Goal Navigation with In-Situ Finetuning
In contrast, our LocCon shows the most robust transfer in the real world among the set of models we compare to, and that the real-world performance of all models can be further improved with self-supervised LocCon in-situ training.
Greedy Modality Selection via Approximate Submodular Maximization
We formulate a theoretical framework for optimizing modality selection in multimodal learning and introduce a utility measure to quantify the benefit of selecting a modality.
Towards Multimodal Multitask Scene Understanding Models for Indoor Mobile Agents
The perception system in personalized mobile agents requires developing indoor scene understanding models, which can understand 3D geometries, capture objectiveness, analyze human behaviors, etc.
Paraphrasing Is All You Need for Novel Object Captioning
Since no ground truth captions are available for novel object images during training, our P2C leverages cross-modality (image-text) association modules to ensure the above caption characteristics can be properly preserved.
Scale dependant layer for self-supervised nuclei encoding
Recent developments in self-supervised learning give us the possibility to further reduce human intervention in multi-step pipelines where the focus evolves around particular objects of interest.
Learning Weakly-Supervised Contrastive Representations
The first stage is to cluster data according to its auxiliary information.
Conditional Contrastive Learning with Kernel
Conditional contrastive learning frameworks consider the conditional sampling procedure that constructs positive or negative data pairs conditioned on specific variables.
Learning Visual-Linguistic Adequacy, Fidelity, and Fluency for Novel Object Captioning
Novel object captioning (NOC) learns image captioning models for describing objects or visual concepts which are unseen (i. e., novel) in the training captions.
HuBERT: Self-Supervised Speech Representation Learning by Masked Prediction of Hidden Units
Self-supervised approaches for speech representation learning are challenged by three unique problems: (1) there are multiple sound units in each input utterance, (2) there is no lexicon of input sound units during the pre-training phase, and (3) sound units have variable lengths with no explicit segmentation.
Ranked #5 on
Speech Recognition
on LibriSpeech test-other
Conditional Contrastive Learning for Improving Fairness in Self-Supervised Learning
In this paper, we propose a Conditional Contrastive Learning (CCL) approach to improve the fairness of contrastive SSL methods.
Integrating Auxiliary Information in Self-supervised Learning
Our approach contributes as follows: 1) Comparing to conventional self-supervised representations, the auxiliary-information-infused self-supervised representations bring the performance closer to the supervised representations; 2) The presented Cl-InfoNCE can also work with unsupervised constructed clusters (e. g., k-means clusters) and outperform strong clustering-based self-supervised learning approaches, such as the Prototypical Contrastive Learning (PCL) method; 3) We show that Cl-InfoNCE may be a better approach to leverage the data clustering information, by comparing it to the baseline approach - learning to predict the clustering assignments with cross-entropy loss.
A Note on Connecting Barlow Twins with Negative-Sample-Free Contrastive Learning
In this report, we relate the algorithmic design of Barlow Twins' method to the Hilbert-Schmidt Independence Criterion (HSIC), thus establishing it as a contrastive learning approach that is free of negative samples.
Self-supervised Representation Learning with Relative Predictive Coding
This paper introduces Relative Predictive Coding (RPC), a new contrastive representation learning objective that maintains a good balance among training stability, minibatch size sensitivity, and downstream task performance.
Feature-Robust Optimal Transport for High-Dimensional Data
To show the effectiveness of FROT, we propose using the FROT algorithm for the layer selection problem in deep neural networks for semantic correspondence.
Unsupervised Domain Adaptation for Visual Navigation
We further show that our method can be used to transfer the navigation policies learned in simulation to the real world.
Self-supervised Learning from a Multi-view Perspective
In particular, we propose a composite objective that bridges the gap between prior contrastive and predictive learning objectives, and introduce an additional objective term to discard task-irrelevant information.
Neural Methods for Point-wise Dependency Estimation
Since its inception, the neural estimation of mutual information (MI) has demonstrated the empirical success of modeling expected dependency between high-dimensional random variables.
Feature Robust Optimal Transport for High-dimensional Data
To show the effectiveness of FROT, we propose using the FROT algorithm for the layer selection problem in deep neural networks for semantic correspondence.
Multimodal Routing: Improving Local and Global Interpretability of Multimodal Language Analysis
The human language can be expressed through multiple sources of information known as modalities, including tones of voice, facial gestures, and spoken language.
Capsules with Inverted Dot-Product Attention Routing
We introduce a new routing algorithm for capsule networks, in which a child capsule is routed to a parent based only on agreement between the parent's state and the child's vote.
Transformer Dissection: An Unified Understanding for Transformer's Attention via the Lens of Kernel
This new formulation gives us a better way to understand individual components of the Transformer{'}s attention, such as the better way to integrate the positional embedding.
Complex Transformer: A Framework for Modeling Complex-Valued Sequence
While deep learning has received a surge of interest in a variety of fields in recent years, major deep learning models barely use complex numbers.
Ranked #2 on
Music Transcription
on MusicNet
LSMI-Sinkhorn: Semi-supervised Mutual Information Estimation with Optimal Transport
To estimate the mutual information from data, a common practice is preparing a set of paired samples $\{(\mathbf{x}_i,\mathbf{y}_i)\}_{i=1}^n \stackrel{\mathrm{i. i. d.
Transformer Dissection: A Unified Understanding of Transformer's Attention via the Lens of Kernel
This new formulation gives us a better way to understand individual components of the Transformer's attention, such as the better way to integrate the positional embedding.
Learning Neural Networks with Adaptive Regularization
Feed-forward neural networks can be understood as a combination of an intermediate representation and a linear hypothesis.
Learning Representations from Imperfect Time Series Data via Tensor Rank Regularization
Our method is based on the observation that high-dimensional multimodal time series data often exhibit correlations across time and modalities which leads to low-rank tensor representations.
Multimodal Transformer for Unaligned Multimodal Language Sequences
Human language is often multimodal, which comprehends a mixture of natural language, facial gestures, and acoustic behaviors.
Ranked #6 on
Multimodal Sentiment Analysis
on MOSI
Strong and Simple Baselines for Multimodal Utterance Embeddings
Human language is a rich multimodal signal consisting of spoken words, facial expressions, body gestures, and vocal intonations.
Learning From the Experience of Others: Approximate Empirical Bayes in Neural Networks
Learning deep neural networks could be understood as the combination of representation learning and learning halfspaces.
Video Relationship Reasoning using Gated Spatio-Temporal Energy Graph
Visual relationship reasoning is a crucial yet challenging task for understanding rich interactions across visual concepts.
Learning Factorized Multimodal Representations
Multimodal discriminative factors are shared across all modalities and contain joint multimodal features required for discriminative tasks such as sentiment prediction.
Post Selection Inference with Incomplete Maximum Mean Discrepancy Estimator
In the paper, we propose a post selection inference (PSI) framework for divergence measure, which can select a set of statistically significant features that discriminate two distributions.
Selecting the Best in GANs Family: a Post Selection Inference Framework
"Which Generative Adversarial Networks (GANs) generates the most plausible images?"
"Dependency Bottleneck" in Auto-encoding Architectures: an Empirical Study
To address this issue, we propose to measure the dependency instead of MI between layers in DNNs.
Discovering Order in Unordered Datasets: Generative Markov Networks
The assumption that data samples are independently identically distributed is the backbone of many learning algorithms.
Learning Markov Chain in Unordered Dataset
In this technical report, we introduce OrderNet that can be used to extract the order of data instances in an unsupervised way.
Improving One-Shot Learning through Fusing Side Information
We introduce two statistical approaches for fusing side information into data representation learning to improve one-shot learning.
Generative-Discriminative Variational Model for Visual Recognition
In other words, our GDVM casts the supervised learning task as a generative learning process, with data discrimination to be jointly exploited for improved classification.
Learning Robust Visual-Semantic Embeddings
Many of the existing methods for learning joint embedding of images and text use only supervised information from paired images and its textual attributes.
Ranked #5 on
Generalized Few-Shot Learning
on CUB
Learning Cross-Domain Landmarks for Heterogeneous Domain Adaptation
With the goal of deriving a domain-invariant feature subspace for HDA, our CDLS is able to identify representative cross-domain data, including the unlabeled ones in the target domain, for performing adaptation.
Unsupervised Domain Adaptation With Imbalanced Cross-Domain Data
For standard unsupervised domain adaptation, one typically obtains labeled data in the source domain and only observes unlabeled data in the target domain.
