Search Results for author:
Found 34 papers, 20 papers with code
Audio-Visual Neural Syntax Acquisition
We study phrase structure induction from visually-grounded speech.
SALMON: Self-Alignment with Instructable Reward Models
Supervised Fine-Tuning (SFT) on response demonstrations combined with Reinforcement Learning from Human Feedback (RLHF) constitutes a powerful paradigm for aligning LLM-based AI agents.
Self-Specialization: Uncovering Latent Expertise within Large Language Models
Recent works have demonstrated the effectiveness of self-alignment in which a large language model is, by itself, aligned to follow general instructions through the automatic generation of instructional data using a handful of human-written seeds.
Principle-Driven Self-Alignment of Language Models from Scratch with Minimal Human Supervision
Recent AI-assistant agents, such as ChatGPT, predominantly rely on supervised fine-tuning (SFT) with human annotations and reinforcement learning from human feedback (RLHF) to align the output of large language models (LLMs) with human intentions, ensuring they are helpful, ethical, and reliable.
Rapid Development of Compositional AI
Compositional AI systems, which combine multiple artificial intelligence components together with other application components to solve a larger problem, have no known pattern of development and are often approached in a bespoke and ad hoc style.
ConStruct-VL: Data-Free Continual Structured VL Concepts Learning
This leads to reasoning mistakes, which need to be corrected as they occur by teaching VL models the missing SVLC skills; often this must be done using private data where the issue was found, which naturally leads to a data-free continual (no task-id) VL learning setting.
VALHALLA: Visual Hallucination for Machine Translation
In particular, given a source sentence an autoregressive hallucination transformer is used to predict a discrete visual representation from the input text, and the combined text and hallucinated representations are utilized to obtain the target translation.
ContentVec: An Improved Self-Supervised Speech Representation by Disentangling Speakers
Self-supervised learning in speech involves training a speech representation network on a large-scale unannotated speech corpus, and then applying the learned representations to downstream tasks.
Drawing Robust Scratch Tickets: Subnetworks with Inborn Robustness Are Found within Randomly Initialized Networks
Deep Neural Networks (DNNs) are known to be vulnerable to adversarial attacks, i. e., an imperceptible perturbation to the input can mislead DNNs trained on clean images into making erroneous predictions.
On the Interplay Between Sparsity, Naturalness, Intelligibility, and Prosody in Speech Synthesis
Are end-to-end text-to-speech (TTS) models over-parametrized?
Global Rhythm Style Transfer Without Text Transcriptions
In this paper, we propose AutoPST, which can disentangle global prosody style from speech without relying on any text transcriptions.
Auto-NBA: Efficient and Effective Search Over the Joint Space of Networks, Bitwidths, and Accelerators
The key challenges include (1) the dilemma of whether to explode the memory consumption due to the huge joint space or achieve sub-optimal designs, (2) the discrete nature of the accelerator design space that is coupled yet different from that of the networks and bitwidths, and (3) the chicken and egg problem associated with network-accelerator co-search, i. e., co-search requires operation-wise hardware cost, which is lacking during search as the optimal accelerator depending on the whole network is still unknown during search.
PARP: Prune, Adjust and Re-Prune for Self-Supervised Speech Recognition
We investigate the existence of sparse subnetworks in pre-trained speech SSL models that achieve even better low-resource ASR results.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+3
Simulating a Primary Visual Cortex at the Front of CNNs Improves Robustness to Image Perturbations
Current state-of-the-art object recognition models are largely based on convolutional neural network (CNN) architectures, which are loosely inspired by the primate visual system.
Lifelong Object Detection
Recent advances in object detection have benefited significantly from rapid developments in deep neural networks.
ThreeDWorld: A Platform for Interactive Multi-Modal Physical Simulation
We introduce ThreeDWorld (TDW), a platform for interactive multi-modal physical simulation.
Unsupervised Speech Decomposition via Triple Information Bottleneck
Speech information can be roughly decomposed into four components: language content, timbre, pitch, and rhythm.
Joint Visual-Temporal Embedding for Unsupervised Learning of Actions in Untrimmed Sequences
Understanding the structure of complex activities in untrimmed videos is a challenging task in the area of action recognition.
More Is Less: Learning Efficient Video Representations by Big-Little Network and Depthwise Temporal Aggregation
Current state-of-the-art models for video action recognition are mostly based on expensive 3D ConvNets.
Ranked #89 on
Action Recognition
on Something-Something V2
(using extra training data)
Self-supervised Moving Vehicle Tracking with Stereo Sound
At test time, the stereo-sound student network can work independently to perform object localization us-ing just stereo audio and camera meta-data, without any visual input.
ZO-AdaMM: Zeroth-Order Adaptive Momentum Method for Black-Box Optimization
In this paper, we propose a zeroth-order AdaMM (ZO-AdaMM) algorithm, that generalizes AdaMM to the gradient-free regime.
Triton: An Intermediate Language and Compiler for Tiled Neural Network Computations
The validation and deployment of novel research ideas in the field of Deep Learning is often limited by the availability of efficient compute kernels for certain basic primitives.
Conditional Infilling GANs for Data Augmentation in Mammogram Classification
Deep learning approaches to breast cancer detection in mammograms have recently shown promising results.
A neural network trained to predict future video frames mimics critical properties of biological neuronal responses and perception
Interestingly, recent work has shown that deep convolutional neural networks (CNNs) trained on large-scale image recognition tasks can serve as strikingly good models for predicting the responses of neurons in visual cortex to visual stimuli, suggesting that analogies between artificial and biological neural networks may be more than superficial.
Learning Nonlinear Brain Dynamics: van der Pol Meets LSTM
Many real-world data sets, especially in biology, are produced by complex nonlinear dynamical systems.
A Multi-Scale CNN and Curriculum Learning Strategy for Mammogram Classification
Screening mammography is an important front-line tool for the early detection of breast cancer, and some 39 million exams are conducted each year in the United States alone.
Recurrent computations for visual pattern completion
First, subjects robustly recognized objects even when rendered <15% visible, but recognition was largely impaired when processing was interrupted by backward masking.
Using Human Brain Activity to Guide Machine Learning
The effectiveness of this approach points to a path forward for a new class of hybrid machine learning algorithms which take both inspiration and direct constraints from neuronal data.
Tensor Switching Networks
We present a novel neural network algorithm, the Tensor Switching (TS) network, which generalizes the Rectified Linear Unit (ReLU) nonlinearity to tensor-valued hidden units.
Delta Epsilon Alpha Star: A PAC-Admissible Search Algorithm
Delta Epsilon Alpha Star is a minimal coverage, real-time robotic search algorithm that yields a moderately aggressive search path with minimal backtracking.
Syntactically Informed Text Compression with Recurrent Neural Networks
We present a self-contained system for constructing natural language models for use in text compression.
Deep Predictive Coding Networks for Video Prediction and Unsupervised Learning
Here, we explore prediction of future frames in a video sequence as an unsupervised learning rule for learning about the structure of the visual world.
Unsupervised Learning of Visual Structure using Predictive Generative Networks
The ability to predict future states of the environment is a central pillar of intelligence.
Large-Scale Optimization of Hierarchical Features for Saliency Prediction in Natural Images
Our models outperform the state of the art on MIT1003, on which features and classifiers are learned.
