Search Results for author:
Found 44 papers, 32 papers with code
Towards Biologically Plausible Deep Learning
Neuroscientists have long criticised deep learning algorithms as incompatible with current knowledge of neurobiology.
Neural Networks with Few Multiplications
For most deep learning algorithms training is notoriously time consuming.
How far can we go without convolution: Improving fully-connected networks
We propose ways to improve the performance of fully connected networks.
Spectral-Spatial Classification of Hyperspectral Image Using Autoencoders
Hyperspectral image (HSI) classification is a hot topic in the remote sensing community.
Architectural Complexity Measures of Recurrent Neural Networks
In this paper, we systematically analyze the connecting architectures of recurrent neural networks (RNNs).
Ranked #23 on
Language Modelling
on Text8
Theano: A Python framework for fast computation of mathematical expressions
Since its introduction, it has been one of the most used CPU and GPU mathematical compilers - especially in the machine learning community - and has shown steady performance improvements.
Recurrent Neural Networks With Limited Numerical Precision
We present results from the use of different stochastic and deterministic reduced precision training methods applied to three major RNN types which are then tested on several datasets.
Recurrent Neural Networks With Limited Numerical Precision
Recurrent Neural Networks (RNNs) produce state-of-art performance on many machine learning tasks but their demand on resources in terms of memory and computational power are often high.
A Structured Self-attentive Sentence Embedding
This paper proposes a new model for extracting an interpretable sentence embedding by introducing self-attention.
A Deep Reinforcement Learning Chatbot
By applying reinforcement learning to crowdsourced data and real-world user interactions, the system has been trained to select an appropriate response from the models in its ensemble.
Neural Language Modeling by Jointly Learning Syntax and Lexicon
In this paper, We propose a novel neural language model, called the Parsing-Reading-Predict Networks (PRPN), that can simultaneously induce the syntactic structure from unannotated sentences and leverage the inferred structure to learn a better language model.
Ranked #13 on
Constituency Grammar Induction
on PTB Diagnostic ECG Database
(Max F1 (WSJ) metric)
A Deep Reinforcement Learning Chatbot (Short Version)
We present MILABOT: a deep reinforcement learning chatbot developed by the Montreal Institute for Learning Algorithms (MILA) for the Amazon Alexa Prize competition.
Straight to the Tree: Constituency Parsing with Neural Syntactic Distance
In this work, we propose a novel constituency parsing scheme.
Focused Hierarchical RNNs for Conditional Sequence Processing
We evaluate this method on several types of tasks with different attributes.
Ranked #3 on
Open-Domain Question Answering
on SearchQA
(Unigram Acc metric)
Learning Hierarchical Structures On-The-Fly with a Recurrent-Recursive Model for Sequences
We propose a hierarchical model for sequential data that learns a tree on-the-fly, i. e. while reading the sequence.
Interactive Language Learning by Question Answering
In QAit, an agent must interact with a partially observable text-based environment to gather information required to answer questions.
Ordered Memory
Inspired by Ordered Neurons (Shen et al., 2018), we introduce a new attention-based mechanism and use its cumulative probability to control the writing and erasing operation of the memory.
Revisit Systematic Generalization via Meaningful Learning
Humans can systematically generalize to novel compositions of existing concepts.
Exploiting Syntactic Structure for Better Language Modeling: A Syntactic Distance Approach
It is commonly believed that knowledge of syntactic structure should improve language modeling.
Reciprocal Supervised Learning Improves Neural Machine Translation
Despite the recent success on image classification, self-training has only achieved limited gains on structured prediction tasks such as neural machine translation (NMT).
Annotation Inconsistency and Entity Bias in MultiWOZ
In this work, we identify an overlooked issue with dialog state annotation inconsistencies in the dataset, where a slot type is tagged inconsistently across similar dialogs leading to confusion for DST modeling.
Incorporating External POS Tagger for Punctuation Restoration
Punctuation restoration is an important post-processing step in automatic speech recognition.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+4
Block-Skim: Efficient Question Answering for Transformer
We further prune the hidden states corresponding to the unnecessary positions early in lower layers, achieving significant inference-time speedup.
Leveraging Unimodal Self-Supervised Learning for Multimodal Audio-Visual Speech Recognition
In particular, audio and visual front-ends are trained on large-scale unimodal datasets, then we integrate components of both front-ends into a larger multimodal framework which learns to recognize parallel audio-visual data into characters through a combination of CTC and seq2seq decoding.
Ranked #2 on
Automatic Speech Recognition (ASR)
on LRS2
Audio-Visual Speech Recognition
Automatic Speech Recognition (ASR)
+7
RASAT: Integrating Relational Structures into Pretrained Seq2Seq Model for Text-to-SQL
Our model can incorporate almost all types of existing relations in the literature, and in addition, we propose introducing co-reference relations for the multi-turn scenario.
Ranked #1 on
Dialogue State Tracking
on CoSQL
Transkimmer: Transformer Learns to Layer-wise Skim
To address the above limitations, we propose the Transkimmer architecture, which learns to identify hidden state tokens that are not required by each layer.
INFINITY: A Simple Yet Effective Unsupervised Framework for Graph-Text Mutual Conversion
As a fully unsupervised framework, INFINITY is empirically verified to outperform state-of-the-art baselines for G2T and T2G tasks.
Text Editing as Imitation Game
Text editing, such as grammatical error correction, arises naturally from imperfect textual data.
Syntax-guided Localized Self-attention by Constituency Syntactic Distance
Recent works have revealed that Transformers are implicitly learning the syntactic information in its lower layers from data, albeit is highly dependent on the quality and scale of the training data.
Ordered GNN: Ordering Message Passing to Deal with Heterophily and Over-smoothing
This is achieved by aligning the hierarchy of the rooted-tree of a central node with the ordered neurons in its node representation.
Ranked #4 on
Node Classification
on Cornell
Few-Shot Table-to-Text Generation with Prompt Planning and Knowledge Memorization
The design of our framework consists of two aspects: a prompt planner and a knowledge adapter.
Text Classification in the Wild: a Large-scale Long-tailed Name Normalization Dataset
In this work, we first collect a large-scale institution name normalization dataset LoT-insts1, which contains over 25k classes that exhibit a naturally long-tailed distribution.
Ranked #1 on
Long-tail Learning
on Lot-insts
Asymmetric Polynomial Loss For Multi-Label Classification
We further employ the asymmetric focusing mechanism to decouple the gradient contribution from the negative and positive samples.
PaD: Program-aided Distillation Can Teach Small Models Reasoning Better than Chain-of-thought Fine-tuning
While large language models (LLMs) excel in various natural language processing tasks, their huge size and the inaccessibility of parameters present challenges for practical deployment.
Fourier Transformer: Fast Long Range Modeling by Removing Sequence Redundancy with FFT Operator
Many researchers have focused on designing new forms of self-attention or introducing new parameters to overcome this limitation, however a large portion of them prohibits the model to inherit weights from large pretrained models.
Ranked #1 on
Open-Domain Question Answering
on ELI5
K2: A Foundation Language Model for Geoscience Knowledge Understanding and Utilization
Large language models (LLMs) have achieved great success in general domains of natural language processing.
Document-Level Relation Extraction with Relation Correlation Enhancement
Document-level relation extraction (DocRE) is a task that focuses on identifying relations between entities within a document.
I2SRM: Intra- and Inter-Sample Relationship Modeling for Multimodal Information Extraction
Multimodal information extraction is attracting research attention nowadays, which requires aggregating representations from different modalities.
Human-Readable Fingerprint for Large Language Models
However, identifying the original base model of an LLM is challenging due to potential parameter alterations.
Towards Controlled Table-to-Text Generation with Scientific Reasoning
The sheer volume of scientific experimental results and complex technical statements, often presented in tabular formats, presents a formidable barrier to individuals acquiring preferred information.
GeoGalactica: A Scientific Large Language Model in Geoscience
To our best knowledge, it is the largest language model for the geoscience domain.
SH2: Self-Highlighted Hesitation Helps You Decode More Truthfully
Therefore, we propose to ''highlight'' the factual information by selecting the tokens with the lowest probabilities and concatenating them to the original context, thus forcing the model to repeatedly read and hesitate on these tokens before generation.
Critical Data Size of Language Models from a Grokking Perspective
We formalize the phase transition under the grokking configuration into the Data Efficiency Hypothesis and identify data insufficiency, sufficiency, and surplus regimes in language models training dynamics.
Graph Parsing Networks
GPN benefits from the discrete assignments generated by the graph parsing algorithm, allowing good memory efficiency while preserving node information intact.
