Search Results for author:
Found 51 papers, 37 papers with code
Cluster-wise Graph Transformer with Dual-granularity Kernelized Attention
In the realm of graph learning, there is a category of methods that conceptualize graphs as hierarchical structures, utilizing node clustering to capture broader structural information.
Enhancing SPARQL Generation by Triplet-order-sensitive Pre-training
Semantic parsing that translates natural language queries to SPARQL is of great importance for Knowledge Graph Question Answering (KGQA) systems.
Training-free LLM-generated Text Detection by Mining Token Probability Sequences
Despite this, existing training-free detection methods typically rely on global text sequence statistics, neglecting the modeling of local discriminative features, thereby limiting their detection efficacy.
Leveraging Grammar Induction for Language Understanding and Generation
Grammar induction has made significant progress in recent years.
Mirror-Consistency: Harnessing Inconsistency in Majority Voting
Self-Consistency, a widely-used decoding strategy, significantly boosts the reasoning capabilities of Large Language Models (LLMs).
Autoregressive Enzyme Function Prediction with Multi-scale Multi-modality Fusion
Accurate prediction of enzyme function is crucial for elucidating biological mechanisms and driving innovation across various sectors.
Graph Parsing Networks
GPN benefits from the discrete assignments generated by the graph parsing algorithm, allowing good memory efficiency while preserving node information intact.
Critical Data Size of Language Models from a Grokking Perspective
We formalize the phase transition under the grokking configuration into the Data Efficiency Hypothesis and identify data insufficiency, sufficiency, and surplus regimes in language models training dynamics.
SH2: Self-Highlighted Hesitation Helps You Decode More Truthfully
Therefore, we propose to ''highlight'' the factual information by selecting the tokens with the lowest probabilities and concatenating them to the original context, thus forcing the model to repeatedly read and hesitate on these tokens before generation.
GeoGalactica: A Scientific Large Language Model in Geoscience
To our best knowledge, it is the largest language model for the geoscience domain.
Towards Controlled Table-to-Text Generation with Scientific Reasoning
The sheer volume of scientific experimental results and complex technical statements, often presented in tabular formats, presents a formidable barrier to individuals acquiring preferred information.
HuRef: HUman-REadable Fingerprint for Large Language Models
In this study, we introduce HuRef, a human-readable fingerprint for LLMs that uniquely identifies the base model without interfering with training or exposing model parameters to the public.
I2SRM: Intra- and Inter-Sample Relationship Modeling for Multimodal Information Extraction
Multimodal information extraction is attracting research attention nowadays, which requires aggregating representations from different modalities.
Tailoring Self-Attention for Graph via Rooted Subtrees
Attention mechanisms have made significant strides in graph learning, yet they still exhibit notable limitations: local attention faces challenges in capturing long-range information due to the inherent problems of the message-passing scheme, while global attention cannot reflect the hierarchical neighborhood structure and fails to capture fine-grained local information.
Document-Level Relation Extraction with Relation Correlation Enhancement
Document-level relation extraction (DocRE) is a task that focuses on identifying relations between entities within a document.
K2: A Foundation Language Model for Geoscience Knowledge Understanding and Utilization
Large language models (LLMs) have achieved great success in general domains of natural language processing.
Fourier Transformer: Fast Long Range Modeling by Removing Sequence Redundancy with FFT Operator
Many researchers have focused on designing new forms of self-attention or introducing new parameters to overcome this limitation, however a large portion of them prohibits the model to inherit weights from large pretrained models.
Ranked #1 on
Open-Domain Question Answering
on ELI5
PaD: Program-aided Distillation Can Teach Small Models Reasoning Better than Chain-of-thought Fine-tuning
While large language models (LLMs) excel in various natural language processing tasks, their huge size and the inaccessibility of parameters present challenges for practical deployment.
Asymmetric Polynomial Loss For Multi-Label Classification
We further employ the asymmetric focusing mechanism to decouple the gradient contribution from the negative and positive samples.
Text Classification in the Wild: a Large-scale Long-tailed Name Normalization Dataset
In this work, we first collect a large-scale institution name normalization dataset LoT-insts1, which contains over 25k classes that exhibit a naturally long-tailed distribution.
Ranked #1 on
Long-tail Learning
on Lot-insts
Few-Shot Table-to-Text Generation with Prompt Planning and Knowledge Memorization
The design of our framework consists of two aspects: a prompt planner and a knowledge adapter.
Ordered GNN: Ordering Message Passing to Deal with Heterophily and Over-smoothing
This is achieved by aligning the hierarchy of the rooted-tree of a central node with the ordered neurons in its node representation.
Ranked #5 on
Node Classification
on Cornell
Syntax-guided Localized Self-attention by Constituency Syntactic Distance
Recent works have revealed that Transformers are implicitly learning the syntactic information in its lower layers from data, albeit is highly dependent on the quality and scale of the training data.
Text Editing as Imitation Game
Text editing, such as grammatical error correction, arises naturally from imperfect textual data.
INFINITY: A Simple Yet Effective Unsupervised Framework for Graph-Text Mutual Conversion
As a fully unsupervised framework, INFINITY is empirically verified to outperform state-of-the-art baselines for G2T and T2G tasks.
Transkimmer: Transformer Learns to Layer-wise Skim
To address the above limitations, we propose the Transkimmer architecture, which learns to identify hidden state tokens that are not required by each layer.
RASAT: Integrating Relational Structures into Pretrained Seq2Seq Model for Text-to-SQL
Our model can incorporate almost all types of existing relations in the literature, and in addition, we propose introducing co-reference relations for the multi-turn scenario.
Ranked #1 on
Dialogue State Tracking
on CoSQL
Leveraging Unimodal Self-Supervised Learning for Multimodal Audio-Visual Speech Recognition
In particular, audio and visual front-ends are trained on large-scale unimodal datasets, then we integrate components of both front-ends into a larger multimodal framework which learns to recognize parallel audio-visual data into characters through a combination of CTC and seq2seq decoding.
Ranked #3 on
Automatic Speech Recognition (ASR)
on LRS2
Audio-Visual Speech Recognition
Automatic Speech Recognition (ASR)
+7
Block-Skim: Efficient Question Answering for Transformer
We further prune the hidden states corresponding to the unnecessary positions early in lower layers, achieving significant inference-time speedup.
Incorporating External POS Tagger for Punctuation Restoration
Punctuation restoration is an important post-processing step in automatic speech recognition.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+4
Annotation Inconsistency and Entity Bias in MultiWOZ
In this work, we identify an overlooked issue with dialog state annotation inconsistencies in the dataset, where a slot type is tagged inconsistently across similar dialogs leading to confusion for DST modeling.
Reciprocal Supervised Learning Improves Neural Machine Translation
Despite the recent success on image classification, self-training has only achieved limited gains on structured prediction tasks such as neural machine translation (NMT).
Exploiting Syntactic Structure for Better Language Modeling: A Syntactic Distance Approach
It is commonly believed that knowledge of syntactic structure should improve language modeling.
Revisit Systematic Generalization via Meaningful Learning
Humans can systematically generalize to novel compositions of existing concepts.
Ordered Memory
Inspired by Ordered Neurons (Shen et al., 2018), we introduce a new attention-based mechanism and use its cumulative probability to control the writing and erasing operation of the memory.
Interactive Language Learning by Question Answering
In QAit, an agent must interact with a partially observable text-based environment to gather information required to answer questions.
Deep Reinforcement Learning
Machine Reading Comprehension
+1
Learning Hierarchical Structures On-The-Fly with a Recurrent-Recursive Model for Sequences
We propose a hierarchical model for sequential data that learns a tree on-the-fly, i. e. while reading the sequence.
Focused Hierarchical RNNs for Conditional Sequence Processing
We evaluate this method on several types of tasks with different attributes.
Ranked #3 on
Open-Domain Question Answering
on SearchQA
(Unigram Acc metric)
Straight to the Tree: Constituency Parsing with Neural Syntactic Distance
In this work, we propose a novel constituency parsing scheme.
A Deep Reinforcement Learning Chatbot (Short Version)
We present MILABOT: a deep reinforcement learning chatbot developed by the Montreal Institute for Learning Algorithms (MILA) for the Amazon Alexa Prize competition.
Neural Language Modeling by Jointly Learning Syntax and Lexicon
In this paper, We propose a novel neural language model, called the Parsing-Reading-Predict Networks (PRPN), that can simultaneously induce the syntactic structure from unannotated sentences and leverage the inferred structure to learn a better language model.
Ranked #16 on
Constituency Grammar Induction
on PTB Diagnostic ECG Database
(Max F1 (WSJ) metric)
A Deep Reinforcement Learning Chatbot
By applying reinforcement learning to crowdsourced data and real-world user interactions, the system has been trained to select an appropriate response from the models in its ensemble.
A Structured Self-attentive Sentence Embedding
This paper proposes a new model for extracting an interpretable sentence embedding by introducing self-attention.
Recurrent Neural Networks With Limited Numerical Precision
Recurrent Neural Networks (RNNs) produce state-of-art performance on many machine learning tasks but their demand on resources in terms of memory and computational power are often high.
Recurrent Neural Networks With Limited Numerical Precision
We present results from the use of different stochastic and deterministic reduced precision training methods applied to three major RNN types which are then tested on several datasets.
Theano: A Python framework for fast computation of mathematical expressions
Since its introduction, it has been one of the most used CPU and GPU mathematical compilers - especially in the machine learning community - and has shown steady performance improvements.
Architectural Complexity Measures of Recurrent Neural Networks
In this paper, we systematically analyze the connecting architectures of recurrent neural networks (RNNs).
Ranked #23 on
Language Modelling
on Text8
Spectral-Spatial Classification of Hyperspectral Image Using Autoencoders
Hyperspectral image (HSI) classification is a hot topic in the remote sensing community.
How far can we go without convolution: Improving fully-connected networks
We propose ways to improve the performance of fully connected networks.
Neural Networks with Few Multiplications
For most deep learning algorithms training is notoriously time consuming.
Towards Biologically Plausible Deep Learning
Neuroscientists have long criticised deep learning algorithms as incompatible with current knowledge of neurobiology.
