Search Results for author:
Found 132 papers, 80 papers with code
Developing and Utilizing a Large-Scale Cantonese Dataset for Multi-Tasking in Large Language Models
After training on our dataset, the model also exhibits improved performance on other mainstream language tasks.
Haste Makes Waste: Evaluating Planning Abilities of LLMs for Efficient and Feasible Multitasking with Time Constraints Between Actions
While Large Language Model-based agents have demonstrated substantial progress in task completion, existing evaluation benchmarks tend to overemphasize single-task performance, with insufficient attention given to the crucial aspects of multitask planning and execution efficiency required in real-world scenarios.
Implicit Search via Discrete Diffusion: A Study on Chess
Furthermore, DiffuSearch demonstrates a notable 30% enhancement in puzzle-solving abilities compared to explicit search-based policies, along with a significant 540 Elo increase in game-playing strength assessment.
Reasoning Does Not Necessarily Improve Role-Playing Ability
The application of role-playing large language models (LLMs) is rapidly expanding in both academic and commercial domains, driving an increasing demand for high-precision role-playing models.
BioMaze: Benchmarking and Enhancing Large Language Models for Biological Pathway Reasoning
The applications of large language models (LLMs) in various biological domains have been explored recently, but their reasoning ability in complex biological systems, such as pathways, remains underexplored, which is crucial for predicting biological phenomena, formulating hypotheses, and designing experiments.
ParallelComp: Parallel Long-Context Compressor for Length Extrapolation
To mitigate the attention sink issue, we propose an attention calibration strategy that reduces biases, ensuring more stable long-range attention.
A Survey on Large Language Model-Based Social Agents in Game-Theoretic Scenarios
Our survey organizes the findings into three core components: Game Framework, Social Agent, and Evaluation Protocol.
VLRewardBench: A Challenging Benchmark for Vision-Language Generative Reward Models
Vision-language generative reward models (VL-GenRMs) play a crucial role in aligning and evaluating multimodal AI systems, yet their own evaluation remains under-explored.
Autoregressive Models in Vision: A Survey
Autoregressive modeling has been a huge success in the field of natural language processing (NLP).
Why Does the Effective Context Length of LLMs Fall Short?
Advancements in distributed training and efficient attention mechanisms have significantly expanded the context window sizes of large language models (LLMs).
Scaling Diffusion Language Models via Adaptation from Autoregressive Models
Diffusion Language Models (DLMs) have emerged as a promising new paradigm for text generative modeling, potentially addressing limitations of autoregressive (AR) models.
Forewarned is Forearmed: Leveraging LLMs for Data Synthesis through Failure-Inducing Exploration
In this paper, we present a novel approach, ReverseGen, designed to automatically generate effective training samples that expose the weaknesses of LLMs.
Non-myopic Generation of Language Models for Reasoning and Planning
Large Language Models have demonstrated remarkable abilities in reasoning and planning by breaking down complex problems into sequential steps.
Beyond Autoregression: Discrete Diffusion for Complex Reasoning and Planning
Our work highlights the potential of diffusion-based approaches in advancing AI capabilities for sophisticated language understanding and problem-solving tasks.
ProReason: Multi-Modal Proactive Reasoning with Decoupled Eyesight and Wisdom
Notably, the disassociation of capabilities allows seamless integration of existing large language models (LLMs) to compensate for the reasoning deficits of LVLMs.
Understanding the Role of LLMs in Multimodal Evaluation Benchmarks
The rapid advancement of Multimodal Large Language Models (MLLMs) has been accompanied by the development of various benchmarks to evaluate their capabilities.
QSpec: Speculative Decoding with Complementary Quantization Schemes
Compared to high-precision quantization methods, QSPEC empirically boosts token generation throughput by up to 1. 64x without any quality compromise, distinguishing it from other low-precision quantization approaches.
TMGBench: A Systematic Game Benchmark for Evaluating Strategic Reasoning Abilities of LLMs
The rapid advancement of large language models (LLMs) has accelerated their application in reasoning, with strategic reasoning drawing increasing attention.
VLFeedback: A Large-Scale AI Feedback Dataset for Large Vision-Language Models Alignment
As large vision-language models (LVLMs) evolve rapidly, the demand for high-quality and diverse data to align these models becomes increasingly crucial.
Ranked #62 on
Visual Question Answering
on MM-Vet
Temporal Reasoning Transfer from Text to Video
Video Large Language Models (Video LLMs) have shown promising capabilities in video comprehension, yet they struggle with tracking temporal changes and reasoning about temporal relationships.
UNComp: Uncertainty-Aware Long-Context Compressor for Efficient Large Language Model Inference
By grouping layers and heads based on their uncertainty, UNComp adaptively compresses both the hidden states and the KV cache.
UncertaintyRAG: Span-Level Uncertainty Enhanced Long-Context Modeling for Retrieval-Augmented Generation
We present UncertaintyRAG, a novel approach for long-context Retrieval-Augmented Generation (RAG) that utilizes Signal-to-Noise Ratio (SNR)-based span uncertainty to estimate similarity between text chunks.
MoS: Unleashing Parameter Efficiency of Low-Rank Adaptation with Mixture of Shards
The rapid scaling of large language models necessitates more lightweight finetuning methods to reduce the explosive GPU memory overhead when numerous customized models are served simultaneously.
CoCA: Regaining Safety-awareness of Multimodal Large Language Models with Constitutional Calibration
The deployment of multimodal large language models (MLLMs) has demonstrated remarkable success in engaging in conversations involving visual inputs, thanks to the superior power of large language models (LLMs).
How Well Do LLMs Handle Cantonese? Benchmarking Cantonese Capabilities of Large Language Models
The rapid evolution of large language models (LLMs) has transformed the competitive landscape in natural language processing (NLP), particularly for English and other data-rich languages.
SubgoalXL: Subgoal-based Expert Learning for Theorem Proving
This paper introduces SubgoalXL, a novel approach that synergizes subgoal-based proofs with expert learning to enhance LLMs' capabilities in formal theorem proving within the Isabelle environment.
Ranked #3 on
Automated Theorem Proving
on miniF2F-test
(using extra training data)
FACTTRACK: Time-Aware World State Tracking in Story Outlines
Moreover, when using GPT4, FACTTRACK significantly outperforms the GPT4 baseline.
Data Augmentation of Multi-turn Psychological Dialogue via Knowledge-driven Progressive Thought Prompting
The thought generated by the progressive thought generator serves as a prompt to prevent the generated dialogue from having significant semantic deviations, while the psychology knowledge generator produces psychological knowledge to serve as the dialogue history for the LLM, guiding the dialogue generator to create multi-turn psychological dialogue.
Jailbreaking as a Reward Misspecification Problem
In this paper, we propose a novel perspective that attributes this vulnerability to reward misspecification during the alignment process.
A Survey of Neural Code Intelligence: Paradigms, Advances and Beyond
Building on our examination of the developmental trajectories, we further investigate the emerging synergies between code intelligence and broader machine intelligence, uncovering new cross-domain opportunities and illustrating the substantial influence of code intelligence across various domains.
ImgTrojan: Jailbreaking Vision-Language Models with ONE Image
In this paper, we propose a novel jailbreaking attack against VLMs, aiming to bypass their safety barrier when a user inputs harmful instructions.
Multimodal ArXiv: A Dataset for Improving Scientific Comprehension of Large Vision-Language Models
To fill this gap, we introduce Multimodal ArXiv, consisting of ArXivCap and ArXivQA, for enhancing LVLMs scientific comprehension.
GSM-Plus: A Comprehensive Benchmark for Evaluating the Robustness of LLMs as Mathematical Problem Solvers
Large language models (LLMs) have achieved impressive performance across various mathematical reasoning benchmarks.
Ranked #1 on
Math Word Problem Solving
on GSM-Plus
Training-Free Long-Context Scaling of Large Language Models
The ability of Large Language Models (LLMs) to process and generate coherent text is markedly weakened when the number of input tokens exceeds their pretraining length.
LoRA Meets Dropout under a Unified Framework
Hence, a possible contradiction arises from negligible trainable parameters of LoRA and the effectiveness of previous dropout methods, which has been largely overlooked.
Empowering Large Language Model Agents through Action Learning
Large Language Model (LLM) Agents have recently garnered increasing interest yet they are limited in their ability to learn from trial and error, a key element of intelligent behavior.
PRoLoRA: Partial Rotation Empowers More Parameter-Efficient LoRA
Hopefully, the conspicuously higher parameter efficiency can establish PRoLoRA as a resource-friendly alternative to LoRA.
BBA: Bi-Modal Behavioral Alignment for Reasoning with Large Vision-Language Models
Multimodal reasoning stands as a pivotal capability for large vision-language models (LVLMs).
OS-Copilot: Towards Generalist Computer Agents with Self-Improvement
Autonomous interaction with the computer has been a longstanding challenge with great potential, and the recent proliferation of large language models (LLMs) has markedly accelerated progress in building digital agents.
Diffusion of Thoughts: Chain-of-Thought Reasoning in Diffusion Language Models
Recently, diffusion models have garnered significant interest in the field of text processing due to their many potential advantages compared to conventional autoregressive models.
AgentBoard: An Analytical Evaluation Board of Multi-turn LLM Agents
Evaluating Large Language Models (LLMs) as general-purpose agents is essential for understanding their capabilities and facilitating their integration into practical applications.
G-LLaVA: Solving Geometric Problem with Multi-Modal Large Language Model
We first analyze the limitations of current Multimodal Large Language Models (MLLMs) in this area: they struggle to accurately comprehending basic geometric elements and their relationships.
Linear Attention via Orthogonal Memory
Given that orthogonal memory compresses global information, we further dissect the context to amplify fine-grained local information.
Silkie: Preference Distillation for Large Visual Language Models
This paper explores preference distillation for large vision language models (LVLMs), improving their ability to generate helpful and faithful responses anchoring the visual context.
Ranked #66 on
Visual Question Answering
on MM-Vet
Self-Infilling Code Generation
This work introduces self-infilling code generation, a general framework that incorporates infilling operations into auto-regressive decoding.
SEGO: Sequential Subgoal Optimization for Mathematical Problem-Solving
Large Language Models (LLMs) have driven substantial progress in artificial intelligence in recent years, exhibiting impressive capabilities across a wide range of tasks, including mathematical problem-solving.
Attentive Multi-Layer Perceptron for Non-autoregressive Generation
Furthermore, we marry AMLP with popular NAR models, deriving a highly efficient NAR-AMLP architecture with linear time and space complexity.
Lemur: Harmonizing Natural Language and Code for Language Agents
We introduce Lemur and Lemur-Chat, openly accessible language models optimized for both natural language and coding capabilities to serve as the backbone of versatile language agents.
DiffuSeq-v2: Bridging Discrete and Continuous Text Spaces for Accelerated Seq2Seq Diffusion Models
Diffusion models have gained prominence in generating high-quality sequences of text.
Corex: Pushing the Boundaries of Complex Reasoning through Multi-Model Collaboration
Large Language Models (LLMs) are evolving at an unprecedented pace and have exhibited considerable capability in the realm of natural language processing (NLP) with world knowledge.
Extrapolating Large Language Models to Non-English by Aligning Languages
We start from targeting individual languages by performing cross-lingual instruction-tuning (CoIT) on LLaMA, i. e. tuning it with translation task data and cross-lingual general task data to obtain cross-lingual models (x-LLaMAs), and formulate underlying scaling laws to investigate the advantages of using scalable translation data.
L-Eval: Instituting Standardized Evaluation for Long Context Language Models
Recently, there has been growing interest in extending the context length of large language models (LLMs), aiming to effectively process long inputs of one turn or conversations with more extensive histories.
Linearized Relative Positional Encoding
Meanwhile, it emphasizes a general paradigm for designing broadly more relative positional encoding methods that are applicable to linear transformers.
Language Versatilists vs. Specialists: An Empirical Revisiting on Multilingual Transfer Ability
First, does multilingual transfer ability exist in English-centric models and how does it compare with multilingual pretrained models?
INK: Injecting kNN Knowledge in Nearest Neighbor Machine Translation
We propose an effective training framework INK to directly smooth the representation space via adjusting representations of kNN neighbors with a small number of new parameters.
M$^3$IT: A Large-Scale Dataset towards Multi-Modal Multilingual Instruction Tuning
To tackle this challenge and promote research in the vision-language field, we introduce the Multi-Modal, Multilingual Instruction Tuning (M$^3$IT) dataset, designed to optimize VLM alignment with human instructions.
GIMLET: A Unified Graph-Text Model for Instruction-Based Molecule Zero-Shot Learning
We pretrain GIMLET on the molecule tasks along with instructions, enabling the model to transfer effectively to a broad range of tasks.
Decomposing the Enigma: Subgoal-based Demonstration Learning for Formal Theorem Proving
Large language models~(LLMs) present an intriguing avenue of exploration in the domain of formal theorem proving.
Ranked #7 on
Automated Theorem Proving
on miniF2F-test
Optimizing Non-Autoregressive Transformers with Contrastive Learning
In this paper, we propose to ease the difficulty of modality learning via sampling from the model distribution instead of the data distribution.
Can Language Models Understand Physical Concepts?
Language models~(LMs) gradually become general-purpose interfaces in the interactive and embodied world, where the understanding of physical concepts is an essential prerequisite.
Generating Data for Symbolic Language with Large Language Models
However, such an approach has primarily been applied to natural language tasks and has not yet been explored for symbolic language tasks with complex structured outputs (e. g., semantic parsing and code generation).
DetGPT: Detect What You Need via Reasoning
Overall, our proposed paradigm and DetGPT demonstrate the potential for more sophisticated and intuitive interactions between humans and machines.
A Cognitive Stimulation Dialogue System with Multi-source Knowledge Fusion for Elders with Cognitive Impairment
In this paper, we propose a multi-source knowledge fusion method for CS dialogue (CSD), to generate open-ended responses guided by the CS principle and emotional support strategy.
Toeplitz Neural Network for Sequence Modeling
Sequence modeling has important applications in natural language processing and computer vision.
TTIDA: Controllable Generative Data Augmentation via Text-to-Text and Text-to-Image Models
In addition, generative data augmentation (GDA) has been shown to produce more diverse and flexible data.
Multilingual Machine Translation with Large Language Models: Empirical Results and Analysis
Large language models (LLMs) have demonstrated remarkable potential in handling multilingual machine translation (MMT).
Fine-grained Audible Video Description
We explore a new task for audio-visual-language modeling called fine-grained audible video description (FAVD).
A Challenging Benchmark for Low-Resource Learning
Experiments on a wide range of models show that neural networks, even pre-trained language models, have sharp performance drops on our benchmark, demonstrating the effectiveness on evaluating the weaknesses of neural networks.
Retrieved Sequence Augmentation for Protein Representation Learning
RSA links query protein sequences to a set of sequences with similar structures or properties in the database and combines these sequences for downstream prediction.
Compositional Exemplars for In-context Learning
The performance of ICL is highly dominated by the quality of the selected in-context examples.
A Reparameterized Discrete Diffusion Model for Text Generation
This work studies discrete diffusion probabilistic models with applications to natural language generation.
Efficient Attention via Control Variates
Built upon previous progress of RFA, we characterize this gap through the lens of control variates and show that RFA can be decomposed into a sum of multiple control variate estimators for each element in the sequence.
Audio-Visual Segmentation with Semantics
To deal with these problems, we propose a new baseline method that uses a temporal pixel-wise audio-visual interaction module to inject audio semantics as guidance for the visual segmentation process.
Self-Adaptive In-Context Learning: An Information Compression Perspective for In-Context Example Selection and Ordering
Despite the surprising few-shot performance of in-context learning (ICL), it is still a common practice to randomly sample examples to serve as context.
Lego-MT: Learning Detachable Models for Massively Multilingual Machine Translation
To address the needs of learning representations for all languages in a unified space, we propose a novel efficient training recipe, upon which we build an effective detachable model, Lego-MT.
Explanation Regeneration via Information Bottleneck
Explaining the black-box predictions of NLP models naturally and accurately is an important open problem in natural language generation.
Unsupervised Explanation Generation via Correct Instantiations
The major difficulty is finding the conflict point, where the statement contradicts our real world.
An Empirical Revisiting of Linguistic Knowledge Fusion in Language Understanding Tasks
Though linguistic knowledge emerges during large-scale language model pretraining, recent work attempt to explicitly incorporate human-defined linguistic priors into task-specific fine-tuning.
ProGen: Progressive Zero-shot Dataset Generation via In-context Feedback
To improve the quality of dataset synthesis, we propose a progressive zero-shot dataset generation framework, ProGen, which leverages the feedback from the task-specific model to guide the generation of new training data via in-context examples.
Ranked #3 on
Data-free Knowledge Distillation
on QNLI
The Devil in Linear Transformer
In this paper, we examine existing kernel-based linear transformers and identify two key issues that lead to such performance gaps: 1) unbounded gradients in the attention computation adversely impact the convergence of linear transformer models; 2) attention dilution which trivially distributes attention scores over long sequences while neglecting neighbouring structures.
DiffuSeq: Sequence to Sequence Text Generation with Diffusion Models
Bringing together theoretical analysis and empirical evidence, we demonstrate the great potential of diffusion models in complex conditional language generation tasks.
CAB: Comprehensive Attention Benchmarking on Long Sequence Modeling
In this paper, we propose Comprehensive Attention Benchmark (CAB) under a fine-grained attention taxonomy with four distinguishable attention patterns, namely, noncausal self, causal self, noncausal cross, and causal cross attentions.
Audio-Visual Segmentation
To deal with the AVS problem, we propose a novel method that uses a temporal pixel-wise audio-visual interaction module to inject audio semantics as guidance for the visual segmentation process.
Vicinity Vision Transformer
Based on this observation, we present a Vicinity Attention that introduces a locality bias to vision transformers with linear complexity.
Ranked #295 on
Image Classification
on ImageNet
CoNT: Contrastive Neural Text Generation
We validate CoNT on five generation tasks with ten benchmarks, including machine translation, summarization, code comment generation, data-to-text generation and commonsense generation.
Self-Guided Noise-Free Data Generation for Efficient Zero-Shot Learning
In this paradigm, the synthesized data from the PLM acts as the carrier of knowledge, which is used to train a task-specific model with orders of magnitude fewer parameters than the PLM, achieving both higher performance and efficiency than prompt-based zero-shot learning methods on PLMs.
Language Models Can See: Plugging Visual Controls in Text Generation
MAGIC is a flexible framework and is theoretically compatible with any text generation tasks that incorporate image grounding.
Lexical Knowledge Internalization for Neural Dialog Generation
We propose knowledge internalization (KI), which aims to complement the lexical knowledge into neural dialog models.
Event Transition Planning for Open-ended Text Generation
Open-ended text generation tasks, such as dialogue generation and story completion, require models to generate a coherent continuation given limited preceding context.
Linear Complexity Randomized Self-attention Mechanism
By combining the expressiveness in RA and the efficiency in RFA, we develop a novel linear complexity self-attention mechanism called linear randomized attention (LARA).
cosFormer: Rethinking Softmax in Attention
As one of its core components, the softmax attention helps to capture long-range dependencies yet prohibits its scale-up due to the quadratic space and time complexity to the sequence length.
Ranked #6 on
D4RL
on D4RL
Revisiting Over-smoothing in BERT from the Perspective of Graph
Recently over-smoothing phenomenon of Transformer-based models is observed in both vision and language fields.
ZeroGen: Efficient Zero-shot Learning via Dataset Generation
There is a growing interest in dataset generation recently due to the superior generative capacity of large pre-trained language models (PLMs).
Ranked #2 on
Data-free Knowledge Distillation
on QNLI
A Contrastive Framework for Neural Text Generation
Text generation is of great importance to many natural language processing applications.
SNCSE: Contrastive Learning for Unsupervised Sentence Embedding with Soft Negative Samples
To alleviate feature suppression, we propose contrastive learning for unsupervised sentence embedding with soft negative samples (SNCSE).
UnifiedSKG: Unifying and Multi-Tasking Structured Knowledge Grounding with Text-to-Text Language Models
Structured knowledge grounding (SKG) leverages structured knowledge to complete user requests, such as semantic parsing over databases and question answering over knowledge bases.
Ranked #1 on
Task-Oriented Dialogue Systems
on KVRET
Linguistic Frameworks Go Toe-to-Toe at Neuro-Symbolic Language Modeling
We examine the extent to which, in principle, linguistic graph representations can complement and improve neural language modeling.
Ripple Attention for Visual Perception with Sub-quadratic Complexity
Transformer architectures are now central to sequence modeling tasks.
ABC: Attention with Bounded-memory Control
One way to improve the efficiency is to bound the memory size.
Cascaded Head-colliding Attention
Transformers have advanced the field of natural language processing (NLP) on a variety of important tasks.
Good for Misconceived Reasons: An Empirical Revisiting on the Need for Visual Context in Multimodal Machine Translation
A neural multimodal machine translation (MMT) system is one that aims to perform better translation by extending conventional text-only translation models with multimodal information.
Random Feature Attention
RFA can be used as a drop-in replacement for conventional softmax attention and offers a straightforward way of learning with recency bias through an optional gating mechanism.
Ranked #28 on
Machine Translation
on IWSLT2014 German-English
Adaptive Semiparametric Language Models
We present a language model that combines a large parametric neural network (i. e., a transformer) with a non-parametric episodic memory component in an integrated architecture.
Good for Misconceived Reasons: Revisiting Neural Multimodal Machine Translation
A neural multimodal machine translation (MMT) system is one that aims to perform better translation by extending conventional text-only translation models with multimodal information.
Syntactic Structure Distillation Pretraining For Bidirectional Encoders
Textual representation learners trained on large amounts of data have achieved notable success on downstream tasks; intriguingly, they have also performed well on challenging tests of syntactic competence.
A Mutual Information Maximization Perspective of Language Representation Learning
We show state-of-the-art word representation learning methods maximize an objective function that is a lower bound on the mutual information between different parts of a word sequence (i. e., a sentence).
Better Document-Level Machine Translation with Bayes' Rule
We show that Bayes' rule provides an effective mechanism for creating document translation models that can be learned from only parallel sentences and monolingual documents---a compelling benefit as parallel documents are not always available.
Putting Machine Translation in Context with the Noisy Channel Model
We show that Bayes' rule provides a compelling mechanism for controlling unconditional document language models, using the long-standing challenge of effectively leveraging document context in machine translation.
Relative Pixel Prediction For Autoregressive Image Generation
In natural images, transitions between adjacent pixels tend to be smooth and gradual, a fact that has long been exploited in image compression models based on predictive coding.
Episodic Memory in Lifelong Language Learning
We introduce a lifelong language learning setup where a model needs to learn from a stream of text examples without any dataset identifier.
Learning and Evaluating General Linguistic Intelligence
We define general linguistic intelligence as the ability to reuse previously acquired knowledge about a language's lexicon, syntax, semantics, and pragmatic conventions to adapt to new tasks quickly.
Variational Smoothing in Recurrent Neural Network Language Models
We present a new theoretical perspective of data noising in recurrent neural network language models (Xie et al., 2017).
Sentence Encoding with Tree-constrained Relation Networks
The meaning of a sentence is a function of the relations that hold between its words.
Neural Phrase-to-Phrase Machine Translation
We also design an efficient dynamic programming algorithm to decode segments that allows the model to be trained faster than the existing neural phrase-based machine translation method by Huang et al. (2018).
End-to-End Neural Segmental Models for Speech Recognition
Segmental models are an alternative to frame-based models for sequence prediction, where hypothesized path weights are based on entire segment scores rather than a single frame at a time.
SyntaxNet Models for the CoNLL 2017 Shared Task
We describe a baseline dependency parsing system for the CoNLL2017 Shared Task.
DRAGNN: A Transition-based Framework for Dynamically Connected Neural Networks
In this work, we present a compact, modular framework for constructing novel recurrent neural architectures.
Multitask Learning with CTC and Segmental CRF for Speech Recognition
Segmental conditional random fields (SCRFs) and connectionist temporal classification (CTC) are two sequence labeling methods used for end-to-end training of speech recognition models.
DyNet: The Dynamic Neural Network Toolkit
In the static declaration strategy that is used in toolkits like Theano, CNTK, and TensorFlow, the user first defines a computation graph (a symbolic representation of the computation), and then examples are fed into an engine that executes this computation and computes its derivatives.
What Do Recurrent Neural Network Grammars Learn About Syntax?
We investigate what information they learn, from a linguistic perspective, through various ablations to the model and the data, and by augmenting the model with an attention mechanism (GA-RNNG) to enable closer inspection.
Ranked #22 on
Constituency Parsing
on Penn Treebank
Distilling an Ensemble of Greedy Dependency Parsers into One MST Parser
We introduce two first-order graph-based dependency parsers achieving a new state of the art.
Ranked #17 on
Dependency Parsing
on Penn Treebank
Segmental Recurrent Neural Networks for End-to-end Speech Recognition
This model connects the segmental conditional random field (CRF) with a recurrent neural network (RNN) used for feature extraction.
Ranked #16 on
Speech Recognition
on TIMIT
Segmental Recurrent Neural Networks
Representations of the input segments (i. e., contiguous subsequences of the input) are computed by encoding their constituent tokens using bidirectional recurrent neural nets, and these "segment embeddings" are used to define compatibility scores with output labels.
Document Context Language Models
Text documents are structured on multiple levels of detail: individual words are related by syntax, but larger units of text are related by discourse structure.
An Empirical Comparison of Parsing Methods for Stanford Dependencies
Stanford typed dependencies are a widely desired representation of natural language sentences, but parsing is one of the major computational bottlenecks in text analysis systems.
