Search Results for author:
Found 25 papers, 11 papers with code
Teaching LLMs According to Their Aptitude: Adaptive Reasoning for Mathematical Problem Solving
Existing approaches to mathematical reasoning with large language models (LLMs) rely on Chain-of-Thought (CoT) for generalizability or Tool-Integrated Reasoning (TIR) for precise computation.
CoIR: A Comprehensive Benchmark for Code Information Retrieval Models
Despite the substantial success of Information Retrieval (IR) in various NLP tasks, most IR systems predominantly handle queries and corpora in natural language, neglecting the domain of code retrieval.
Ranked #1 on
Code Search
on CoIR
Preparing Lessons for Progressive Training on Language Models
The rapid progress of Transformers in artificial intelligence has come at the cost of increased resource consumption and greenhouse gas emissions due to growing model sizes.
TRIGO: Benchmarking Formal Mathematical Proof Reduction for Generative Language Models
Automated theorem proving (ATP) has become an appealing domain for exploring the reasoning ability of the recent successful generative language models.
DQ-LoRe: Dual Queries with Low Rank Approximation Re-ranking for In-Context Learning
Dual Queries first query LLM to obtain LLM-generated knowledge such as CoT, then query the retriever to obtain the final exemplars via both question and the knowledge.
FIMO: A Challenge Formal Dataset for Automated Theorem Proving
We present FIMO, an innovative dataset comprising formal mathematical problem statements sourced from the International Mathematical Olympiad (IMO) Shortlisted Problems.
AutoConv: Automatically Generating Information-seeking Conversations with Large Language Models
Information-seeking conversation, which aims to help users gather information through conversation, has achieved great progress in recent years.
NewsDialogues: Towards Proactive News Grounded Conversation
In this paper, we propose a novel task, Proactive News Grounded Conversation, in which a dialogue system can proactively lead the conversation based on some key topics of the news.
G-MAP: General Memory-Augmented Pre-trained Language Model for Domain Tasks
Recently, domain-specific PLMs have been proposed to boost the task performance of specific domains (e. g., biomedical and computer science) by continuing to pre-train general PLMs with domain-specific corpora.
FPT: Improving Prompt Tuning Efficiency via Progressive Training
Inspired by these observations, we propose Fast Prompt Tuning (FPT), which starts by conducting PT using a small-scale partial PLM, and then progressively expands its depth and width until the full-model size.
bert2BERT: Towards Reusable Pretrained Language Models
However, large language model pre-training costs intensive computational resources and most of the models are trained from scratch without reusing the existing pre-trained models, which is wasteful.
Integrating Regular Expressions with Neural Networks via DFA
Human-designed rules are widely used to build industry applications.
Generate & Rank: A Multi-task Framework for Math Word Problems
Math word problem (MWP) is a challenging and critical task in natural language processing.
Ranked #3 on
Math Word Problem Solving
on Math23K
AutoTinyBERT: Automatic Hyper-parameter Optimization for Efficient Pre-trained Language Models
Specifically, we carefully design the techniques of one-shot learning and the search space to provide an adaptive and efficient development way of tiny PLMs for various latency constraints.
Extract then Distill: Efficient and Effective Task-Agnostic BERT Distillation
Task-agnostic knowledge distillation, a teacher-student framework, has been proved effective for BERT compression.
LightMBERT: A Simple Yet Effective Method for Multilingual BERT Distillation
The multilingual pre-trained language models (e. g, mBERT, XLM and XLM-R) have shown impressive performance on cross-lingual natural language understanding tasks.
Improving Task-Agnostic BERT Distillation with Layer Mapping Search
Comprehensive experiments on the evaluation benchmarks demonstrate that 1) layer mapping strategy has a significant effect on task-agnostic BERT distillation and different layer mappings can result in quite different performances; 2) the optimal layer mapping strategy from the proposed search process consistently outperforms the other heuristic ones; 3) with the optimal layer mapping, our student model achieves state-of-the-art performance on the GLUE tasks.
TernaryBERT: Distillation-aware Ultra-low Bit BERT
Transformer-based pre-training models like BERT have achieved remarkable performance in many natural language processing tasks. However, these models are both computation and memory expensive, hindering their deployment to resource-constrained devices.
PoD: Positional Dependency-Based Word Embedding for Aspect Term Extraction
Dependency context-based word embedding jointly learns the representations of word and dependency context, and has been proved effective in aspect term extraction.
TinyBERT: Distilling BERT for Natural Language Understanding
To accelerate inference and reduce model size while maintaining accuracy, we first propose a novel Transformer distillation method that is specially designed for knowledge distillation (KD) of the Transformer-based models.
Ranked #1 on
Natural Language Inference
on MultiNLI Dev
Dialog State Tracking with Reinforced Data Augmentation
Neural dialog state trackers are generally limited due to the lack of quantity and diversity of annotated training data.
Document-Level Multi-Aspect Sentiment Classification as Machine Comprehension
Document-level multi-aspect sentiment classification is an important task for customer relation management.
NNEMBs at SemEval-2017 Task 4: Neural Twitter Sentiment Classification: a Simple Ensemble Method with Different Embeddings
In this paper, we propose a simple and effective ensemble method to further boost the performances of neural models.
Unsupervised Word and Dependency Path Embeddings for Aspect Term Extraction
In this paper, we develop a novel approach to aspect term extraction based on unsupervised learning of distributed representations of words and dependency paths.
