Search Results for author:
Found 32 papers, 17 papers with code
AnnoLLM: Making Large Language Models to Be Better Crowdsourced Annotators
Many natural language processing (NLP) tasks rely on labeled data to train machine learning models with high performance.
Deep Crossing: Web-Scale Modeling without Manually Crafted Combinatorial Features
Manually crafted combinatorial features have been the “secret sauce” behind many successful models.
AR-Diffusion: Auto-Regressive Diffusion Model for Text Generation
Diffusion models have gained significant attention in the realm of image generation due to their exceptional performance.
PromptBERT: Improving BERT Sentence Embeddings with Prompts
We propose PromptBERT, a novel contrastive learning method for learning better sentence representation.
Rho-1: Not All Tokens Are What You Need
After fine-tuning, Rho-1-1B and 7B achieved state-of-the-art results of 40. 6% and 51. 8% on MATH dataset, respectively - matching DeepSeekMath with only 3% of the pretraining tokens.
HittER: Hierarchical Transformers for Knowledge Graph Embeddings
Our proposed model consists of two different Transformer blocks: the bottom block extracts features of each entity-relation pair in the local neighborhood of the source entity and the top block aggregates the relational information from outputs of the bottom block.
Ranked #1 on
Link Prediction
on FB15k-237
(Hit@10 metric)
PROD: Progressive Distillation for Dense Retrieval
It is common that a better teacher model results in a bad student via distillation due to the nonnegligible gap between teacher and student.
CAPSTONE: Curriculum Sampling for Dense Retrieval with Document Expansion
The dual-encoder has become the de facto architecture for dense retrieval.
GLGE: A New General Language Generation Evaluation Benchmark
Multi-task benchmarks such as GLUE and SuperGLUE have driven great progress of pretraining and transfer learning in Natural Language Processing (NLP).
Taming Sparsely Activated Transformer with Stochastic Experts
While most on-going research focuses on improving SAMs models by exploring methods of routing inputs to experts, our analysis reveals that such research might not lead to the solution we expect, i. e., the commonly-used routing methods based on gating mechanisms do not work better than randomly routing inputs to experts.
Efficient Long Sequence Modeling via State Space Augmented Transformer
Specifically, we augment a SSM into the bottom layer of SPADE, and we employ efficient local attention methods for the other layers.
TwinBERT: Distilling Knowledge to Twin-Structured BERT Models for Efficient Retrieval
Experimental results showed that the inference time was significantly reduced and was firstly controlled around 20ms on CPUs while at the same time the performance gain from fine-tuned BERT-Base model was mostly retained.
GalaXC: Graph Neural Networks with Labelwise Attention for Extreme Classification
An efficient end-to-end implementation of GalaXC is presented that could be trained on a dataset with 50M labels and 97M training documents in less than 100 hours on 4×V100 GPUs.
BANG: Bridging Autoregressive and Non-autoregressive Generation with Large Scale Pretraining
In this paper, we propose BANG, a new pretraining model to Bridge the gap between Autoregressive (AR) and Non-autoregressive (NAR) Generation.
Dual-Alignment Pre-training for Cross-lingual Sentence Embedding
Recent studies have shown that dual encoder models trained with the sentence-level translation ranking task are effective methods for cross-lingual sentence embedding.
Mask Attention Networks: Rethinking and Strengthen Transformer
We therefore introduce a new layer named dynamic mask attention network (DMAN) with a learnable mask matrix which is able to model localness adaptively.
Ranked #11 on
Machine Translation
on WMT2014 English-German
AutoHint: Automatic Prompt Optimization with Hint Generation
This paper presents AutoHint, a novel framework for automatic prompt engineering and optimization for Large Language Models (LLM).
Recurrent Binary Embedding for GPU-Enabled Exhaustive Retrieval from Billion-Scale Semantic Vectors
Building on top of the powerful concept of semantic learning, this paper proposes a Recurrent Binary Embedding (RBE) model that learns compact representations for real-time retrieval.
MagicVO: End-to-End Monocular Visual Odometry through Deep Bi-directional Recurrent Convolutional Neural Network
This paper proposes a new framework to solve the problem of monocular visual odometry, called MagicVO .
ProphetNet-Ads: A Looking Ahead Strategy for Generative Retrieval Models in Sponsored Search Engine
We build a dataset from a real-word sponsored search engine and carry out experiments to analyze different generative retrieval models.
An Enhanced Knowledge Injection Model for Commonsense Generation
Commonsense generation aims at generating plausible everyday scenario description based on a set of provided concepts.
KFCNet: Knowledge Filtering and Contrastive Learning Network for Generative Commonsense Reasoning
Pre-trained language models have led to substantial gains over a broad range of natural language processing (NLP) tasks, but have been shown to have limitations for natural language generation tasks with high-quality requirements on the output, such as commonsense generation and ad keyword generation.
KFCNet: Knowledge Filtering and Contrastive Learning for Generative Commonsense Reasoning
Pre-trained language models have led to substantial gains over a broad range of natural language processing (NLP) tasks, but have been shown to have limitations for natural language generation tasks with high-quality requirements on the output, such as commonsense generation and ad keyword generation.
A Self-Paced Mixed Distillation Method for Non-Autoregressive Generation
To further illustrate the commercial value of our approach, we conduct experiments on three generation tasks in real-world advertisements applications.
CULG: Commercial Universal Language Generation
Pre-trained language models (PLMs) have dramatically improved performance for many natural language processing (NLP) tasks in domains such as finance and healthcare.
NGAME: Negative Mining-aware Mini-batching for Extreme Classification
This paper identifies that memory overheads of popular negative mining techniques often force mini-batch sizes to remain small and slow training down.
The Counterfactual-Shapley Value: Attributing Change in System Metrics
Specifically, we propose a method to estimate counterfactuals using time-series predictive models and construct an attribution score, CF-Shapley, that is consistent with desirable axioms for attributing an observed change in the output metric.
Metric-guided Distillation: Distilling Knowledge from the Metric to Ranker and Retriever for Generative Commonsense Reasoning
Commonsense generation aims to generate a realistic sentence describing a daily scene under the given concepts, which is very challenging, since it requires models to have relational reasoning and compositional generalization capabilities.
Pre-training Transformers for Knowledge Graph Completion
Learning transferable representation of knowledge graphs (KGs) is challenging due to the heterogeneous, multi-relational nature of graph structures.
DeepTagger: Knowledge Enhanced Named Entity Recognition for Web-Based Ads Queries
For model-free enhancement, we collect unlabeled web queries to augment domain knowledge; and we collect web search results to enrich the information of ads queries.
Evoke: Evoking Critical Thinking Abilities in LLMs via Reviewer-Author Prompt Editing
In Evoke, there are two instances of a same LLM: one as a reviewer (LLM-Reviewer), it scores the current prompt; the other as an author (LLM-Author), it edits the prompt by considering the edit history and the reviewer's feedback.
Slightly Shift New Classes to Remember Old Classes for Video Class-Incremental Learning
So we propose SNRO, which slightly shifts the features of new classes to remember old classes.
