Search Results for author:
Found 55 papers, 20 papers with code
Self-training with Few-shot Rationalization
While pre-trained language models have obtained state-of-the-art performance for several natural language understanding tasks, they are quite opaque in terms of their decision-making process.
Teaching Language Models to Hallucinate Less with Synthetic Tasks
We also find that optimizing the system message rather than the model weights can be critical; fine-tuning the entire model on the synthetic task can counterintuitively increase hallucination.
Task-Based MoE for Multitask Multilingual Machine Translation
Mixture-of-experts (MoE) architecture has been proven a powerful method for diverse tasks in training deep models in many applications.
SkipDecode: Autoregressive Skip Decoding with Batching and Caching for Efficient LLM Inference
While existing token-level early exit methods show promising results for online inference, they cannot be readily applied for batch inferencing and Key-Value caching.
Adversarial Robustness of Prompt-based Few-Shot Learning for Natural Language Understanding
Our results on six GLUE tasks indicate that compared to fully fine-tuned models, vanilla FSL methods lead to a notable relative drop in task performance (i. e., are less robust) in the face of adversarial perturbations.
Orca: Progressive Learning from Complex Explanation Traces of GPT-4
To address these challenges, we develop Orca (We are working with our legal team to publicly release a diff of the model weights in accordance with LLaMA's release policy to be published at https://aka. ms/orca-lm), a 13-billion parameter model that learns to imitate the reasoning process of LFMs.
GRILL: Grounded Vision-language Pre-training via Aligning Text and Image Regions
Generalization to unseen tasks is an important ability for few-shot learners to achieve better zero-/few-shot performance on diverse tasks.
ReWOO: Decoupling Reasoning from Observations for Efficient Augmented Language Models
Augmented Language Models (ALMs) blend the reasoning capabilities of Large Language Models (LLMs) with tools that allow for knowledge retrieval and action execution.
A Systematic Study of Knowledge Distillation for Natural Language Generation with Pseudo-Target Training
In this work, we study the potential of compressing them, which is crucial for real-world applications serving millions of users.
Accelerating Dataset Distillation via Model Augmentation
Dataset Distillation (DD), a newly emerging field, aims at generating much smaller but efficient synthetic training datasets from large ones.
AdaMix: Mixture-of-Adaptations for Parameter-efficient Model Tuning
Standard fine-tuning of large pre-trained language models (PLMs) for downstream tasks requires updating hundreds of millions to billions of parameters, and storing a large copy of the PLM weights for every task resulting in increased cost for storing, sharing and serving the models.
AutoMoE: Heterogeneous Mixture-of-Experts with Adaptive Computation for Efficient Neural Machine Translation
Furthermore, existing MoE works do not consider computational constraints (e. g., FLOPs, latency) to guide their design.
Small Character Models Match Large Word Models for Autocomplete Under Memory Constraints
In this work, we study the more challenging open-domain setting consisting of low frequency user prompt patterns (or broad prompts, e. g., prompt about 93rd academy awards) and demonstrate the effectiveness of character-based language models.
ADMoE: Anomaly Detection with Mixture-of-Experts from Noisy Labels
In this work, we propose a method to leverage weak/noisy labels (e. g., risk scores generated by machine rules for detecting malware) that are cheaper to obtain for anomaly detection.
AdaMix: Mixture-of-Adaptations for Parameter-efficient Model Tuning
Standard fine-tuning of large pre-trained language models (PLMs) for downstream tasks requires updating hundreds of millions to billions of parameters, and storing a large copy of the PLM weights for every task resulting in increased cost for storing, sharing and serving the models.
Sparsely Activated Mixture-of-Experts are Robust Multi-Task Learners
Traditional multi-task learning (MTL) methods use dense networks that use the same set of shared weights across several different tasks.
LiteTransformerSearch: Training-free Neural Architecture Search for Efficient Language Models
Results show that the perplexity of 16-layer GPT-2 and Transformer-XL can be achieved with up to 1. 5x, 2. 5x faster runtime and 1. 2x, 2. 0x lower peak memory utilization.
AutoDistil: Few-shot Task-agnostic Neural Architecture Search for Distilling Large Language Models
Our framework AutoDistil addresses above challenges with the following steps: (a) Incorporates inductive bias and heuristics to partition Transformer search space into K compact sub-spaces (K=3 for typical student sizes of base, small and tiny); (b) Trains one SuperLM for each sub-space using task-agnostic objective (e. g., self-attention distillation) with weight-sharing of students; (c) Lightweight search for the optimal student without re-training.
CLUES: Few-Shot Learning Evaluation in Natural Language Understanding
We demonstrate that while recent models reach human performance when they have access to large amounts of labeled data, there is a huge gap in performance in the few-shot setting for most tasks.
Robustness Challenges in Model Distillation and Pruning for Natural Language Understanding
Recent work has focused on compressing pre-trained language models (PLMs) like BERT where the major focus has been to improve the in-distribution performance for downstream tasks.
LiST: Lite Prompted Self-training Makes Parameter-Efficient Few-shot Learners
The first is the use of self-training to leverage large amounts of unlabeled data for prompt-based FN in few-shot settings.
Self-training with Few-shot Rationalization: Teacher Explanations Aid Student in Few-shot NLU
While pre-trained language models have obtained state-of-the-art performance for several natural language understanding tasks, they are quite opaque in terms of their decision-making process.
Fairness via Representation Neutralization
This process not only requires a lot of instance-level annotations for sensitive attributes, it also does not guarantee that all fairness sensitive information has been removed from the encoder.
XtremeDistilTransformers: Task Transfer for Task-agnostic Distillation
While deep and large pre-trained models are the state-of-the-art for various natural language processing tasks, their huge size poses significant challenges for practical uses in resource constrained settings.
MetaXL: Meta Representation Transformation for Low-resource Cross-lingual Learning
Extensive experiments on real-world low-resource languages - without access to large-scale monolingual corpora or large amounts of labeled data - for tasks like cross-lingual sentiment analysis and named entity recognition show the effectiveness of our approach.
Self-Training with Weak Supervision
In this work, we develop a weak supervision framework (ASTRA) that leverages all the available data for a given task.
Adaptive Self-training for Neural Sequence Labeling with Few Labels
Neural sequence labeling is an important technique employed for many Natural Language Processing (NLP) tasks, such as Named Entity Recognition (NER), slot tagging for dialog systems and semantic parsing.
Uncertainty-aware Self-training for Few-shot Text Classification
Recent success of pre-trained language models crucially hinges on fine-tuning them on large amounts of labeled data for the downstream task, that are typically expensive to acquire or difficult to access for many applications.
Adaptive Self-training for Few-shot Neural Sequence Labeling
While self-training serves as an effective mechanism to learn from large amounts of unlabeled data -- meta-learning helps in adaptive sample re-weighting to mitigate error propagation from noisy pseudo-labels.
Smart To-Do: Automatic Generation of To-Do Items from Emails
Intelligent features in email service applications aim to increase productivity by helping people organize their folders, compose their emails and respond to pending tasks.
Uncertainty-aware Self-training for Text Classification with Few Labels
Standard self-training mechanism randomly samples instances from the unlabeled pool to pseudo-label and augment labeled data.
Learning with Weak Supervision for Email Intent Detection
In this paper, we propose to leverage user actions as a source of weak supervision, in addition to a limited set of annotated examples, to detect intents in emails.
Product Insights: Analyzing Product Intents in Web Search
Web search engines are frequently used to access information about products.
Smart To-Do : Automatic Generation of To-Do Items from Emails
Intelligent features in email service applications aim to increase productivity by helping people organize their folders, compose their emails and respond to pending tasks.
Gender Bias in Multilingual Embeddings and Cross-Lingual Transfer
In this paper, we study gender bias in multilingual embeddings and how it affects transfer learning for NLP applications.
XtremeDistil: Multi-stage Distillation for Massive Multilingual Models
Deep and large pre-trained language models are the state-of-the-art for various natural language processing tasks.
Leveraging Multi-Source Weak Social Supervision for Early Detection of Fake News
Social media has greatly enabled people to participate in online activities at an unprecedented rate.
STANCY: Stance Classification Based on Consistency Cues
Controversial claims are abundant in online media and discussion forums.
Distilling BERT into Simple Neural Networks with Unlabeled Transfer Data
We show that our student models can compress the huge teacher by up to 26x while still matching or even marginally exceeding the teacher performance in low-resource settings with small amount of labeled data.
GhostLink: Latent Network Inference for Influence-aware Recommendation
As additional use-cases, we show that GhostLink can be used to differentiate between users' latent preferences and influenced ones, as well as to detect influential users based on the learned influence graph.
OpenKI: Integrating Open Information Extraction and Knowledge Bases with Relation Inference
In this paper, we consider advancing web-scale knowledge extraction and alignment by integrating OpenIE extractions in the form of (subject, predicate, object) triples with Knowledge Bases (KB).
DeClarE: Debunking Fake News and False Claims using Evidence-Aware Deep Learning
Misinformation such as fake news is one of the big challenges of our society.
OpenTag: Open Attribute Value Extraction from Product Profiles [Deep Learning, Active Learning, Named Entity Recognition]
We study this problem in the context of product catalogs that often have missing values for many attributes of interest.
Probabilistic Graphical Models for Credibility Analysis in Evolving Online Communities
To address the above limitations, we propose probabilistic graphical models that can leverage the joint interplay between multiple factors in online communities --- like user interactions, community dynamics, and textual content --- to automatically assess the credibility of user-contributed online content, and the expertise of users and their evolution with user-interpretable explanation.
Item Recommendation with Continuous Experience Evolution of Users using Brownian Motion
Online review communities are dynamic as users join and leave, adopt new vocabulary, and adapt to evolving trends.
People on Media: Jointly Identifying Credible News and Trustworthy Citizen Journalists in Online Communities
This paper presents a model to systematically analyze the different interactions in a news community between users, news, and sources.
Credible Review Detection with Limited Information using Consistency Analysis
Online reviews provide viewpoints on the strengths and shortcomings of products/services, influencing potential customers' purchasing decisions.
Exploring Latent Semantic Factors to Find Useful Product Reviews
In this work, we attempt to automatically identify review quality in terms of its helpfulness to the end consumers.
People on Drugs: Credibility of User Statements in Health Communities
Online health communities are a valuable source of information for patients and physicians.
Item Recommendation with Evolving User Preferences and Experience
As only item ratings and review texts are observables, we capture the user's experience and interests in a latent model learned from her reviews, vocabulary and writing style.
Author-Specific Sentiment Aggregation for Polarity Prediction of Reviews
Furthermore, we also show the effectiveness of our approach in capturing thwarting in reviews, achieving an accuracy improvement of 11. 53{\%} over the SVM baseline.
Sentiment Analysis : A Literature Survey
We will discuss in details various approaches to perform a computational treatment of sentiments and opinions.
