Search Results for author:
Found 14 papers, 8 papers with code
Emergent Abilities in Reduced-Scale Generative Language Models
Large language models can solve new tasks without task-specific fine-tuning.
Deconstructing In-Context Learning: Understanding Prompts via Corruption
In contrast, the underlying pre-trained LLMs they use as a backbone are known to be brittle in this respect.
Recent Advances, Applications, and Open Challenges in Machine Learning for Health: Reflections from Research Roundtables at ML4H 2023 Symposium
The organization of the research roundtables at the conference involved 17 Senior Chairs and 19 Junior Chairs across 11 tables.
Let's Reinforce Step by Step
While recent advances have boosted LM proficiency in linguistic benchmarks, LMs consistently struggle to reason correctly on complex tasks like mathematics.
ReLoRA: High-Rank Training Through Low-Rank Updates
Despite the dominance and effectiveness of scaling, resulting in large networks with hundreds of billions of parameters, the necessity to train overparameterized models remains poorly understood, while training costs grow exponentially.
Honey, I Shrunk the Language: Language Model Behavior at Reduced Scale
The majority of recent scaling laws studies focused on high-compute high-parameter count settings, leaving the question of when these abilities begin to emerge largely unanswered.
Scalable and Accurate Self-supervised Multimodal Representation Learning without Aligned Video and Text Data
Currently popular video-text data mining approach via automatic speech recognition (ASR) used in HowTo100M provides low-quality captions that often do not refer to the video content.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+5
Larger Probes Tell a Different Story: Extending Psycholinguistic Datasets Via In-Context Learning
Finally, we observe that while GPT3 has generated all the examples in ROLE-1500 is only able to solve 24. 6% of them during probing.
Scaling Down to Scale Up: A Guide to Parameter-Efficient Fine-Tuning
This paper presents a systematic overview and comparison of parameter-efficient fine-tuning methods covering over 40 papers published between February 2019 and February 2023.
Learning to Ask Like a Physician
The questions are generated by medical experts from 100+ MIMIC-III discharge summaries.
Life after BERT: What do Other Muppets Understand about Language?
Existing pre-trained transformer analysis works usually focus only on one or two model families at a time, overlooking the variability of the architecture and pre-training objectives.
Update Frequently, Update Fast: Retraining Semantic Parsing Systems in a Fraction of Time
To reduce training time, one can fine-tune the previously trained model on each patch, but naive fine-tuning exhibits catastrophic forgetting - degradation of the model performance on the data not represented in the data patch.
Injecting Hierarchy with U-Net Transformers
The Transformer architecture has become increasingly popular over the past two years, owing to its impressive performance on a number of natural language processing (NLP) tasks.
NarrativeTime: Dense Temporal Annotation on a Timeline
For the past decade, temporal annotation has been sparse: only a small portion of event pairs in a text was annotated.
