Search Results for author:
Found 22 papers, 13 papers with code
T3-Vis: visual analytic for Training and fine-Tuning Transformers in NLP
Transformers are the dominant architecture in NLP, but their training and fine-tuning is still very challenging.
The Llama 3 Herd of Models
This paper presents a new set of foundation models, called Llama 3.
Ranked #2 on
Multi-task Language Understanding
on MMLU
DeTriever: Decoder-representation-based Retriever for Improving NL2SQL In-Context Learning
While in-context Learning (ICL) has proven to be an effective technique to improve the performance of Large Language Models (LLMs) in a variety of complex tasks, notably in translating natural language questions into Structured Query Language (NL2SQL), the question of how to select the most beneficial demonstration examples remains an open research problem.
SQL-Encoder: Improving NL2SQL In-Context Learning Through a Context-Aware Encoder
Furthermore, compared to these competitive models, our proposed encoder enhances the downstream performance of NL2SQL models in 1-shot in-context learning scenarios by 1-2\% for GPT-3. 5-turbo, 4-8\% for CodeLlama-7B, and 2-3\% for CodeLlama-13B.
StarCoder 2 and The Stack v2: The Next Generation
Our large model, StarCoder2- 15B, significantly outperforms other models of comparable size.
Ranked #32 on
Code Generation
on MBPP
Visual Analytics for Generative Transformer Models
While transformer-based models have achieved state-of-the-art results in a variety of classification and generation tasks, their black-box nature makes them challenging for interpretability.
Mixture-of-Linguistic-Experts Adapters for Improving and Interpreting Pre-trained Language Models
In this work, we propose a method that combines two popular research areas by injecting linguistic structures into pre-trained language models in the parameter-efficient fine-tuning (PEFT) setting.
Diversity-Aware Coherence Loss for Improving Neural Topic Models
The standard approach for neural topic modeling uses a variational autoencoder (VAE) framework that jointly minimizes the KL divergence between the estimated posterior and prior, in addition to the reconstruction loss.
StarCoder: may the source be with you!
The BigCode community, an open-scientific collaboration working on the responsible development of Large Language Models for Code (Code LLMs), introduces StarCoder and StarCoderBase: 15. 5B parameter models with 8K context length, infilling capabilities and fast large-batch inference enabled by multi-query attention.
Ranked #50 on
Code Generation
on MBPP
NL4Opt Competition: Formulating Optimization Problems Based on Their Natural Language Descriptions
The Natural Language for Optimization (NL4Opt) Competition was created to investigate methods of extracting the meaning and formulation of an optimization problem based on its text description.
SantaCoder: don't reach for the stars!
The BigCode project is an open-scientific collaboration working on the responsible development of large language models for code.
The Stack: 3 TB of permissively licensed source code
Large Language Models (LLMs) play an ever-increasing role in the field of Artificial Intelligence (AI)--not only for natural language processing but also for code understanding and generation.
OSLAT: Open Set Label Attention Transformer for Medical Entity Retrieval and Span Extraction
In this paper, we propose a method for linking an open set of entities that does not require any span annotations.
Evaluating the Text-to-SQL Capabilities of Large Language Models
We perform an empirical evaluation of Text-to-SQL capabilities of the Codex language model.
Human Guided Exploitation of Interpretable Attention Patterns in Summarization and Topic Segmentation
The multi-head self-attention mechanism of the transformer model has been thoroughly investigated recently.
T3-Vis: a visual analytic framework for Training and fine-Tuning Transformers in NLP
Transformers are the dominant architecture in NLP, but their training and fine-tuning is still very challenging.
ConVIScope: Visual Analytics for Exploring Patient Conversations
The proliferation of text messaging for mobile health is generating a large amount of patient-doctor conversations that can be extremely valuable to health care professionals.
byteSteady: Fast Classification Using Byte-Level n-Gram Embeddings
This article introduces byteSteady -- a fast model for classification using byte-level n-gram embeddings.
DuoRAT: Towards Simpler Text-to-SQL Models
Recent neural text-to-SQL models can effectively translate natural language questions to corresponding SQL queries on unseen databases.
Video Monitoring Queries
We demonstrate that the application of the techniques proposed in conjunction with declarative queries on video streams can dramatically increase the frame processing rate and speed up query processing by at least two orders of magnitude.
On Extractive and Abstractive Neural Document Summarization with Transformer Language Models
We present a method to produce abstractive summaries of long documents that exceed several thousand words via neural abstractive summarization.
Ranked #19 on
Text Summarization
on Pubmed
Towards Deep Conversational Recommendations
There has been growing interest in using neural networks and deep learning techniques to create dialogue systems.
