Search Results for author:
Found 26 papers, 12 papers with code
Fast MLE and MAPE-Based Device Activity Detection for Grant-Free Access via PSCA and PSCA-Net
To address these outstanding issues, we propose new maximum likelihood estimation (MLE) and maximum a posterior estimation (MAPE) based device activity detection methods for known and unknown pathloss that achieve superior error rate and computation time tradeoffs using optimization and deep learning techniques.
Synthesizing Privacy-Preserving Text Data via Finetuning without Finetuning Billion-Scale LLMs
To further adapt to the private domain, the generator is DP finetuned on private data for fine-grained textual information, while the topic model extracts a DP histogram representing distributional information.
LLM360 K2: Building a 65B 360-Open-Source Large Language Model from Scratch
We detail the training of the LLM360 K2-65B model, scaling up our 360-degree OPEN SOURCE approach to the largest and most powerful models under project LLM360.
Crystal: Illuminating LLM Abilities on Language and Code
In this work, we propose a pretraining strategy to enhance the integration of natural language and coding capabilities within a single LLM.
ARIC: An Activity Recognition Dataset in Classroom Surveillance Images
The ARIC dataset has advantages of multiple perspectives, 32 activity categories, three modalities, and real-world classroom scenarios.
Model-GLUE: Democratized LLM Scaling for A Large Model Zoo in the Wild
As Large Language Models (LLMs) excel across tasks and specialized domains, scaling LLMs based on existing models has garnered significant attention, which faces the challenge of decreasing performance when combining disparate models.
LLM360: Towards Fully Transparent Open-Source LLMs
The recent surge in open-source Large Language Models (LLMs), such as LLaMA, Falcon, and Mistral, provides diverse options for AI practitioners and researchers.
RedCoast: A Lightweight Tool to Automate Distributed Training of LLMs on Any GPU/TPUs
In this work, we present RedCoast (Redco), a lightweight and user-friendly tool crafted to automate distributed training and inference for LLMs, as well as to simplify ML pipeline development.
SlimPajama-DC: Understanding Data Combinations for LLM Training
This paper aims to understand the impacts of various data combinations (e. g., web text, Wikipedia, GitHub, books) on the pretraining of large language models using SlimPajama.
BertNet: Harvesting Knowledge Graphs with Arbitrary Relations from Pretrained Language Models
The resulting KGs as a symbolic interpretation of the source LMs also reveal new insights into the LMs' knowledge capacities.
Text Generation with Efficient (Soft) $Q$-Learning
We apply the approach to a wide range of text generation tasks, including learning from noisy/negative examples, adversarial attacks, and prompt generation.
Compression, Transduction, and Creation: A Unified Framework for Evaluating Natural Language Generation
Based on the nature of information change from input to output, we classify NLG tasks into compression (e. g., summarization), transduction (e. g., text rewriting), and creation (e. g., dialog).
On the Generation of Medical Dialogs for COVID-19
Training complex dialog generation models on small datasets bears high risk of overfitting.
Efficient (Soft) Q-Learning for Text Generation with Limited Good Data
We apply the approach to a wide range of novel text generation tasks, including learning from noisy/negative examples, adversarial attacks, and prompt generation.
Summarizing Text on Any Aspects: A Knowledge-Informed Weakly-Supervised Approach
Given a document and a target aspect (e. g., a topic of interest), aspect-based abstractive summarization attempts to generate a summary with respect to the aspect.
Progressive Generation of Long Text with Pretrained Language Models
However, as our systematic examination reveals, it is still challenging for such models to generate coherent long passages of text (e. g., 1000 tokens), especially when the models are fine-tuned to the target domain on a small corpus.
Automatic Text Summarization of COVID-19 Medical Research Articles using BERT and GPT-2
With the COVID-19 pandemic, there is a growing urgency for medical community to keep up with the accelerating growth in the new coronavirus-related literature.
On the Generation of Medical Dialogues for COVID-19
On these two datasets, we train several dialogue generation models based on Transformer, GPT, and BERT-GPT.
Schema-Guided Multi-Domain Dialogue State Tracking with Graph Attention Neural Networks
For multi-domain DST, the data sparsity problem is also a major obstacle due to the increased number of state candidates.
Ranked #12 on
Multi-domain Dialogue State Tracking
on MULTIWOZ 2.1
Learning Data Manipulation for Augmentation and Weighting
In this work, we propose a new method that supports learning different manipulation schemes with the same gradient-based algorithm.
AgentGraph: Towards Universal Dialogue Management with Structured Deep Reinforcement Learning
Experiments show that AgentGraph models significantly outperform traditional reinforcement learning approaches on most of the 18 tasks of the PyDial benchmark.
Connecting the Dots Between MLE and RL for Sequence Prediction
Reinforcement learning such as policy gradient addresses the issue but can have prohibitively poor exploration efficiency.
Texar: A Modularized, Versatile, and Extensible Toolkit for Text Generation
The versatile toolkit also fosters technique sharing across different text generation tasks.
Structured Dialogue Policy with Graph Neural Networks
The proposed structured deep reinforcement learning is based on graph neural networks (GNN), which consists of some sub-networks, each one for a node on a directed graph.
Texar: A Modularized, Versatile, and Extensible Toolbox for Text Generation
The features make Texar particularly suitable for technique sharing and generalization across different text generation applications.
Autonomous Driving in Reality with Reinforcement Learning and Image Translation
Supervised learning is widely used in training autonomous driving vehicle.
