Search Results for author:
Found 24 papers, 12 papers with code
Improving Model Alignment Through Collective Intelligence of Open-Source LLMS
Building helpful and harmless large language models (LLMs) requires effective model alignment approach based on human instructions and feedback, which necessitates high-quality human-labeled data.
How Well Can General Vision-Language Models Learn Medicine By Watching Public Educational Videos?
On these benchmarks, the 2B model achieves gains of 99. 1% and 98. 1%, while the 7B model shows gains of 22. 5% and 52. 1%, respectively, demonstrating the models' ability to generalize and perform biomedical video understanding on cleaner and more standardized datasets than those seen during training.
Think Deep, Think Fast: Investigating Efficiency of Verifier-free Inference-time-scaling Methods
There is intense interest in investigating how inference time compute (ITC) (e. g. repeated sampling, refinements, etc) can improve large language model (LLM) capabilities.
Scaling Instruction-Tuned LLMs to Million-Token Contexts via Hierarchical Synthetic Data Generation
Large Language Models (LLMs) struggle with long-context reasoning, not only due to the quadratic scaling of computational complexity with sequence length but also because of the scarcity and expense of annotating long-context data.
Ladder-residual: parallelism-aware architecture for accelerating large model inference with communication overlapping
Our insight is that in addition to systems optimization, one can also redesign the model architecture to decouple communication from computation.
RedPajama: an Open Dataset for Training Large Language Models
In addition, we release RedPajama-V2, a massive web-only dataset consisting of raw, unfiltered text data together with quality signals and metadata.
Training-Free Activation Sparsity in Large Language Models
Activation sparsity can enable practical inference speedups in large language models (LLMs) by reducing the compute and memory-movement required for matrix multiplications during the forward pass.
Reasoning in Token Economies: Budget-Aware Evaluation of LLM Reasoning Strategies
A diverse array of reasoning strategies has been proposed to elicit the capabilities of large language models.
Mixture-of-Agents Enhances Large Language Model Capabilities
With the growing number of LLMs, how to harness the collective expertise of multiple LLMs is an exciting open direction.
Dragonfly: Multi-Resolution Zoom-In Encoding Enhances Vision-Language Models
Recent advances in vision-language models (VLMs) have demonstrated the advantages of processing images at higher resolutions and utilizing multi-crop features to preserve native resolution details.
Ranked #150 on
Visual Question Answering
on MM-Vet
Bifurcated Attention: Accelerating Massively Parallel Decoding with Shared Prefixes in LLMs
This study introduces bifurcated attention, a method designed to enhance language model inference in shared-context batch decoding scenarios.
Token Alignment via Character Matching for Subword Completion
Generative models, widely utilized in various applications, can often struggle with prompts corresponding to partial tokens.
Greener yet Powerful: Taming Large Code Generation Models with Quantization
Such large models incur significant resource usage (in terms of memory, latency, and dollars) as well as carbon footprint.
Multi-lingual Evaluation of Code Generation Models
Using these benchmarks, we are able to assess the performance of code generation models in a multi-lingual fashion, and discovered generalization ability of language models on out-of-domain languages, advantages of multi-lingual models over mono-lingual, the ability of few-shot prompting to teach the model new languages, and zero-shot translation abilities even on mono-lingual settings.
Joint Text and Label Generation for Spoken Language Understanding
In this work, we propose to leverage the prior information embedded in pretrained language models (LM) to improve generalization for intent classification and slot labeling tasks with limited training data.
Generative Context Pair Selection for Multi-hop Question Answering
Compositional reasoning tasks like multi-hop question answering, require making latent decisions to get the final answer, given a question.
Structured Prediction as Translation between Augmented Natural Languages
We propose a new framework, Translation between Augmented Natural Languages (TANL), to solve many structured prediction language tasks including joint entity and relation extraction, nested named entity recognition, relation classification, semantic role labeling, event extraction, coreference resolution, and dialogue state tracking.
Ranked #3 on
Relation Classification
on TACRED
Augmented Natural Language for Generative Sequence Labeling
We set a new state-of-the-art for few-shot slot labeling, improving substantially upon the previous 5-shot ($75. 0\% \rightarrow 90. 9\%$) and 1-shot ($70. 4\% \rightarrow 81. 0\%$) state-of-the-art results.
There Are Many Consistent Explanations of Unlabeled Data: Why You Should Average
Presently the most successful approaches to semi-supervised learning are based on consistency regularization, whereby a model is trained to be robust to small perturbations of its inputs and parameters.
Probabilistic FastText for Multi-Sense Word Embeddings
We introduce Probabilistic FastText, a new model for word embeddings that can capture multiple word senses, sub-word structure, and uncertainty information.
Hierarchical Density Order Embeddings
By representing words with probability densities rather than point vectors, probabilistic word embeddings can capture rich and interpretable semantic information and uncertainty.
Ranked #2 on
Lexical Entailment
on HyperLex
Multimodal Word Distributions
Word embeddings provide point representations of words containing useful semantic information.
Adversarial Deep Averaging Networks for Cross-Lingual Sentiment Classification
To tackle the sentiment classification problem in low-resource languages without adequate annotated data, we propose an Adversarial Deep Averaging Network (ADAN) to transfer the knowledge learned from labeled data on a resource-rich source language to low-resource languages where only unlabeled data exists.
Feature Representation in Convolutional Neural Networks
Our results show that CNN feature maps can be used with Random Forests and SVM to yield classification results that outperforms the original CNN.
