Search Results for author:
Found 20 papers, 5 papers with code
Bifurcated Attention for Single-Context Large-Batch Sampling
In our study, we present bifurcated attention, a method developed for language model inference in single-context batch sampling contexts.
Elastic Multi-Gradient Descent for Parallel Continual Learning
The goal of Continual Learning (CL) is to continuously learn from new data streams and accomplish the corresponding tasks.
Enhancing Control Performance through ESN-Based Model Compensation in MPC for Dynamic Systems
Deriving precise system dynamic models through traditional numerical methods is often a challenging endeavor.
Data-Driven Moving Horizon Estimation Using Bayesian Optimization
In this work, an innovative data-driven moving horizon state estimation is proposed for model dynamic-unknown systems based on Bayesian optimization.
Exploring Continual Learning for Code Generation Models
Large-scale code generation models such as Codex and CodeT5 have achieved impressive performance.
Greener yet Powerful: Taming Large Code Generation Models with Quantization
Such large models incur significant resource usage (in terms of memory, latency, and dollars) as well as carbon footprint.
Measuring Asymmetric Gradient Discrepancy in Parallel Continual Learning
In Parallel Continual Learning (PCL), the parallel multiple tasks start and end training unpredictably, thus suffering from training conflict and catastrophic forgetting issues.
Multi-lingual Evaluation of Code Generation Models
Using these benchmarks, we are able to assess the performance of code generation models in a multi-lingual fashion, and discovered generalization ability of language models on out-of-domain languages, advantages of multi-lingual models over mono-lingual, the ability of few-shot prompting to teach the model new languages, and zero-shot translation abilities even on mono-lingual settings.
Exploring Example Influence in Continual Learning
Continual Learning (CL) sequentially learns new tasks like human beings, with the goal to achieve better Stability (S, remembering past tasks) and Plasticity (P, adapting to new tasks).
Learning to Revise References for Faithful Summarization
In real-world scenarios with naturally occurring datasets, reference summaries are noisy and may contain information that cannot be inferred from the source text.
Rethinking Rehearsal in Lifelong Learning: Does An Example Contribute the Plasticity or Stability?
Traditionally, the primary goal of LL is to achieve the trade-off between the Stability (remembering past tasks) and Plasticity (adapting to new tasks).
Amortized Posterior on Latent Variables in Gaussian Process
Deep neural networks have achieved impressive performance on a variety of domains.
Neural Entity Recognition with Gazetteer based Fusion
Our gazetteer based fusion model is data efficient, achieving +1. 7 micro-F1 gains on the i2b2 dataset using 20% training data, and brings + 4. 7 micro-F1 gains on novel entity mentions never presented during training.
An Empirical Investigation Towards Efficient Multi-Domain Language Model Pre-training
However, in order to achieve state-of-the-art performance on out of domain tasks such as clinical named entity recognition and relation extraction, additional in domain pre-training is required.
Learn to Talk via Proactive Knowledge Transfer
In this paper, we provide an in-depth analysis of KL-divergence minimization in Forward and Backward orders, which shows that learners are reinforced via on-policy learning in Backward.
Proactive Sequence Generator via Knowledge Acquisition
Sequence-to-sequence models such as transformers, which are now being used in a wide variety of NLP tasks, typically need to have very high capacity in order to perform well.
Bidirectional Beam Search: Forward-Backward Inference in Neural Sequence Models for Fill-in-the-Blank Image Captioning
We develop the first approximate inference algorithm for 1-Best (and M-Best) decoding in bidirectional neural sequence models by extending Beam Search (BS) to reason about both forward and backward time dependencies.
Diverse Beam Search: Decoding Diverse Solutions from Neural Sequence Models
We observe that our method consistently outperforms BS and previously proposed techniques for diverse decoding from neural sequence models.
SubmodBoxes: Near-Optimal Search for a Set of Diverse Object Proposals
This paper formulates the search for a set of bounding boxes (as needed in object proposal generation) as a monotone submodular maximization problem over the space of all possible bounding boxes in an image.
