Search Results for author:
Found 8 papers, 6 papers with code
Evaluating In-Context Learning of Libraries for Code Generation
Contemporary Large Language Models (LLMs) exhibit a high degree of code generation and comprehension capability.
MAGNIFICo: Evaluating the In-Context Learning Ability of Large Language Models to Generalize to Novel Interpretations
Additionally, our analysis uncovers the semantic predispositions in LLMs and reveals the impact of recency bias for information presented in long contexts.
Understanding In-Context Learning in Transformers and LLMs by Learning to Learn Discrete Functions
In this work, we take a step towards answering these questions by demonstrating the following: (a) On a test-bed with a variety of Boolean function classes, we find that Transformers can nearly match the optimal learning algorithm for 'simpler' tasks, while their performance deteriorates on more 'complex' tasks.
Simplicity Bias in Transformers and their Ability to Learn Sparse Boolean Functions
(ii) When trained on Boolean functions, both Transformers and LSTMs prioritize learning functions of low sensitivity, with Transformers ultimately converging to functions of lower sensitivity.
When Can Transformers Ground and Compose: Insights from Compositional Generalization Benchmarks
On analyzing the task, we find that identifying the target location in the grid world is the main challenge for the models.
Revisiting the Compositional Generalization Abilities of Neural Sequence Models
Compositional generalization is a fundamental trait in humans, allowing us to effortlessly combine known phrases to form novel sentences.
Are NLP Models really able to Solve Simple Math Word Problems?
Since existing solvers achieve high performance on the benchmark datasets for elementary level MWPs containing one-unknown arithmetic word problems, such problems are often considered "solved" with the bulk of research attention moving to more complex MWPs.
Ranked #1 on
Math Word Problem SolvingΩ
on MAWPS
On the Computational Power of Transformers and its Implications in Sequence Modeling
Transformers are being used extensively across several sequence modeling tasks.
