Search Results for author:
Found 30 papers, 17 papers with code
Vision-Language Pretraining: Current Trends and the Future
In addition to the larger pretraining datasets, the transformer architecture (Vaswani et al., 2017) and in particular self-attention applied to two modalities are responsible for the impressive performance of the recent pretrained models on downstream tasks (Hendricks et al., 2021).
Evaluating Numerical Reasoning in Text-to-Image Models
Text-to-image generative models are capable of producing high-quality images that often faithfully depict concepts described using natural language.
Revisiting Text-to-Image Evaluation with Gecko: On Metrics, Prompts, and Human Ratings
Human-rated prompt sets are generally small and the reliability of the ratings -- and thereby the prompt set used to compare models -- is not evaluated.
How FaR Are Large Language Models From Agents with Theory-of-Mind?
We propose a new evaluation paradigm for large language models (LLMs): Thinking for Doing (T4D), which requires models to connect inferences about others' mental states to actions in social scenarios.
Weakly-Supervised Learning of Visual Relations in Multimodal Pretraining
Recent work in vision-and-language pretraining has investigated supervised signals from object detection data to learn better, fine-grained multimodal representations.
Measuring Progress in Fine-grained Vision-and-Language Understanding
While pretraining on large-scale image-text data from the Web has facilitated rapid progress on many vision-and-language (V&L) tasks, recent work has demonstrated that pretrained models lack "fine-grained" understanding, such as the ability to recognise relationships, verbs, and numbers in images.
Ranked #13 on
Visual Reasoning
on Winoground
Evaluating Visual Number Discrimination in Deep Neural Networks
The ability to discriminate between large and small quantities is a core aspect of basic numerical competence in both humans and animals.
Pragmatics in Language Grounding: Phenomena, Tasks, and Modeling Approaches
People rely heavily on context to enrich meaning beyond what is literally said, enabling concise but effective communication.
MAPL: Parameter-Efficient Adaptation of Unimodal Pre-Trained Models for Vision-Language Few-Shot Prompting
Large pre-trained models have proved to be remarkable zero- and (prompt-based) few-shot learners in unimodal vision and language tasks.
Reassessing Evaluation Practices in Visual Question Answering: A Case Study on Out-of-Distribution Generalization
Vision-and-language (V&L) models pretrained on large-scale multimodal data have demonstrated strong performance on various tasks such as image captioning and visual question answering (VQA).
Flamingo: a Visual Language Model for Few-Shot Learning
Building models that can be rapidly adapted to novel tasks using only a handful of annotated examples is an open challenge for multimodal machine learning research.
Ranked #1 on
Action Recognition
on RareAct
Scaling Language Models: Methods, Analysis & Insights from Training Gopher
Language modelling provides a step towards intelligent communication systems by harnessing large repositories of written human knowledge to better predict and understand the world.
Ranked #1 on
Language Modelling
on StackExchange
A Systematic Investigation of Commonsense Knowledge in Large Language Models
Language models (LMs) trained on large amounts of data have shown impressive performance on many NLP tasks under the zero-shot and few-shot setup.
Probing Image-Language Transformers for Verb Understanding
Multimodal image-language transformers have achieved impressive results on a variety of tasks that rely on fine-tuning (e. g., visual question answering and image retrieval).
Decoupling the Role of Data, Attention, and Losses in Multimodal Transformers
Recently multimodal transformer models have gained popularity because their performance on language and vision tasks suggest they learn rich visual-linguistic representations.
Competition in Cross-situational Word Learning: A Computational Study
Children learn word meanings by tapping into the commonalities across different situations in which words are used and overcome the high level of uncertainty involved in early word learning experiences.
Learning to Segment Actions from Observation and Narration
We apply a generative segmental model of task structure, guided by narration, to action segmentation in video.
Visual Grounding in Video for Unsupervised Word Translation
Given this shared embedding we demonstrate that (i) we can map words between the languages, particularly the 'visual' words; (ii) that the shared embedding provides a good initialization for existing unsupervised text-based word translation techniques, forming the basis for our proposed hybrid visual-text mapping algorithm, MUVE; and (iii) our approach achieves superior performance by addressing the shortcomings of text-based methods -- it is more robust, handles datasets with less commonality, and is applicable to low-resource languages.
Language Learning and Processing in People and Machines
We explain the current computational models of language acquisition, their limitations, and how the insights from these models can be incorporated into NLP applications.
Exploiting Attention to Reveal Shortcomings in Memory Models
The decision making processes of deep networks are difficult to understand and while their accuracy often improves with increased architectural complexity, so too does their opacity.
Evaluating Theory of Mind in Question Answering
We propose a new dataset for evaluating question answering models with respect to their capacity to reason about beliefs.
Learning Hierarchical Visual Representations in Deep Neural Networks Using Hierarchical Linguistic Labels
Modern convolutional neural networks (CNNs) are able to achieve human-level object classification accuracy on specific tasks, and currently outperform competing models in explaining complex human visual representations.
Predicting and Explaining Human Semantic Search in a Cognitive Model
Recent work has attempted to characterize the structure of semantic memory and the search algorithms which, together, best approximate human patterns of search revealed in a semantic fluency task.
Calculating Probabilities Simplifies Word Learning
Children can use the statistical regularities of their environment to learn word meanings, a mechanism known as cross-situational learning.
The Interaction of Memory and Attention in Novel Word Generalization: A Computational Investigation
People exhibit a tendency to generalize a novel noun to the basic-level in a hierarchical taxonomy -- a cognitively salient category such as "dog" -- with the degree of generalization depending on the number and type of exemplars.
Simple Search Algorithms on Semantic Networks Learned from Language Use
Recent empirical and modeling research has focused on the semantic fluency task because it is informative about semantic memory.
