Search Results for author:
Found 44 papers, 30 papers with code
Fine-Grained Controllable Text Generation Using Non-Residual Prompting
We propose a resource-efficient method for converting a pre-trained CLM into this architecture, and demonstrate its potential on various experiments, including the novel task of contextualized word inclusion.
Integrating Transformers and Knowledge Graphs for Twitter Stance Detection
Stance detection (SD) entails classifying the sentiment of a text towards a given target, and is a relevant sub-task for opinion mining and social media analysis.
Compositional Zero-Shot Domain Transfer with Text-to-Text Models
We evaluate DoT5 on the biomedical domain and the resource-lean subdomain of radiology, focusing on NLI, text summarisation and embedding learning.
SAT: Size-Aware Transformer for 3D Point Cloud Semantic Segmentation
In this paper, we propose the Size-Aware Transformer (SAT) that can tailor effective receptive fields for objects of different sizes.
Ranked #2 on
Semantic Segmentation
on S3DIS Area5
DePlot: One-shot visual language reasoning by plot-to-table translation
Compared with a SOTA model finetuned on more than >28k data points, DePlot+LLM with just one-shot prompting achieves a 24. 0% improvement over finetuned SOTA on human-written queries from the task of chart QA.
Ranked #1 on
Chart Question Answering
on ChartQA
On-the-fly Denoising for Data Augmentation in Natural Language Understanding
However, data augmentation may introduce noisy data that impairs training.
MatCha: Enhancing Visual Language Pretraining with Math Reasoning and Chart Derendering
Visual language data such as plots, charts, and infographics are ubiquitous in the human world.
Ranked #1 on
Visual Question Answering (VQA)
on PlotQA-D1
Reranking Overgenerated Responses for End-to-End Task-Oriented Dialogue Systems
At training, the high-scoring partition comprises all generated responses whose similarity to the gold response is higher than the similarity of the greedy response to the gold response.
Improving Bilingual Lexicon Induction with Cross-Encoder Reranking
This crucial step is done via 1) creating a word similarity dataset, comprising positive word pairs (i. e., true translations) and hard negative pairs induced from the original CLWE space, and then 2) fine-tuning an mPLM (e. g., mBERT or XLM-R) in a cross-encoder manner to predict the similarity scores.
Bilingual Lexicon Induction
Cross-Lingual Word Embeddings
+7
Pix2Struct: Screenshot Parsing as Pretraining for Visual Language Understanding
Visually-situated language is ubiquitous -- sources range from textbooks with diagrams to web pages with images and tables, to mobile apps with buttons and forms.
Ranked #3 on
Chart Question Answering
on ChartQA
How to tackle an emerging topic? Combining strong and weak labels for Covid news NER
Being able to train Named Entity Recognition (NER) models for emerging topics is crucial for many real-world applications especially in the medical domain where new topics are continuously evolving out of the scope of existing models and datasets.
Do ever larger octopi still amplify reporting biases? Evidence from judgments of typical colour
Language models (LMs) trained on raw texts have no direct access to the physical world.
WinoDict: Probing language models for in-context word acquisition
We introduce a new in-context learning paradigm to measure Large Language Models' (LLMs) ability to learn novel words during inference.
On Reality and the Limits of Language Data: Aligning LLMs with Human Norms
Recent advancements in Large Language Models (LLMs) harness linguistic associations in vast natural language data for practical applications.
TweetNLP: Cutting-Edge Natural Language Processing for Social Media
In this paper we present TweetNLP, an integrated platform for Natural Language Processing (NLP) in social media.
Language Models Can See: Plugging Visual Controls in Text Generation
MAGIC is a flexible framework and is theoretically compatible with any text generation tasks that incorporate image grounding.
Visual Spatial Reasoning
Spatial relations are a basic part of human cognition.
Ranked #1 on
Visual Reasoning
on VSR
Probing Cross-Lingual Lexical Knowledge from Multilingual Sentence Encoders
In this work, we probe SEs for the amount of cross-lingual lexical knowledge stored in their parameters, and compare them against the original multilingual LMs.
Modality-Balanced Embedding for Video Retrieval
This modality imbalanceresults from a) modality gap: the relevance between a query and a video text is much easier to learn as the query is also a piece of text, with the same modality as the video text; b) data bias: most training samples can be solved solely by text matching.
Improving Word Translation via Two-Stage Contrastive Learning
At Stage C1, we propose to refine standard cross-lingual linear maps between static word embeddings (WEs) via a contrastive learning objective; we also show how to integrate it into the self-learning procedure for even more refined cross-lingual maps.
Revisiting Parameter-Efficient Tuning: Are We Really There Yet?
Parameter-Efficient Tuning (PETuning) methods have been deemed by many as the new paradigm for using pretrained language models (PLMs).
IGLUE: A Benchmark for Transfer Learning across Modalities, Tasks, and Languages
Our benchmark enables the evaluation of multilingual multimodal models for transfer learning, not only in a zero-shot setting, but also in newly defined few-shot learning setups.
Sharpness-Aware Minimization with Dynamic Reweighting
Deep neural networks are often overparameterized and may not easily achieve model generalization.
TaCL: Improving BERT Pre-training with Token-aware Contrastive Learning
Masked language models (MLMs) such as BERT and RoBERTa have revolutionized the field of Natural Language Understanding in the past few years.
Prix-LM: Pretraining for Multilingual Knowledge Base Construction
To achieve this, it is crucial to represent multilingual knowledge in a shared/unified space.
Rewire-then-Probe: A Contrastive Recipe for Probing Biomedical Knowledge of Pre-trained Language Models
To catalyse the research in this direction, we release a well-curated biomedical knowledge probing benchmark, MedLAMA, which is constructed based on the Unified Medical Language System (UMLS) Metathesaurus.
Visually Grounded Reasoning across Languages and Cultures
The design of widespread vision-and-language datasets and pre-trained encoders directly adopts, or draws inspiration from, the concepts and images of ImageNet.
Ranked #1 on
Zero-Shot Cross-Lingual Transfer
on MaRVL
Trans-Encoder: Unsupervised sentence-pair modelling through self- and mutual-distillations
Predominantly, two formulations are used for sentence-pair tasks: bi-encoders and cross-encoders.
Ranked #1 on
Semantic Textual Similarity
on STS16
MirrorWiC: On Eliciting Word-in-Context Representations from Pretrained Language Models
Recent work indicated that pretrained language models (PLMs) such as BERT and RoBERTa can be transformed into effective sentence and word encoders even via simple self-supervised techniques.
Mixture-of-Partitions: Infusing Large Biomedical Knowledge Graphs into BERT
Infusing factual knowledge into pre-trained models is fundamental for many knowledge-intensive tasks.
Learning Domain-Specialised Representations for Cross-Lingual Biomedical Entity Linking
To this end, we propose and evaluate a series of cross-lingual transfer methods for the XL-BEL task, and demonstrate that general-domain bitext helps propagate the available English knowledge to languages with little to no in-domain data.
Contrastive Out-of-Distribution Detection for Pretrained Transformers
Pretrained Transformers achieve remarkable performance when training and test data are from the same distribution.
Fast, Effective, and Self-Supervised: Transforming Masked Language Models into Universal Lexical and Sentence Encoders
In this work, we demonstrate that it is possible to turn MLMs into effective universal lexical and sentence encoders even without any additional data and without any supervision.
Ranked #9 on
Semantic Textual Similarity
on STS16
Contrastive Learning
Cross-Lingual Semantic Textual Similarity
+4
DeepOpht: Medical Report Generation for Retinal Images via Deep Models and Visual Explanation
To train and validate the effectiveness of our DNN-based module, we propose a large-scale retinal disease image dataset.
Self-Alignment Pretraining for Biomedical Entity Representations
Despite the widespread success of self-supervised learning via masked language models (MLM), accurately capturing fine-grained semantic relationships in the biomedical domain remains a challenge.
COMETA: A Corpus for Medical Entity Linking in the Social Media
Whilst there has been growing progress in Entity Linking (EL) for general language, existing datasets fail to address the complex nature of health terminology in layman's language.
Visual Pivoting for (Unsupervised) Entity Alignment
This work studies the use of visual semantic representations to align entities in heterogeneous knowledge graphs (KGs).
Ranked #11 on
Entity Alignment
on dbp15k ja-en
(using extra training data)
Upgrading the Newsroom: An Automated Image Selection System for News Articles
The system fuses multiple textual sources extracted from news articles and accepts multilingual inputs.
HAL: Improved Text-Image Matching by Mitigating Visual Semantic Hubs
The hubness problem widely exists in high-dimensional embedding space and is a fundamental source of error for cross-modal matching tasks.
A Strong and Robust Baseline for Text-Image Matching
We review the current schemes of text-image matching models and propose improvements for both training and inference.
Auto-Classification of Retinal Diseases in the Limit of Sparse Data Using a Two-Streams Machine Learning Model
Automatic clinical diagnosis of retinal diseases has emerged as a promising approach to facilitate discovery in areas with limited access to specialists.
3D Depthwise Convolution: Reducing Model Parameters in 3D Vision Tasks
Standard 3D convolution operations require much larger amounts of memory and computation cost than 2D convolution operations.
A Novel Hybrid Machine Learning Model for Auto-Classification of Retinal Diseases
Automatic clinical diagnosis of retinal diseases has emerged as a promising approach to facilitate discovery in areas with limited access to specialists.
3DCNN-DQN-RNN: A Deep Reinforcement Learning Framework for Semantic Parsing of Large-scale 3D Point Clouds
Our method provides an automatic process that maps the raw data to the classification results.
