Search Results for author:
Found 51 papers, 11 papers with code
You reap what you sow: On the Challenges of Bias Evaluation Under Multilingual Settings
Evaluating bias, fairness, and social impact in monolingual language models is a difficult task.
Constructing the CORD-19 Vaccine Dataset
We introduce new dataset 'CORD-19-Vaccination' to cater to scientists specifically looking into COVID-19 vaccine-related research.
The FineWeb Datasets: Decanting the Web for the Finest Text Data at Scale
The performance of a large language model (LLM) depends heavily on the quality and size of its pretraining dataset.
CIVICS: Building a Dataset for Examining Culturally-Informed Values in Large Language Models
This paper introduces the "CIVICS: Culturally-Informed & Values-Inclusive Corpus for Societal impacts" dataset, designed to evaluate the social and cultural variation of Large Language Models (LLMs) across multiple languages and value-sensitive topics.
Evaluating the Social Impact of Generative AI Systems in Systems and Society
Generative AI systems across modalities, ranging from text (including code), image, audio, and video, have broad social impacts, but there is no official standard for means of evaluating those impacts or for which impacts should be evaluated.
The BigScience ROOTS Corpus: A 1.6TB Composite Multilingual Dataset
As language models grow ever larger, the need for large-scale high-quality text datasets has never been more pressing, especially in multilingual settings.
Measuring Data
We identify the task of measuring data to quantitatively characterize the composition of machine learning data and datasets.
The Stack: 3 TB of permissively licensed source code
Large Language Models (LLMs) play an ever-increasing role in the field of Artificial Intelligence (AI)--not only for natural language processing but also for code understanding and generation.
BLOOM: A 176B-Parameter Open-Access Multilingual Language Model
Large language models (LLMs) have been shown to be able to perform new tasks based on a few demonstrations or natural language instructions.
A Human Rights-Based Approach to Responsible AI
Research on fairness, accountability, transparency and ethics of AI-based interventions in society has gained much-needed momentum in recent years.
Evaluate & Evaluation on the Hub: Better Best Practices for Data and Model Measurements
We introduce Evaluate and Evaluation on the Hub --a set of tools to facilitate the evaluation of models and datasets in ML.
Data Governance in the Age of Large-Scale Data-Driven Language Technology
The recent emergence and adoption of Machine Learning technology, and specifically of Large Language Models, has drawn attention to the need for systematic and transparent management of language data.
Documenting Large Webtext Corpora: A Case Study on the Colossal Clean Crawled Corpus
Finally, we conclude with some recommendations for how to created and document web-scale datasets from a scrape of the internet.
Measuring Model Biases in the Absence of Ground Truth
By treating a classification model's predictions for a given image as a set of labels analogous to a bag of words, we rank the biases that a model has learned with respect to different identity labels.
White Paper - Creating a Repository of Objectionable Online Content: Addressing Undesirable Biases and Ethical Considerations
This white paper summarizes the authors' structured brainstorming regarding ethical considerations for creating an extensive repository of online content labeled with tags that describe potentially questionable content for young viewers.
Towards Accountability for Machine Learning Datasets: Practices from Software Engineering and Infrastructure
In this paper, we introduce a rigorous framework for dataset development transparency which supports decision-making and accountability.
Diversity and Inclusion Metrics in Subset Selection
The ethical concept of fairness has recently been applied in machine learning (ML) settings to describe a wide range of constraints and objectives.
Saving Face: Investigating the Ethical Concerns of Facial Recognition Auditing
Although essential to revealing biased performance, well intentioned attempts at algorithmic auditing can have effects that may harm the very populations these measures are meant to protect.
Computers and Society
Closing the AI Accountability Gap: Defining an End-to-End Framework for Internal Algorithmic Auditing
Rising concern for the societal implications of artificial intelligence systems has inspired a wave of academic and journalistic literature in which deployed systems are audited for harm by investigators from outside the organizations deploying the algorithms.
Computers and Society
Perturbation Sensitivity Analysis to Detect Unintended Model Biases
Data-driven statistical Natural Language Processing (NLP) techniques leverage large amounts of language data to build models that can understand language.
Image Counterfactual Sensitivity Analysis for Detecting Unintended Bias
Facial analysis models are increasingly used in applications that have serious impacts on people's lives, ranging from authentication to surveillance tracking.
50 Years of Test (Un)fairness: Lessons for Machine Learning
We trace how the notion of fairness has been defined within the testing communities of education and hiring over the past half century, exploring the cultural and social context in which different fairness definitions have emerged.
Model Cards for Model Reporting
Model cards also disclose the context in which models are intended to be used, details of the performance evaluation procedures, and other relevant information.
Mitigating Unwanted Biases with Adversarial Learning
Machine learning is a tool for building models that accurately represent input training data.
Multi-Task Learning for Mental Health using Social Media Text
We introduce initial groundwork for estimating suicide risk and mental health in a deep learning framework.
InclusiveFaceNet: Improving Face Attribute Detection with Race and Gender Diversity
We demonstrate an approach to face attribute detection that retains or improves attribute detection accuracy across gender and race subgroups by learning demographic information prior to learning the attribute detection task.
Memory-augmented Attention Modelling for Videos
We present a method to improve video description generation by modeling higher-order interactions between video frames and described concepts.
Measuring Machine Intelligence Through Visual Question Answering
As machines have become more intelligent, there has been a renewed interest in methods for measuring their intelligence.
Visual Storytelling
We introduce the first dataset for sequential vision-to-language, and explore how this data may be used for the task of visual storytelling.
Generating Natural Questions About an Image
There has been an explosion of work in the vision & language community during the past few years from image captioning to video transcription, and answering questions about images.
Seeing through the Human Reporting Bias: Visual Classifiers from Noisy Human-Centric Labels
When human annotators are given a choice about what to label in an image, they apply their own subjective judgments on what to ignore and what to mention.
deltaBLEU: A Discriminative Metric for Generation Tasks with Intrinsically Diverse Targets
We introduce Discriminative BLEU (deltaBLEU), a novel metric for intrinsic evaluation of generated text in tasks that admit a diverse range of possible outputs.
A Survey of Current Datasets for Vision and Language Research
Integrating vision and language has long been a dream in work on artificial intelligence (AI).
A Neural Network Approach to Context-Sensitive Generation of Conversational Responses
We present a novel response generation system that can be trained end to end on large quantities of unstructured Twitter conversations.
Exploring Nearest Neighbor Approaches for Image Captioning
We explore a variety of nearest neighbor baseline approaches for image captioning.
Language Models for Image Captioning: The Quirks and What Works
Two recent approaches have achieved state-of-the-art results in image captioning.
VQA: Visual Question Answering
Given an image and a natural language question about the image, the task is to provide an accurate natural language answer.
From Captions to Visual Concepts and Back
The language model learns from a set of over 400, 000 image descriptions to capture the statistics of word usage.
Ranked #1 on
Image Captioning
on COCO Captions test
