no code implementations • 22 Mar 2024 • Erik Miehling, Manish Nagireddy, Prasanna Sattigeri, Elizabeth M. Daly, David Piorkowski, John T. Richards
Modern language models, while sophisticated, exhibit some inherent shortcomings, particularly in conversational settings.
no code implementations • 9 Mar 2024 • Swapnaja Achintalwar, Adriana Alvarado Garcia, Ateret Anaby-Tavor, Ioana Baldini, Sara E. Berger, Bishwaranjan Bhattacharjee, Djallel Bouneffouf, Subhajit Chaudhury, Pin-Yu Chen, Lamogha Chiazor, Elizabeth M. Daly, Kirushikesh DB, Rogério Abreu de Paula, Pierre Dognin, Eitan Farchi, Soumya Ghosh, Michael Hind, Raya Horesh, George Kour, Ja Young Lee, Nishtha Madaan, Sameep Mehta, Erik Miehling, Keerthiram Murugesan, Manish Nagireddy, Inkit Padhi, David Piorkowski, Ambrish Rawat, Orna Raz, Prasanna Sattigeri, Hendrik Strobelt, Sarathkrishna Swaminathan, Christoph Tillmann, Aashka Trivedi, Kush R. Varshney, Dennis Wei, Shalisha Witherspooon, Marcel Zalmanovici
Large language models (LLMs) are susceptible to a variety of risks, from non-faithful output to biased and toxic generations.
no code implementations • 13 Sep 2022 • David Piorkowski, Michael Hind, John Richards
These risks have led to proposed regulations, litigation, and general societal concerns.
no code implementations • 24 Jan 2022 • David Piorkowski, John Richards, Michael Hind
The methodology was found to be usable by developers not trained in user-centered techniques, guiding them to creating a documentation template that addressed the specific needs of their consumers while still being reusable across different models and use cases.
no code implementations • 29 Jan 2021 • Soya Park, April Wang, Ban Kawas, Q. Vera Liao, David Piorkowski, Marina Danilevsky
Data scientists face a steep learning curve in understanding a new domain for which they want to build machine learning (ML) models.
no code implementations • 13 Jan 2021 • David Piorkowski, Soya Park, April Yi Wang, Dakuo Wang, Michael Muller, Felix Portnoy
The development of AI applications is a multidisciplinary effort, involving multiple roles collaborating with the AI developers, an umbrella term we use to include data scientists and other AI-adjacent roles on the same team.
no code implementations • 17 Nov 2020 • David Piorkowski, Daniel González, John Richards, Stephanie Houde
In this paper, we propose and evaluate a set of quality dimensions to identify in what ways this type of documentation falls short.
no code implementations • 24 Jun 2020 • John Richards, David Piorkowski, Michael Hind, Stephanie Houde, Aleksandra Mojsilović
This is the first work to describe a methodology for creating the form of AI documentation we call FactSheets.
no code implementations • 22 Aug 2018 • Matthew Arnold, Rachel K. E. Bellamy, Michael Hind, Stephanie Houde, Sameep Mehta, Aleksandra Mojsilovic, Ravi Nair, Karthikeyan Natesan Ramamurthy, Darrell Reimer, Alexandra Olteanu, David Piorkowski, Jason Tsay, Kush R. Varshney
We envision such documents to contain purpose, performance, safety, security, and provenance information to be completed by AI service providers for examination by consumers.
no code implementations • NAACL 2018 • Tommy Sandbank, Michal Shmueli-Scheuer, Jonathan Herzig, David Konopnicki, John Richards, David Piorkowski
In this paper, we outline an approach to detecting such egregious conversations, using behavioral cues from the user, patterns in agent responses, and user-agent interaction.