Search Results for author:
Found 28 papers, 17 papers with code
Belief in the Machine: Investigating Epistemological Blind Spots of Language Models
These findings highlight significant concerns about current LMs' ability to reason about truth, belief, and knowledge while emphasizing the need for advancements in these areas before broad deployment in critical sectors.
Cybench: A Framework for Evaluating Cybersecurity Capabilities and Risks of Language Models
Without subtask guidance, agents leveraging Claude 3. 5 Sonnet, GPT-4o, OpenAI o1-preview, and Claude 3 Opus successfully solved complete tasks that took human teams up to 11 minutes to solve.
Regulating AI Adaptation: An Analysis of AI Medical Device Updates
In particular, the adaptive nature of AI models presents unique challenges to regulators as updating a model can improve its performance but also introduce safety risks.
Locating and measuring marine aquaculture production from space: a computer vision approach in the French Mediterranean
Overall, our study presents an efficient, scalable and highly adaptable method for monitoring aquaculture production from remote sensing imagery.
Statistical Uncertainty in Word Embeddings: GloVe-V
Static word embeddings are ubiquitous in computational social science applications and contribute to practical decision-making in a variety of fields including law and healthcare.
Hallucination-Free? Assessing the Reliability of Leading AI Legal Research Tools
While hallucinations are reduced relative to general-purpose chatbots (GPT-4), we find that the AI research tools made by LexisNexis (Lexis+ AI) and Thomson Reuters (Westlaw AI-Assisted Research and Ask Practical Law AI) each hallucinate between 17% and 33% of the time.
FLawN-T5: An Empirical Examination of Effective Instruction-Tuning Data Mixtures for Legal Reasoning
In this work, we curate LawInstruct, a large legal instruction dataset, covering 17 jurisdictions, 24 languages and a total of 12M examples.
On the Societal Impact of Open Foundation Models
To understand their risks of misuse, we design a risk assessment framework for analyzing their marginal risk.
How well do LLMs cite relevant medical references? An evaluation framework and analyses
In this paper, we ask: do the sources that LLMs generate actually support the claims that they make?
Large Legal Fictions: Profiling Legal Hallucinations in Large Language Models
Second, we find that legal hallucinations are alarmingly prevalent, occurring between 58% of the time with ChatGPT 4 and 88% with Llama 2, when these models are asked specific, verifiable questions about random federal court cases.
Estimating and Implementing Conventional Fairness Metrics With Probabilistic Protected Features
Unlike similar existing approaches, our methods take advantage of contextual information -- specifically, the relationships between a model's predictions and the probabilistic prediction of protected attributes, given the true protected attribute, and vice versa -- to provide tighter bounds on the true disparity.
Toward Operationalizing Pipeline-aware ML Fairness: A Research Agenda for Developing Practical Guidelines and Tools
While algorithmic fairness is a thriving area of research, in practice, mitigating issues of bias often gets reduced to enforcing an arbitrarily chosen fairness metric, either by enforcing fairness constraints during the optimization step, post-processing model outputs, or by manipulating the training data.
LegalBench: A Collaboratively Built Benchmark for Measuring Legal Reasoning in Large Language Models
The advent of large language models (LLMs) and their adoption by the legal community has given rise to the question: what types of legal reasoning can LLMs perform?
One Law, Many Languages: Benchmarking Multilingual Legal Reasoning for Judicial Support
Our benchmark contains diverse datasets from the Swiss legal system, allowing for a comprehensive study of the underlying non-English, inherently multilingual legal system.
MultiLegalPile: A 689GB Multilingual Legal Corpus
Large, high-quality datasets are crucial for training Large Language Models (LLMs).
LegalBench: Prototyping a Collaborative Benchmark for Legal Reasoning
Finally-inspired by the Open Science movement-we make a call for the legal and computer science communities to join our efforts by contributing new tasks.
Entropy Regularization for Population Estimation
Entropy regularization is known to improve exploration in sequential decision-making problems.
Detecting Environmental Violations with Satellite Imagery in Near Real Time: Land Application under the Clean Water Act
We show that this system effectively appears to detect land application (PR AUC = 0. 93) and we uncover several outlier facilities which appear to apply regularly and excessively.
Pile of Law: Learning Responsible Data Filtering from the Law and a 256GB Open-Source Legal Dataset
One concern with the rise of large language models lies with their potential for significant harm, particularly from pretraining on biased, obscene, copyrighted, and private information.
Algorithmic Fairness and Vertical Equity: Income Fairness with IRS Tax Audit Models
First, we show how the use of more flexible machine learning (classification) methods -- as opposed to simpler models -- shifts audit burdens from high to middle-income taxpayers.
Integrating Reward Maximization and Population Estimation: Sequential Decision-Making for Internal Revenue Service Audit Selection
This approach has the potential to improve audit efficacy, while maintaining policy-relevant estimates of the tax gap.
Reconciling Risk Allocation and Prevalence Estimation in Public Health Using Batched Bandits
In many public health settings, there is a perceived tension between allocating resources to known vulnerable areas and learning about the overall prevalence of the problem.
On the Opportunities and Risks of Foundation Models
AI is undergoing a paradigm shift with the rise of models (e. g., BERT, DALL-E, GPT-3) that are trained on broad data at scale and are adaptable to a wide range of downstream tasks.
Context-Aware Legal Citation Recommendation using Deep Learning
Lawyers and judges spend a large amount of time researching the proper legal authority to cite while drafting decisions.
Enhancing Environmental Enforcement with Near Real-Time Monitoring: Likelihood-Based Detection of Structural Expansion of Intensive Livestock Farms
Much environmental enforcement in the United States has historically relied on either self-reported data or physical, resource-intensive, infrequent inspections.
When Does Pretraining Help? Assessing Self-Supervised Learning for Law and the CaseHOLD Dataset
While a Transformer architecture (BERT) pretrained on a general corpus (Google Books and Wikipedia) improves performance, domain pretraining (using corpus of approximately 3. 5M decisions across all courts in the U. S. that is larger than BERT's) with a custom legal vocabulary exhibits the most substantial performance gains with CaseHOLD (gain of 7. 2% on F1, representing a 12% improvement on BERT) and consistent performance gains across two other legal tasks.
Ranked #1 on
Text Classification
on Overruling
Temporal Cluster Matching for Change Detection of Structures from Satellite Imagery
For instance, in rural settings, the pre-construction area may look similar to the surrounding environment until the building is constructed.
Affirmative Algorithms: The Legal Grounds for Fairness as Awareness
While there has been a flurry of research in algorithmic fairness, what is less recognized is that modern antidiscrimination law may prohibit the adoption of such techniques.
