Search Results for author: Edward Kim

Found 41 papers, 10 papers with code

Nevermind: Instruction Override and Moderation in Large Language Models

no code implementations5 Feb 2024 Edward Kim

Given the impressive capabilities of recent Large Language Models (LLMs), we investigate and benchmark the most popular proprietary and different sized open source models on the task of explicit instruction following in conflicting situations, e. g. overrides.

Instruction Following Language Modelling

Elo Uncovered: Robustness and Best Practices in Language Model Evaluation

no code implementations29 Nov 2023 Meriem Boubdir, Edward Kim, Beyza Ermis, Sara Hooker, Marzieh Fadaee

In Natural Language Processing (NLP), the Elo rating system, originally designed for ranking players in dynamic games such as chess, is increasingly being used to evaluate Large Language Models (LLMs) through "A vs B" paired comparisons.

Language Modelling

Which Prompts Make The Difference? Data Prioritization For Efficient Human LLM Evaluation

no code implementations22 Oct 2023 Meriem Boubdir, Edward Kim, Beyza Ermis, Marzieh Fadaee, Sara Hooker

Human evaluation is increasingly critical for assessing large language models, capturing linguistic nuances, and reflecting user preferences more accurately than traditional automated metrics.

Language Modelling Large Language Model

Semantic Adversarial Attacks via Diffusion Models

1 code implementation14 Sep 2023 Chenan Wang, Jinhao Duan, Chaowei Xiao, Edward Kim, Matthew Stamm, Kaidi Xu

Then there are two variants of this framework: 1) the Semantic Transformation (ST) approach fine-tunes the latent space of the generated image and/or the diffusion model itself; 2) the Latent Masking (LM) approach masks the latent space with another target image and local backpropagation-based interpretation methods.

Adversarial Attack

DiffuGen: Adaptable Approach for Generating Labeled Image Datasets using Stable Diffusion Models

1 code implementation1 Sep 2023 Michael Shenoda, Edward Kim

Generating high-quality labeled image datasets is crucial for training accurate and robust machine learning models in the field of computer vision.

Image Generation

Lessons in Reproducibility: Insights from NLP Studies in Materials Science

no code implementations28 Jul 2023 Xiangyun Lei, Edward Kim, Viktoriia Baibakova, Shijing Sun

In summary, our study appreciates the benchmark set by these seminal papers while advocating for further enhancements in research reproducibility practices in the field of NLP for materials science.

Word Embeddings

Investigating Sindy As a Tool For Causal Discovery In Time Series Signals

no code implementations29 Dec 2022 Andrew O'Brien, Rosina Weber, Edward Kim

The SINDy algorithm has been successfully used to identify the governing equations of dynamical systems from time series data.

Causal Discovery Time Series +1

Dictionary Learning with Accumulator Neurons

no code implementations30 May 2022 Gavin Parpart, Carlos Gonzalez, Terrence C. Stewart, Edward Kim, Jocelyn Rego, Andrew O'Brien, Steven Nesbit, Garrett T. Kenyon, Yijing Watkins

The Locally Competitive Algorithm (LCA) uses local competition between non-spiking leaky integrator neurons to infer sparse representations, allowing for potentially real-time execution on massively parallel neuromorphic architectures such as Intel's Loihi processor.

Dictionary Learning

Perception Over Time: Temporal Dynamics for Robust Image Understanding

no code implementations11 Mar 2022 Maryam Daniali, Edward Kim

Next, we demonstrate how our novel visual perception framework can utilize this information "over time" using a biologically plausible algorithm with recurrent units, and as a result, significantly improving its accuracy and robustness over standard CNNs.

Adversarial Robustness

A Scenario-Based Platform for Testing Autonomous Vehicle Behavior Prediction Models in Simulation

no code implementations28 Oct 2021 Francis Indaheng, Edward Kim, Kesav Viswanadha, Jay Shenoy, Jinkyu Kim, Daniel J. Fremont, Sanjit A. Seshia

Hence, it is important that these prediction models are extensively tested in various test scenarios involving interactive behaviors prior to deployment.

Probabilistic Programming

Multi-Agent Algorithmic Recourse

no code implementations1 Oct 2021 Andrew O'Brien, Edward Kim

The recent adoption of machine learning as a tool in real world decision making has spurred interest in understanding how these decisions are being made.

BIG-bench Machine Learning counterfactual +2

Parallel and Multi-Objective Falsification with Scenic and VerifAI

1 code implementation9 Jul 2021 Kesav Viswanadha, Edward Kim, Francis Indaheng, Daniel J. Fremont, Sanjit A. Seshia

Falsification has emerged as an important tool for simulation-based verification of autonomous systems.

Scenic4RL: Programmatic Modeling and Generation of Reinforcement Learning Environments

no code implementations18 Jun 2021 Abdus Salam Azad, Edward Kim, Qiancheng Wu, Kimin Lee, Ion Stoica, Pieter Abbeel, Sanjit A. Seshia

To showcase the benefits, we interfaced SCENIC to an existing RTS environment Google Research Football(GRF) simulator and introduced a benchmark consisting of 32 realistic scenarios, encoded in SCENIC, to train RL agents and testing their generalization capabilities.

reinforcement-learning Reinforcement Learning (RL)

ATRAS: Adversarially Trained Robust Architecture Search

no code implementations13 Jun 2021 Yigit Alparslan, Edward Kim

In this paper, we explore the effect of architecture completeness on adversarial robustness.

Adversarial Robustness

Functional Protein Structure Annotation Using a Deep Convolutional Generative Adversarial Network

no code implementations18 Apr 2021 Ethan Moyer, Jeff Winchell, Isamu Isozaki, Yigit Alparslan, Mali Halac, Edward Kim

Identifying novel functional protein structures is at the heart of molecular engineering and molecular biology, requiring an often computationally exhaustive search.

Generative Adversarial Network Protein Structure Prediction

Extreme Volatility Prediction in Stock Market: When GameStop meets Long Short-Term Memory Networks

1 code implementation1 Mar 2021 Yigit Alparslan, Edward Kim

We compare both strategies to buying and holding one single share for the period that we picked as a benchmark.

Robust SleepNets

1 code implementation24 Feb 2021 Yigit Alparslan, Edward Kim

In this study, we investigate eye closedness detection to prevent vehicle accidents related to driver disengagements and driver drowsiness.

BIG-bench Machine Learning Data Augmentation +1

Evaluating Online and Offline Accuracy Traversal Algorithms for k-Complete Neural Network Architectures

1 code implementation16 Jan 2021 Yigit Alparslan, Ethan Jacob Moyer, Edward Kim

In this paper, we study compact neural network architectures for binary classification and investigate improvements in speed and accuracy when favoring overcomplete architecture candidates that have a very high-dimensional representation of the input.

Binary Classification

A Customizable Dynamic Scenario Modeling and Data Generation Platform for Autonomous Driving

no code implementations30 Nov 2020 Jay Shenoy, Edward Kim, Xiangyu Yue, Taesung Park, Daniel Fremont, Alberto Sangiovanni-Vincentelli, Sanjit Seshia

In this paper, we present a platform to model dynamic and interactive scenarios, generate the scenarios in simulation with different modalities of labeled sensor data, and collect this information for data augmentation.

Autonomous Driving Data Augmentation

The Interpretable Dictionary in Sparse Coding

no code implementations24 Nov 2020 Edward Kim, Connor Onweller, Andrew O'Brien, Kathleen Mccoy

Artificial neural networks (ANNs), specifically deep learning networks, have often been labeled as black boxes due to the fact that the internal representation of the data is not easily interpretable.

The Selectivity and Competition of the Mind's Eye in Visual Perception

no code implementations23 Nov 2020 Edward Kim, Maryam Daniali, Jocelyn Rego, Garrett T. Kenyon

Research has shown that neurons within the brain are selective to certain stimuli.

Scenic: A Language for Scenario Specification and Data Generation

2 code implementations13 Oct 2020 Daniel J. Fremont, Edward Kim, Tommaso Dreossi, Shromona Ghosh, Xiangyu Yue, Alberto L. Sangiovanni-Vincentelli, Sanjit A. Seshia

We design a domain-specific language, Scenic, for describing scenarios that are distributions over scenes and the behaviors of their agents over time.

Probabilistic Programming Synthetic Data Generation

A Programmatic and Semantic Approach to Explaining and Debugging Neural Network Based Object Detectors

no code implementations CVPR 2020 Edward Kim, Divya Gopinath, Corina Pasareanu, Sanjit A. Seshia

Our approach is semantic in that it employs a high-level representation of the distribution of environment scenarios that the detector is intended to work on.

Probabilistic Programming

Modeling Biological Immunity to Adversarial Examples

no code implementations CVPR 2020 Edward Kim, Jocelyn Rego, Yijing Watkins, Garrett T. Kenyon

These exploits, or adversarial examples, are a type of signal attack that can change the output class of a classifier by perturbing the stimulus signal by an imperceptible amount.

Adversarial Attack BIG-bench Machine Learning

Formal Scenario-Based Testing of Autonomous Vehicles: From Simulation to the Real World

no code implementations17 Mar 2020 Daniel J. Fremont, Edward Kim, Yash Vardhan Pant, Sanjit A. Seshia, Atul Acharya, Xantha Bruso, Paul Wells, Steve Lemke, Qiang Lu, Shalin Mehta

We present a new approach to automated scenario-based testing of the safety of autonomous vehicles, especially those using advanced artificial intelligence-based components, spanning both simulation-based evaluation as well as testing in the real world.

Autonomous Vehicles

A Programmatic and Semantic Approach to Explaining and DebuggingNeural Network Based Object Detectors

no code implementations1 Dec 2019 Edward Kim, Divya Gopinath, Corina Pasareanu, Sanjit Seshia

It is programmatic in that scenario representation is a program in a domain-specific probabilistic programming language which can be used to generate synthetic data to test a given perception module.

Probabilistic Programming

Machine-learned metrics for predicting the likelihood of success in materials discovery

no code implementations25 Nov 2019 Yoolhee Kim, Edward Kim, Erin Antono, Bryce Meredig, Julia Ling

Materials discovery is often compared to the challenge of finding a needle in a haystack.

VERIFAI: A Toolkit for the Design and Analysis of Artificial Intelligence-Based Systems

1 code implementation12 Feb 2019 Tommaso Dreossi, Daniel J. Fremont, Shromona Ghosh, Edward Kim, Hadi Ravanbakhsh, Marcell Vazquez-Chanlatte, Sanjit A. Seshia

We present VERIFAI, a software toolkit for the formal design and analysis of systems that include artificial intelligence (AI) and machine learning (ML) components.

BIG-bench Machine Learning

Classifiers Based on Deep Sparse Coding Architectures are Robust to Deep Learning Transferable Examples

no code implementations17 Nov 2018 Jacob M. Springer, Charles S. Strauss, Austin M. Thresher, Edward Kim, Garrett T. Kenyon

Although deep learning has shown great success in recent years, researchers have discovered a critical flaw where small, imperceptible changes in the input to the system can drastically change the output classification.

General Classification

Deep Sparse Coding for Invariant Multimodal Halle Berry Neurons

no code implementations CVPR 2018 Edward Kim, Darryl Hannan, Garrett Kenyon

The brain does not work solely in a feed-forward fashion, but rather all of the neurons are in competition with each other; neurons are integrating information in a bottom up and top down fashion and incorporating expectation and feedback in the modeling process.

BIG-bench Machine Learning

Automatically Extracting Action Graphs from Materials Science Synthesis Procedures

no code implementations18 Nov 2017 Sheshera Mysore, Edward Kim, Emma Strubell, Ao Liu, Haw-Shiuan Chang, Srikrishna Kompella, Kevin Huang, Andrew McCallum, Elsa Olivetti

In this work, we present a system for automatically extracting structured representations of synthesis procedures from the texts of materials science journal articles that describe explicit, experimental syntheses of inorganic compounds.

Cannot find the paper you are looking for? You can Submit a new open access paper.