Search Results for author: Guy Katz

Found 44 papers, 11 papers with code

NLP Verification: Towards a General Methodology for Certifying Robustness

no code implementations15 Mar 2024 Marco Casadio, Tanvi Dinkar, Ekaterina Komendantskaya, Luca Arnaboldi, Omri Isac, Matthew L. Daggitt, Guy Katz, Verena Rieser, Oliver Lemon

We propose a number of practical NLP methods that can help to identify the effects of the embedding gap; and in particular we propose the metric of falsifiability of semantic subpspaces as another fundamental metric to be reported as part of the NLP verification pipeline.

Analyzing Adversarial Inputs in Deep Reinforcement Learning

no code implementations7 Feb 2024 Davide Corsi, Guy Amir, Guy Katz, Alessandro Farinelli

In recent years, Deep Reinforcement Learning (DRL) has become a popular paradigm in machine learning due to its successful applications to real-world and complex systems.

reinforcement-learning

Robustness Assessment of a Runway Object Classifier for Safe Aircraft Taxiing

no code implementations8 Jan 2024 Yizhak Elboher, Raya Elsaleh, Omri Isac, Mélanie Ducoffe, Audrey Galametz, Guillaume Povéda, Ryma Boumazouza, Noémie Cohen, Guy Katz

As deep neural networks (DNNs) are becoming the prominent solution for many computational problems, the aviation industry seeks to explore their potential in alleviating pilot workload and in improving operational safety.

DEM: A Method for Certifying Deep Neural Network Classifier Outputs in Aerospace

no code implementations4 Jan 2024 Guy Katz, Natan Levy, Idan Refaeli, Raz Yerushalmi

Software development in the aerospace domain requires adhering to strict, high-quality standards.

On Reducing Undesirable Behavior in Deep Reinforcement Learning Models

no code implementations6 Sep 2023 Ophir M. Carmel, Guy Katz

Further, it incurs only a very slight hit to performance, or even in some cases - improves it, while significantly reducing the frequency of undesirable behavior.

reinforcement-learning

Formally Explaining Neural Networks within Reactive Systems

no code implementations31 Jul 2023 Shahaf Bassan, Guy Amir, Davide Corsi, Idan Refaeli, Guy Katz

We evaluate our approach on two popular benchmarks from the domain of automated navigation; and observe that our methods allow the efficient computation of minimal and minimum explanations, significantly outperforming the state of the art.

Explainable Artificial Intelligence (XAI)

DelBugV: Delta-Debugging Neural Network Verifiers

no code implementations29 May 2023 Raya Elsaleh, Guy Katz

We were able to simplify many of the verification queries that trigger these faulty behaviors, by as much as 99%.

Verifying Generalization in Deep Learning

no code implementations11 Feb 2023 Guy Amir, Osher Maayan, Tom Zelazny, Guy Katz, Michael Schapira

Deep neural networks (DNNs) are the workhorses of deep learning, which constitutes the state of the art in numerous application domains.

OccRob: Efficient SMT-Based Occlusion Robustness Verification of Deep Neural Networks

no code implementations27 Jan 2023 Xingwu Guo, Ziwei Zhou, Yueling Zhang, Guy Katz, Min Zhang

The experimental results demonstrate our approach's effectiveness and efficiency in verifying DNNs' robustness against various occlusions, and its ability to generate counterexamples when these DNNs are not robust.

Enhancing Deep Learning with Scenario-Based Override Rules: a Case Study

1 code implementation19 Jan 2023 Adiel Ashrov, Guy Katz

Deep neural networks (DNNs) have become a crucial instrument in the software development toolkit, due to their ability to efficiently solve complex problems.

gRoMA: a Tool for Measuring the Global Robustness of Deep Neural Networks

no code implementations5 Jan 2023 Natan Levy, Raz Yerushalmi, Guy Katz

Multiple studies have demonstrated that even modern DNNs are susceptible to adversarial inputs, and this risk must thus be measured and mitigated to allow the deployment of DNNs in critical settings.

veriFIRE: Verifying an Industrial, Learning-Based Wildfire Detection System

no code implementations6 Dec 2022 Guy Amir, Ziv Freund, Guy Katz, Elad Mandelbaum, Idan Refaeli

In this short paper, we present our ongoing work on the veriFIRE project -- a collaboration between industry and academia, aimed at using verification for increasing the reliability of a real-world, safety-critical system.

Taming Reachability Analysis of DNN-Controlled Systems via Abstraction-Based Training

no code implementations21 Nov 2022 Jiaxu Tian, Dapeng Zhi, Si Liu, Peixin Wang, Guy Katz, Min Zhang

The experimental results on a wide range of benchmarks show that the DNNs trained by using our approach exhibit comparable performance, while the reachability analysis of the corresponding systems becomes more amenable with significant tightness and efficiency improvement over the state-of-the-art white-box approaches.

Decision Making Reinforcement Learning (RL)

Efficiently Finding Adversarial Examples with DNN Preprocessing

no code implementations16 Nov 2022 Avriti Chauhan, Mohammad Afzal, Hrishikesh Karmarkar, Yizhak Elboher, Kumar Madhukar, Guy Katz

Deep Neural Networks (DNNs) are everywhere, frequently performing a fairly complex task that used to be unimaginable for machines to carry out.

Decision Making

Towards Formal XAI: Formally Approximate Minimal Explanations of Neural Networks

no code implementations25 Oct 2022 Shahaf Bassan, Guy Katz

We (1) suggest an efficient, verification-based method for finding minimal explanations, which constitute a provable approximation of the global, minimum explanation; (2) show how DNN verification can assist in calculating lower and upper bounds on the optimal explanation; (3) propose heuristics that significantly improve the scalability of the verification process; and (4) suggest the use of bundles, which allows us to arrive at more succinct and interpretable explanations.

Explainable Artificial Intelligence (XAI)

Tighter Abstract Queries in Neural Network Verification

1 code implementation23 Oct 2022 Elazar Cohen, Yizhak Yisrael Elboher, Clark Barrett, Guy Katz

Recent attempts have demonstrated that abstraction-refinement approaches could play a significant role in mitigating these limitations; but these approaches can often produce networks that are so abstract, that they become unsuitable for verification.

On Optimizing Back-Substitution Methods for Neural Network Verification

no code implementations16 Aug 2022 Tom Zelazny, Haoze Wu, Clark Barrett, Guy Katz

A key component in many state-of-the-art verification schemes is computing lower and upper bounds on the values that neurons in the network can obtain for a specific input domain -- and the tighter these bounds, the more likely the verification is to succeed.

Neural Network Verification using Residual Reasoning

no code implementations5 Aug 2022 Yizhak Yisrael Elboher, Elazar Cohen, Guy Katz

Recent work has proposed enhancing such verification techniques with abstraction-refinement capabilities, which have been shown to boost scalability: instead of verifying a large and complex network, the verifier constructs and then verifies a much smaller network, whose correctness implies the correctness of the original network.

Neural Network Verification with Proof Production

no code implementations1 Jun 2022 Omri Isac, Clark Barrett, Min Zhang, Guy Katz

In this work, we present a novel mechanism for enhancing Simplex-based DNN verifiers with proof production capabilities: the generation of an easy-to-check witness of unsatisfiability, which attests to the absence of errors.

Collision Avoidance LEMMA

Verifying Learning-Based Robotic Navigation Systems

no code implementations26 May 2022 Guy Amir, Davide Corsi, Raz Yerushalmi, Luca Marzari, David Harel, Alessandro Farinelli, Guy Katz

Our work is the first to establish the usefulness of DNN verification in identifying and filtering out suboptimal DRL policies in real-world robots, and we believe that the methods presented here are applicable to a wide range of systems that incorporate deep-learning-based agents.

Model Selection Navigate

Efficient Neural Network Analysis with Sum-of-Infeasibilities

2 code implementations19 Mar 2022 Haoze Wu, Aleksandar Zeljić, Guy Katz, Clark Barrett

Given a convex relaxation which over-approximates the non-convex activation functions, we encode the violations of activation functions as a cost function and optimize it with respect to the convex relaxation.

Adversarial Attack Efficient Neural Network

Scenario-Assisted Deep Reinforcement Learning

no code implementations9 Feb 2022 Raz Yerushalmi, Guy Amir, Achiya Elyasaf, David Harel, Guy Katz, Assaf Marron

In this work-in-progress report, we propose a technique for enhancing the reinforcement learning training process (specifically, its reward calculation), in a way that allows human engineers to directly contribute their expert knowledge, making the agent under training more likely to comply with various relevant constraints.

reinforcement-learning Reinforcement Learning (RL)

Verification-Aided Deep Ensemble Selection

no code implementations8 Feb 2022 Guy Amir, Tom Zelazny, Guy Katz, Michael Schapira

Deep neural networks (DNNs) have become the technology of choice for realizing a variety of complex tasks.

Classification

An Abstraction-Refinement Approach to Verifying Convolutional Neural Networks

no code implementations6 Jan 2022 Matan Ostrovsky, Clark Barrett, Guy Katz

Convolutional neural networks have gained vast popularity due to their excellent performance in the fields of computer vision, image processing, and others.

RoMA: a Method for Neural Network Robustness Measurement and Assessment

no code implementations21 Oct 2021 Natan Levy, Guy Katz

In this paper, we present a new statistical method, called Robustness Measurement and Assessment (RoMA), which can measure the expected robustness of a neural network model.

Protein Folding

Minimal Multi-Layer Modifications of Deep Neural Networks

no code implementations18 Oct 2021 Idan Refaeli, Guy Katz

The novel repair procedure implemented in 3M-DNN computes a modification to the network's weights that corrects its behavior, and attempts to minimize this change via a sequence of calls to a backend, black-box DNN verification engine.

Autonomous Driving Collision Avoidance +1

Pruning and Slicing Neural Networks using Formal Verification

1 code implementation28 May 2021 Ori Lahav, Guy Katz

Our approach can produce DNNs that are significantly smaller than the original, rendering them suitable for deployment on additional kinds of systems, and even more amenable to subsequent formal verification.

Towards Scalable Verification of Deep Reinforcement Learning

1 code implementation25 May 2021 Guy Amir, Michael Schapira, Guy Katz

Deep neural networks (DNNs) have gained significant popularity in recent years, becoming the state of the art in a variety of domains.

reinforcement-learning Reinforcement Learning (RL)

Neural Network Robustness as a Verification Property: A Principled Case Study

1 code implementation3 Apr 2021 Marco Casadio, Ekaterina Komendantskaya, Matthew L. Daggitt, Wen Kokke, Guy Katz, Guy Amir, Idan Refaeli

Neural networks are very successful at detecting patterns in noisy data, and have become the technology of choice in many fields.

Data Augmentation

An SMT-Based Approach for Verifying Binarized Neural Networks

1 code implementation5 Nov 2020 Guy Amir, Haoze Wu, Clark Barrett, Guy Katz

One novelty of our technique is that it allows the verification of neural networks that include both binarized and non-binarized components.

Global Optimization of Objective Functions Represented by ReLU Networks

no code implementations7 Oct 2020 Christopher A. Strong, Haoze Wu, Aleksandar Zeljić, Kyle D. Julian, Guy Katz, Clark Barrett, Mykel J. Kochenderfer

However, individual "yes or no" questions cannot answer qualitative questions such as "what is the largest error within these bounds"; the answers to these lie in the domain of optimization.

Parallelization Techniques for Verifying Neural Networks

no code implementations17 Apr 2020 Haoze Wu, Alex Ozdemir, Aleksandar Zeljić, Ahmed Irfan, Kyle Julian, Divya Gopinath, Sadjad Fouladi, Guy Katz, Corina Pasareanu, Clark Barrett

Inspired by recent successes with parallel optimization techniques for solving Boolean satisfiability, we investigate a set of strategies and heuristics that aim to leverage parallel computing to improve the scalability of neural network verification.

Verifying Recurrent Neural Networks using Invariant Inference

1 code implementation6 Apr 2020 Yuval Jacoby, Clark Barrett, Guy Katz

Deep neural networks are revolutionizing the way complex systems are developed.

An Abstraction-Based Framework for Neural Network Verification

1 code implementation31 Oct 2019 Yizhak Yisrael Elboher, Justin Gottschlich, Guy Katz

In this paper, we propose a framework that can enhance neural network verification techniques by using over-approximation to reduce the size of the network - thus making it more amenable to verification.

Simplifying Neural Networks using Formal Verification

no code implementations25 Oct 2019 Sumathi Gokulanathan, Alexander Feldsher, Adi Malca, Clark Barrett, Guy Katz

Deep neural network (DNN) verification is an emerging field, with diverse verification engines quickly becoming available.

Collision Avoidance

Toward Scalable Verification for Safety-Critical Deep Networks

no code implementations18 Jan 2018 Lindsey Kuper, Guy Katz, Justin Gottschlich, Kyle Julian, Clark Barrett, Mykel Kochenderfer

The increasing use of deep neural networks for safety-critical applications, such as autonomous driving and flight control, raises concerns about their safety and reliability.

Autonomous Driving

Ground-Truth Adversarial Examples

no code implementations ICLR 2018 Nicholas Carlini, Guy Katz, Clark Barrett, David L. Dill

We demonstrate how ground truths can serve to assess the effectiveness of attack techniques, by comparing the adversarial examples produced by those attacks to the ground truths; and also of defense techniques, by computing the distance to the ground truths before and after the defense is applied, and measuring the improvement.

DeepSafe: A Data-driven Approach for Checking Adversarial Robustness in Neural Networks

no code implementations2 Oct 2017 Divya Gopinath, Guy Katz, Corina S. Pasareanu, Clark Barrett

We propose a novel approach for automatically identifying safe regions of the input space, within which the network is robust against adversarial perturbations.

Adversarial Robustness Clustering +4

Provably Minimally-Distorted Adversarial Examples

1 code implementation29 Sep 2017 Nicholas Carlini, Guy Katz, Clark Barrett, David L. Dill

Using this approach, we demonstrate that one of the recent ICLR defense proposals, adversarial retraining, provably succeeds at increasing the distortion required to construct adversarial examples by a factor of 4. 2.

Reluplex: An Efficient SMT Solver for Verifying Deep Neural Networks

8 code implementations3 Feb 2017 Guy Katz, Clark Barrett, David Dill, Kyle Julian, Mykel Kochenderfer

Deep neural networks have emerged as a widely used and effective means for tackling complex, real-world problems.

Collision Avoidance

Cannot find the paper you are looking for? You can Submit a new open access paper.