Search Results for author:
Found 52 papers, 17 papers with code
Smooth Loss Functions for Deep Top-k Classification
We compare the performance of the cross-entropy loss and our margin-based losses in various regimes of noise and data size, for the predominant use case of k=5.
Trusting SVM for Piecewise Linear CNNs
We present a novel layerwise optimization algorithm for the learning objective of Piecewise-Linear Convolutional Neural Networks (PL-CNNs), a large class of convolutional neural networks.
Make Sure You're Unsure: A Framework for Verifying Probabilistic Specifications
In this direction, we first introduce a general formulation of probabilistic specifications for neural networks, which captures both probabilistic networks (e. g., Bayesian neural networks, MC-Dropout networks) and uncertain inputs (distributions over inputs arising from sensor noise or other perturbations).
Deep Frank-Wolfe For Neural Network Optimization
Furthermore, we compare our algorithm to SGD with a hand-designed learning rate schedule, and show that it provides similar generalization while converging faster.
Adaptive Neural Compilation
We show that it is possible to compile programs written in a low-level language to a differentiable representation.
Hybrid Models for Learning to Branch
First, in a more realistic setting where only a CPU is available, is the GNN model still competitive?
A Unified View of Piecewise Linear Neural Network Verification
The success of Deep Learning and its potential use in many safety-critical applications has motivated research on formal verification of Neural Network (NN) models.
Training Neural Networks for and by Interpolation
In modern supervised learning, many deep neural networks are able to interpolate the data: the empirical loss can be driven to near zero on all samples simultaneously.
In Defense of the Unitary Scalarization for Deep Multi-Task Learning
We show that unitary scalarization, coupled with standard regularization and stabilization techniques from single-task learning, matches or improves upon the performance of complex multi-task optimizers in popular supervised and reinforcement learning settings.
A Statistical Approach to Assessing Neural Network Robustness
Furthermore, it provides an ability to scale to larger networks than formal verification approaches.
Neural Network Branching for Neural Network Verification
Empirically, our framework achieves roughly $50\%$ reduction in both the number of branches and the time required for verification on various convolutional networks when compared to the best available hand-designed branching strategy.
Lagrangian Decomposition for Neural Network Verification
Both the algorithms offer three advantages: (i) they yield bounds that are provably at least as tight as previous dual algorithms relying on Lagrangian relaxations; (ii) they are based on operations analogous to forward/backward pass of neural networks layers and are therefore easily parallelizable, amenable to GPU implementation and able to take advantage of the convolutional structure of problems; and (iii) they allow for anytime stopping while still providing valid bounds.
ANCER: Anisotropic Certification via Sample-wise Volume Maximization
Randomized smoothing has recently emerged as an effective tool that enables certification of deep neural network classifiers at scale.
A Stochastic Bundle Method for Interpolating Networks
We propose a novel method for training deep neural networks that are capable of interpolation, that is, driving the empirical loss to zero.
IBP Regularization for Verified Adversarial Robustness via Branch-and-Bound
Recent works have tried to increase the verifiability of adversarially trained networks by running the attacks over domains larger than the original perturbations and adding various regularization terms to the objective.
Expressive Losses for Verified Robustness via Convex Combinations
In order to train networks for verified adversarial robustness, it is common to over-approximate the worst-case loss over perturbation regions, resulting in networks that attain verifiability at the expense of standard performance.
Efficient Relaxations for Dense CRFs with Sparse Higher Order Potentials
The presented algorithms can be applied to any labelling problem using a dense CRF with sparse higher-order potentials.
Efficient Optimization for Rank-based Loss Functions
We provide a complete characterization of the loss functions that are amenable to our algorithm, and show that it includes both AP and NDCG based loss functions.
Worst-case Optimal Submodular Extensions for Marginal Estimation
Submodular extensions of an energy function can be used to efficiently compute approximate marginals via variational inference.
Coplanar Repeats by Energy Minimization
This paper proposes an automated method to detect, group and rectify arbitrarily-arranged coplanar repeated elements via energy minimization.
Learning to superoptimize programs
This approach involves repeated sampling of modifications to the program from a proposal distribution, which are accepted or rejected based on whether they preserve correctness, and the improvement they achieve.
Efficient Linear Programming for Dense CRFs
To this end, we develop a proximal minimization framework, where the dual of each proximal problem is optimized via block coordinate descent.
Learning to superoptimize programs - Workshop Version
Superoptimization requires the estimation of the best program for a given computational task.
Truncated Max-of-Convex Models
In order to minimize the energy function of a TMCM over all possible labelings, we design an efficient st-MINCUT based range expansion algorithm.
DISCO Nets: DISsimilarity COefficient Networks
We present a new type of probabilistic model which we call DISsimilarity COefficient Networks (DISCO Nets).
Efficient Continuous Relaxations for Dense CRF
In contrast to the continuous relaxation-based energy minimisation algorithms used for sparse CRFs, the mean-field algorithm fails to provide strong theoretical guarantees on the quality of its solutions.
Parsimonious Labeling
Furthermore, we propose an efficient graph-cuts based algorithm for the parsimonious labeling problem that provides strong theoretical guarantees on the quality of the solution.
Discriminative Parameter Estimation for Random Walks Segmentation
However, one of the main drawbacks of using the RW algorithm is that its parameters have to be hand-tuned.
Discriminative Parameter Estimation for Random Walks Segmentation: Technical Report
However, one of the main drawbacks of using the RW algorithm is that its parameters have to be hand-tuned.
Learning Human Poses from Actions
In order to avoid the high cost of full supervision, we propose to use a diverse data set, which consists of two types of annotations: (i) a small number of images are labeled using the expensive ground-truth pose; and (ii) other images are labeled using the inexpensive action label.
Dissimilarity Coefficient based Weakly Supervised Object Detection
This allows us to use a state of the art discrete generative model that can provide annotation consistent samples from the conditional distribution.
Rounding-based Moves for Metric Labeling
Metric labeling is a special case of energy minimization for pairwise Markov random fields.
Efficient Optimization for Average Precision SVM
The accuracy of information retrieval systems is often measured using average precision (AP).
Piecewise Linear Neural Networks verification: A comparative study
Motivated by the need of accelerating progress in this very important area, we investigate the trade-offs of a number of different approaches based on Mixed Integer Programming, Satisfiability Modulo Theory, as well as a novel method based on the Branch-and-Bound framework.
Optimizing Average Precision using Weakly Supervised Data
The performance of binary classification tasks, such as action classification and object detection, is often measured in terms of the average precision (AP).
Entropy-Based Latent Structured Output Prediction
To this end, we propose a novel prediction criterion that includes as special cases all previous prediction criteria that have been used in the literature.
Branch and Bound for Piecewise Linear Neural Network Verification
We use the data sets to conduct a thorough experimental comparison of existing and new algorithms and to provide an inclusive analysis of the factors impacting the hardness of verification problems.
Weakly Supervised Instance Segmentation by Learning Annotation Consistent Instances
Recent approaches for weakly supervised instance segmentations depend on two components: (i) a pseudo label generation model that provides instances which are consistent with a given annotation; and (ii) an instance segmentation model, which is trained in a supervised manner using the pseudo labels as ground-truth.
Ranked #4 on
Image-level Supervised Instance Segmentation
on PASCAL VOC 2012 val
(using extra training data)
Image-level Supervised Instance Segmentation
Pseudo Label
+3
Improving Local Effectiveness for Global Robustness Training
We demonstrate that, by maximizing the use of adversaries, we achieve high robust accuracy with weak adversaries.
Scaling the Convex Barrier with Sparse Dual Algorithms
Tight and efficient neural network bounding is crucial to the scaling of neural network verification systems.
Improved Branch and Bound for Neural Network Verification via Lagrangian Decomposition
Finally, we design a BaB framework, named Branch and Dual Network Bound (BaDNB), based on our novel bounding and branching algorithms.
Comment on Stochastic Polyak Step-Size: Performance of ALI-G
This is a short note on the performance of the ALI-G algorithm (Berrada et al., 2020) as reported in (Loizou et al., 2021).
Generating Adversarial Examples with Graph Neural Networks
Recent years have witnessed the deployment of adversarial attacks to evaluate the robustness of Neural Networks.
Neural Network Branch-and-Bound for Neural Network Verification
Our combined framework achieves a 50\% reduction in both the number of branches and the time required for verification on various convolutional networks when compared to several state-of-the-art verification methods.
Faking Interpolation Until You Make It
We introduce a novel extension of this idea to tasks where the interpolation property does not hold.
Improving Local Effectiveness for Global robust training
However, many of them rely on strong adversaries, which can be prohibitively expensive to generate when the input dimension is high and the model structure is complicated.
Overcoming the Convex Barrier for Simplex Inputs
Recent progress in neural network verification has challenged the notion of a convex barrier, that is, an inherent weakness in the convex relaxation of the output of a neural network.
Learning to be adversarially robust and differentially private
We study the difficulties in learning that arise from robust and differentially private optimization.
Attention for Adversarial Attacks: Learning from your Mistakes
The GNN outputs a smaller subspace for the PGD attack to focus on.
Lookback for Learning to Branch
Given that B&B results in a tree of sub-MILPs, we ask (a) whether there are strong dependencies exhibited by the target heuristic among the neighboring nodes of the B&B tree, and (b) if so, whether we can incorporate them in our training procedure.
Provably Correct Physics-Informed Neural Networks
Recent work provides promising evidence that Physics-informed neural networks (PINN) can efficiently solve partial differential equations (PDE).
Faithful Knowledge Distillation
To address these questions, we introduce a faithful imitation framework to discuss the relative calibration of confidences and provide empirical and certified methods to evaluate the relative calibration of a student w. r. t.
