Search Results for author:
Found 17 papers, 12 papers with code
A Three-regime Model of Network Pruning
Our approach uses temperature-like and load-like parameters to model the impact of neural network (NN) training hyperparameters on pruning performance.
When are ensembles really effective?
Ensembling has a long history in statistical data analysis, with many impactful applications.
Neurotoxin: Durable Backdoors in Federated Learning
In this type of attack, the goal of the attacker is to use poisoned updates to implant so-called backdoors into the learned model such that, at test time, the model's outputs can be fixed to a given target for certain inputs.
Evaluating natural language processing models with generalization metrics that do not need access to any training or testing data
Our analyses consider (I) hundreds of Transformers trained in different settings, in which we systematically vary the amount of data, the model size and the optimization hyperparameters, (II) a total of 51 pretrained Transformers from eight families of Huggingface NLP models, including GPT2, BERT, etc., and (III) a total of 28 existing and novel generalization metrics.
The Effect of Model Size on Worst-Group Generalization
Overparameterization is shown to result in poor test accuracy on rare subgroups under a variety of settings where subgroup information is known.
A Dataset-Dispersion Perspective on Reconstruction Versus Recognition in Single-View 3D Reconstruction Networks
Our finding shows that a leading factor in determining recognition versus reconstruction is how dispersed the training data is.
Augmentations in Graph Contrastive Learning: Current Methodological Flaws & Towards Better Practices
Unsupervised graph representation learning is critical to a wide range of applications where labels may be scarce or expensive to procure.
Taxonomizing local versus global structure in neural network loss landscapes
Viewing neural network models in terms of their loss landscapes has a long history in the statistical mechanics approach to learning, and in recent years it has received attention within machine learning proper.
Self-supervised Spatial Reasoning on Multi-View Line Drawings
Based on the fact that self-supervised learning is helpful when a large number of data are available, we propose two self-supervised learning approaches to improve the baseline performance for view consistency reasoning and camera pose reasoning tasks on the SPARE3D dataset.
Two Sides of the Same Coin: Heterophily and Oversmoothing in Graph Convolutional Neural Networks
We are the first to take a unified perspective to jointly explain the oversmoothing and heterophily problems at the node level.
Improving Semi-supervised Federated Learning by Reducing the Gradient Diversity of Models
Replacing BN with the recently-proposed Group Normalization (GN) can reduce gradient diversity and improve test accuracy.
Boundary thickness and robustness in learning models
Using these observations, we show that noise-augmentation on mixup training further increases boundary thickness, thereby combating vulnerability to various forms of adversarial attacks and OOD transforms.
Serverless Straggler Mitigation using Local Error-Correcting Codes
Inexpensive cloud services, such as serverless computing, are often vulnerable to straggling nodes that increase end-to-end latency for distributed computation.
Distributed, Parallel, and Cluster Computing Information Theory Information Theory
Deep Unsupervised Learning of 3D Point Clouds via Graph Topology Inference and Filtering
The experimental results show that (1) the proposed networks outperform the state-of-the-art methods in various tasks; (2) a graph topology can be inferred as auxiliary information without specific supervision on graph topology inference; and (3) graph filtering refines the reconstruction, leading to better performances.
FoldingNet: Point Cloud Auto-encoder via Deep Grid Deformation
Recent deep networks that directly handle points in a point set, e. g., PointNet, have been state-of-the-art for supervised learning tasks on point clouds such as classification and segmentation.
Ranked #14 on
3D Point Cloud Linear Classification
on ModelNet40
3D Point Cloud Linear Classification
General Classification
+1
Mining Point Cloud Local Structures by Kernel Correlation and Graph Pooling
Unlike on images, semantic learning on 3D point clouds using a deep network is challenging due to the naturally unordered data structure.
Coded Distributed Computing for Inverse Problems
Our experiments for personalized PageRank performed on real systems and real social networks show that this ratio can be as large as $10^4$.
