Search Results for author:
Found 16 papers, 11 papers with code
Pre-training Vision Transformers with Very Limited Synthesized Images
Prior work on FDSL has shown that pre-training vision transformers on such synthetic datasets can yield competitive accuracy on a wide range of downstream tasks.
ASDL: A Unified Interface for Gradient Preconditioning in PyTorch
Gradient preconditioning is a key technique to integrate the second-order information into gradients for improving and extending gradient-based learning algorithms.
Visual Atoms: Pre-training Vision Transformers with Sinusoidal Waves
Unlike JFT-300M which is a static dataset, the quality of synthetic datasets will continue to improve, and the current work is a testament to this possibility.
Informative Sample-Aware Proxy for Deep Metric Learning
Among various supervised deep metric learning methods proxy-based approaches have achieved high retrieval accuracies.
Empirical Study on Optimizer Selection for Out-of-Distribution Generalization
Modern deep learning systems do not generalize well when the test data distribution is slightly different to the training data distribution.
Replacing Labeled Real-image Datasets with Auto-generated Contours
In the present work, we show that the performance of formula-driven supervised learning (FDSL) can match or even exceed that of ImageNet-21k without the use of real images, human-, and self-supervision during the pre-training of Vision Transformers (ViTs).
Takeuchi's Information Criteria as Generalization Measures for DNNs Close to NTK Regime
Generalization measures are intensively studied in the machine learning community for better modeling generalization gaps.
OPIRL: Sample Efficient Off-Policy Inverse Reinforcement Learning via Distribution Matching
Inverse Reinforcement Learning (IRL) is attractive in scenarios where reward engineering can be tedious.
RePOSE: Fast 6D Object Pose Refinement via Deep Texture Rendering
Furthermore, we utilize differentiable Levenberg-Marquardt (LM) optimization to refine a pose fast and accurately by minimizing the feature-metric error between the input and rendered image representations without the need of zooming in.
Ranked #4 on
6D Pose Estimation using RGB
on LineMOD
Epipolar-Guided Deep Object Matching for Scene Change Detection
To cope with the difficulty, we introduce a deep graph matching network that establishes object correspondence between an image pair.
Scalable and Practical Natural Gradient for Large-Scale Deep Learning
Large-scale distributed training of deep neural networks results in models with worse generalization performance as a result of the increase in the effective mini-batch size.
Effect of Mixed Precision Computing on H-Matrix Vector Multiplication in BEM Analysis
Hierarchical Matrix (H-matrix) is an approximation technique which splits a target dense matrix into multiple submatrices, and where a selected portion of submatrices are low-rank approximated.
Mathematical Software Distributed, Parallel, and Cluster Computing
Practical Deep Learning with Bayesian Principles
Importantly, the benefits of Bayesian principles are preserved: predictive probabilities are well-calibrated, uncertainties on out-of-distribution data are improved, and continual-learning performance is boosted.
Large-Scale Distributed Second-Order Optimization Using Kronecker-Factored Approximate Curvature for Deep Convolutional Neural Networks
Large-scale distributed training of deep neural networks suffer from the generalization gap caused by the increase in the effective mini-batch size.
Extreme Scale FMM-Accelerated Boundary Integral Equation Solver for Wave Scattering
With distributed memory optimizations, on the other hand, we report near-optimal efficiency in the weak scalability study with respect to both the logarithmic communication complexity as well as the theoretical scaling complexity of FMM.
Performance Computational Engineering, Finance, and Science Mathematical Software
Asynchronous Execution of the Fast Multipole Method Using Charm++
Fast multipole methods (FMM) on distributed mem- ory have traditionally used a bulk-synchronous model of com- municating the local essential tree (LET) and overlapping it with computation of the local data.
Distributed, Parallel, and Cluster Computing 70F10 D.1.2; D.1.3; G.1.0; G.1.2
