Search Results for author:
Found 29 papers, 11 papers with code
MimiQ: Low-Bit Data-Free Quantization of Vision Transformers with Encouraging Inter-Head Attention Similarity
Data-free quantization (DFQ) is a technique that creates a lightweight network from its full-precision counterpart without the original training data, often through a synthetic dataset.
DeepClair: Utilizing Market Forecasts for Effective Portfolio Selection
To integrate the forecasting model into a deep reinforcement learning-driven portfolio selection framework, we introduced a two-step strategy: first, pre-training the time-series model on market data, followed by fine-tuning the portfolio selection architecture using this model.
DataFreeShield: Defending Adversarial Attacks without Training Data
Recent advances in adversarial robustness rely on an abundant set of training data, where using external or additional datasets has become a common setting.
Pipette: Automatic Fine-grained Large Language Model Training Configurator for Real-World Clusters
In addition, they often fail to consider the memory requirement per GPU, often recommending solutions that could not be executed.
Smart-Infinity: Fast Large Language Model Training using Near-Storage Processing on a Real System
Our work, Smart-Infinity, addresses the storage bandwidth bottleneck of storage-offloaded LLM training using near-storage processing devices on a real system.
PeerAiD: Improving Adversarial Distillation from a Specialized Peer Tutor
In this situation, adversarial distillation is a promising option which aims to distill the robustness of the teacher network to improve the robustness of a small student network.
GraNNDis: Efficient Unified Distributed Training Framework for Deep GNNs on Large Clusters
Lastly, we introduce One-Hop Graph Masking, a computation and communication structure to realize the above methods in multi-server environments.
Pipe-BD: Pipelined Parallel Blockwise Distillation
However, this results in a high overhead of redundant teacher execution, low GPU utilization, and extra data loading.
SGCN: Exploiting Compressed-Sparse Features in Deep Graph Convolutional Network Accelerators
A GCN takes as input an arbitrarily structured graph and executes a series of layers which exploit the graph's structure to calculate their output features.
Slice-and-Forge: Making Better Use of Caches for Graph Convolutional Network Accelerators
Graph convolutional networks (GCNs) are becoming increasingly popular as they can process a wide variety of data formats that prior deep neural networks cannot easily support.
Optimus-CC: Efficient Large NLP Model Training with 3D Parallelism Aware Communication Compression
Compressing the communication is one way to mitigate the overhead by reducing the inter-node traffic volume; however, the existing compression techniques have critical limitations to be applied for NLP models with 3D parallelism in that 1) only the data parallelism traffic is targeted, and 2) the existing compression schemes already harm the model quality too much.
Enabling Hard Constraints in Differentiable Neural Network and Accelerator Co-Exploration
Co-exploration of an optimal neural architecture and its hardware accelerator is an approach of rising interest which addresses the computational cost problem, especially in low-profile systems.
ETF Portfolio Construction via Neural Network trained on Financial Statement Data
Therefore, we use the data of individual stocks to train our neural networks to predict the future performance of individual stocks and use these predictions and the portfolio deposit file (PDF) to construct a portfolio of ETFs.
Shai-am: A Machine Learning Platform for Investment Strategies
The finance industry has adopted machine learning (ML) as a form of quantitative research to support better investment decisions, yet there are several challenges often overlooked in practice.
It's All In the Teacher: Zero-Shot Quantization Brought Closer to the Teacher
To deal with the performance drop induced by quantization errors, a popular method is to use training data to fine-tune quantized networks.
Qimera: Data-free Quantization with Synthetic Boundary Supporting Samples
We find that this is often insufficient to capture the distribution of the original data, especially around the decision boundaries.
Ranked #1 on
Data Free Quantization
on CIFAR-100
ConCoDE: Hard-constrained Differentiable Co-Exploration Method for Neural Architectures and Hardware Accelerators
To handle the hard constraint problem of differentiable co-exploration, we propose ConCoDE, which searches for hard-constrained solutions without compromising the global design objectives.
An Attention Module for Convolutional Neural Networks
To solve the two problems together, we initially propose an attention module for convolutional neural networks by developing an AW-convolution, where the shape of attention maps matches that of the weights rather than the activations.
AutoReCon: Neural Architecture Search-based Reconstruction for Data-free Compression
Thus, a common approach is to compute a reconstructed training dataset before compression.
GradPIM: A Practical Processing-in-DRAM Architecture for Gradient Descent
In this paper, we present GradPIM, a processing-in-memory architecture which accelerates parameter updates of deep neural networks training.
Homogeneous superconducting gap in DBCO synthesized by oxide molecular beam epitaxy
This indicates that the pseudogap and superconductivity are of different origins.
Superconductivity Materials Science
Deep Composer Classification Using Symbolic Representation
In this study, we train deep neural networks to classify composer on a symbolic domain.
DANCE: Differentiable Accelerator/Network Co-Exploration
In such circumstances, this work presents DANCE, a differentiable approach towards the co-exploration of the hardware accelerator and network architecture design.
SoFAr: Shortcut-based Fractal Architectures for Binary Convolutional Neural Networks
Inspired by the shortcuts and fractal architectures, we propose two Shortcut-based Fractal Architectures (SoFAr) specifically designed for BCNNs: 1. residual connection-based fractal architectures for binary ResNet, and 2. dense connection-based fractal architectures for binary DenseNet.
MAPS: Multi-agent Reinforcement Learning-based Portfolio Management System
Generating an investment strategy using advanced deep learning methods in stock markets has recently been a topic of interest.
SimEx: Express Prediction of Inter-dataset Similarity by a Fleet of Autoencoders
Our intuition is that, the more similarity exists between the unknown data samples and the part of known data that an autoencoder was trained with, the better chances there could be that this autoencoder makes use of its trained knowledge, reconstructing output samples closer to the originals.
MUTE: Data-Similarity Driven Multi-hot Target Encoding for Neural Network Design
Target encoding is an effective technique to deliver better performance for conventional machine learning methods, and recently, for deep neural networks as well.
Global Stock Market Prediction Based on Stock Chart Images Using Deep Q-Network
Moreover, the results show that future stock prices can be predicted even if the training and testing procedures are done in different countries.
Globally Optimal Object Tracking with Fully Convolutional Networks
Thus, we propose a new and robust tracking method using a Fully Convolutional Network (FCN) to obtain an object probability map and Dynamic Programming (DP) to seek the globally optimal path through all frames of video.
