Search Results for author:
Found 30 papers, 18 papers with code
ProxyBNN: Learning Binarized Neural Networks via Proxy Matrices
The matrix composed of basis vectors is referred to as the proxy matrix, and auxiliary variables serve as the coefficients of this linear combination.
FastGL: A GPU-Efficient Framework for Accelerating Sampling-Based GNN Training at Large Scale
However, through an in-depth analysis, we observe that the efficiency of existing sampling-based training frameworks is still limited due to the key bottlenecks lying in all three phases of sampling-based training, i. e., subgraph sample, memory IO, and computation.
LongVILA: Scaling Long-Context Visual Language Models for Long Videos
We introduce the long-context Multi-Modal Sequence Parallelism (MM-SP) system that efficiently parallelizes long video training and inference, enabling 2M context length training on 256 GPUs without any gradient checkpointing.
TorchGT: A Holistic System for Large-scale Graph Transformer Training
While there emerge inspiring algorithm advancements, their practical adoption is still limited, particularly on real-world graphs involving up to millions of nodes.
Characterization of Large Language Model Development in the Datacenter
Large Language Models (LLMs) have presented impressive performance across several transformative tasks.
DeltaZip: Efficient Serving of Multiple Full-Model-Tuned LLMs
Fine-tuning large language models (LLMs) greatly improves model quality for downstream tasks.
SpikingNeRF: Making Bio-inspired Neural Networks See through the Real World
In SpikingNeRF, each sampled point on the ray is matched to a particular time step and represented in a hybrid manner where the voxel grids are maintained as well.
Boosting Distributed Full-graph GNN Training with Asynchronous One-bit Communication
Training Graph Neural Networks (GNNs) on large graphs is challenging due to the conflict between the high memory demand and limited GPU memory.
$\rm A^2Q$: Aggregation-Aware Quantization for Graph Neural Networks
Through an in-depth analysis of the topology of GNNs, we observe that the topology of the graph leads to significant differences between nodes, and most of the nodes in a graph appear to have a small aggregation value.
DATE: Dual Assignment for End-to-End Fully Convolutional Object Detection
In this paper, we revisit these two assignment methods and find that bringing one-to-many assignment back to end-to-end fully convolutional detectors helps with model convergence.
Accumulated Trivial Attention Matters in Vision Transformers on Small Datasets
After carefully examining the self-attention modules, we discover that the number of trivial attention weights is far greater than the important ones and the accumulated trivial weights are dominating the attention in Vision Transformers due to their large quantity, which is not handled by the attention itself.
PalQuant: Accelerating High-precision Networks on Low-precision Accelerators
In this paper, we propose the PArallel Low-precision Quantization (PalQuant) method that approximates high-precision computations via learning parallel low-precision representations from scratch.
Deep Learning Workload Scheduling in GPU Datacenters: Taxonomy, Challenges and Vision
Deep learning (DL) shows its prosperity in a wide variety of fields.
Soft Threshold Ternary Networks
The latest STTN shows that ResNet-18 with ternary weights and ternary activations achieves up to 68. 2% Top-1 accuracy on ImageNet.
Revisiting Quantization Error in Face Alignment
The statistical re- sults show the NME generated by quantization error is even larger than 1/3 of the SOTA item, which is a serious obsta- cle for making a new breakthrough in face alignment.
Ranked #8 on
Face Alignment
on COFW
Characterization and Prediction of Deep Learning Workloads in Large-Scale GPU Datacenters
Modern GPU datacenters are critical for delivering Deep Learning (DL) models and services in both the research community and industry.
Architecture Aware Latency Constrained Sparse Neural Networks
Acceleration of deep neural networks to meet a specific latency constraint is essential for their deployment on mobile devices.
HIH: Towards More Accurate Face Alignment via Heatmap in Heatmap
In particular, our HIH reaches 4. 08 NME (Normalized Mean Error) on WFLW, and 3. 21 on COFW, which exceeds previous methods by a significant margin.
Ranked #4 on
Face Alignment
on WFW (Extra Data)
Generative Zero-shot Network Quantization
Convolutional neural networks are able to learn realistic image priors from numerous training samples in low-level image generation and restoration.
ATF: Towards Robust Face Alignment via Leveraging Similarity and Diversity across Different Datasets
Face alignment is an important task in the field of multi-media.
Ranked #9 on
Face Alignment
on COFW
(using extra training data)
A System-Level Solution for Low-Power Object Detection
As a case study, we evaluate our object detection system on a real-world surveillance video with input size of 512x512, and it turns out that the system can achieve an inference speed of 18 fps at the cost of 6. 9W (with display) with an mAP of 66. 4 verified on the PASCAL VOC 2012 dataset.
AirFace: Lightweight and Efficient Model for Face Recognition
With the development of convolutional neural network, significant progress has been made in computer vision tasks.
Compact Global Descriptor for Neural Networks
Long-range dependencies modeling, widely used in capturing spatiotemporal correlation, has shown to be effective in CNN dominated computer vision tasks.
Ranked #212 on
Object Detection
on COCO test-dev
(using extra training data)
Training Binary Weight Networks via Semi-Binary Decomposition
In this paper, we propose a novel semi-binary decomposition method which decomposes a matrix into two binary matrices and a diagonal matrix.
Semi-Supervised Generative Adversarial Hashing for Image Retrieval
Secondly, we design novel structure of the generative model and the discriminative model to learn the distribution of triplet-wise information in a semi-supervised way.
Two-Step Quantization for Low-Bit Neural Networks
In this paper, we propose a simple yet effective Two-Step Quantization (TSQ) framework, by decomposing the network quantization problem into two steps: code learning and transformation function learning based on the learned codes.
From Hashing to CNNs: Training BinaryWeight Networks via Hashing
To achieve this goal, we propose a novel approach named BWNH to train Binary Weight Networks via Hashing.
Recent Advances in Efficient Computation of Deep Convolutional Neural Networks
As for hardware implementation of deep neural networks, a batch of accelerators based on FPGA/ASIC have been proposed in recent years.
Quantized Convolutional Neural Networks for Mobile Devices
Recently, convolutional neural networks (CNN) have demonstrated impressive performance in various computer vision tasks.
