Search Results for author:
Found 20 papers, 8 papers with code
Towards Foundation Models for Knowledge Graph Reasoning
The key challenge of designing foundation models on KGs is to learn such transferable representations that enable inference on any graph with arbitrary entity and relation vocabularies.
Exploiting Long-Term Dependencies for Generating Dynamic Scene Graphs
Dynamic scene graph generation from a video is challenging due to the temporal dynamics of the scene and the inherent temporal fluctuations of predictions.
Implicit SVD for Graph Representation Learning
Recent improvements in the performance of state-of-the-art (SOTA) methods for Graph Representational Learning (GRL) have come at the cost of significant computational resource requirements for training, e. g., for calculating gradients via backprop over many data epochs.
Sequential Aggregation and Rematerialization: Distributed Full-batch Training of Graph Neural Networks on Large Graphs
We present the Sequential Aggregation and Rematerialization (SAR) scheme for distributed full-batch training of Graph Neural Networks (GNNs) on large graphs.
On Local Aggregation in Heterophilic Graphs
We also show that homophily is a poor measure of the information in a node's local neighborhood and propose the Neighborhood Information Content(NIC) metric, which is a novel information-theoretic graph metric.
Attention-based Image Upsampling
Convolutional layers are an integral part of many deep neural network solutions in computer vision.
Permutohedral-GCN: Graph Convolutional Networks with Global Attention
The attention coefficients depend on the Euclidean distance between learnable node embeddings, and we show that the resulting attention-based global aggregation scheme is analogous to high-dimensional Gaussian filtering.
Robust Federated Learning Through Representation Matching and Adaptive Hyper-parameters
We propose a novel representation matching scheme that reduces the divergence of local models by ensuring the feature representations in the global (aggregate) model can be derived from the locally learned representations.
Single-bit-per-weight deep convolutional neural networks without batch-normalization layers for embedded systems
We found, following experiments with wide residual networks applied to the ImageNet, CIFAR 10 and CIFAR 100 image classification datasets, that BN layers do not consistently offer a significant advantage.
Ranked #94 on
Image Classification
on CIFAR-100
(using extra training data)
Parameter Efficient Training of Deep Convolutional Neural Networks by Dynamic Sparse Reparameterization
We evaluate the performance of dynamic reallocation methods in training deep convolutional networks and show that our method outperforms previous static and dynamic reparameterization methods, yielding the best accuracy for a fixed parameter budget, on par with accuracies obtained by iteratively pruning a pre-trained dense model.
Surrogate Gradient Learning in Spiking Neural Networks
Spiking neural networks are nature's versatile solution to fault-tolerant and energy efficient signal processing.
Synaptic Plasticity Dynamics for Deep Continuous Local Learning (DECOLLE)
A relatively smaller body of work, however, discusses similarities between learning dynamics employed in deep artificial neural networks and synaptic plasticity in spiking neural networks.
Dynamic parameter reallocation improves trainability of deep convolutional networks
Network pruning has emerged as a powerful technique for reducing the size of deep neural networks.
Deep supervised learning using local errors
Updating the features or weights in one layer, however, requires waiting for the propagation of error signals from higher layers.
A learning framework for winner-take-all networks with stochastic synapses
This allows us to use the proposed networks in a variational learning setting where stochastic backpropagation is used to optimize a lower bound on the data log likelihood, thereby learning a generative model of the data.
Hardware-efficient on-line learning through pipelined truncated-error backpropagation in binary-state networks
In this paper, we describe a hardware-efficient on-line learning technique for feedforward multi-layer ANNs that is based on pipelined backpropagation.
NullHop: A Flexible Convolutional Neural Network Accelerator Based on Sparse Representations of Feature Maps
By exploiting sparsity, NullHop achieves an efficiency of 368%, maintains over 98% utilization of the MAC units, and achieves a power efficiency of over 3TOp/s/W in a core area of 6. 3mm$^2$.
Supervised learning based on temporal coding in spiking neural networks
Gradient descent training techniques are remarkably successful in training analog-valued artificial neural networks (ANNs).
Stochastic Interpretation of Quasi-periodic Event-based Systems
We show that stochastic artificial neurons can be realized on silicon chips by exploiting the quasi-periodic behavior of mismatched analog oscillators to approximate the neuron's stochastic activation function.
Recurrent networks of coupled Winner-Take-All oscillators for solving constraint satisfaction problems
If there is no solution that satisfies all constraints, the network state changes in a pseudo-random manner and its trajectory approximates a sampling procedure that selects a variable assignment with a probability that increases with the fraction of constraints satisfied by this assignment.
