1 code implementation • 22 Jan 2025 • Abhinav Moudgil, Boris Knyazev, Guillaume Lajoie, Eugene Belilovsky
Learned optimization has emerged as a promising alternative to hand-crafted optimizers, with the potential to discover stronger learned update rules that enable faster, hyperparameter-free training of neural networks.
1 code implementation • 6 Sep 2024 • Boris Knyazev, Abhinav Moudgil, Guillaume Lajoie, Eugene Belilovsky, Simon Lacoste-Julien
Neural network training can be accelerated when a learnable update rule is used in lieu of classic adaptive optimizers (e. g. Adam).
1 code implementation • 12 Jul 2024 • Alexia Jolicoeur-Martineau, Aristide Baratin, Kisoo Kwon, Boris Knyazev, Yan Zhang
Generating novel molecules is challenging, with most representations leading to generative models producing many invalid molecules.
1 code implementation • 31 May 2024 • Benjamin Thérien, Charles-Étienne Joseph, Boris Knyazev, Edouard Oyallon, Irina Rish, Eugene Belilovsky
We extend $\mu$P theory to learned optimizers, treating the meta-training problem as finding the learned optimizer under $\mu$P.
1 code implementation • 25 May 2024 • Xinyu Zhou, Boris Knyazev, Alexia Jolicoeur-Martineau, Jie Fu
Unfortunately, predicting parameters of very wide networks relies on copying small chunks of parameters multiple times and requires an extremely large number of parameters to support full prediction, which greatly hinders its adoption in practice.
1 code implementation • 18 Mar 2024 • Miltiadis Kofinas, Boris Knyazev, Yan Zhang, Yunlu Chen, Gertjan J. Burghouts, Efstratios Gavves, Cees G. M. Snoek, David W. Zhang
Neural networks that process the parameters of other neural networks find applications in domains as diverse as classifying implicit neural representations, generating neural network weights, and predicting generalization errors.
no code implementations • 2 Dec 2023 • Charles-Étienne Joseph, Benjamin Thérien, Abhinav Moudgil, Boris Knyazev, Eugene Belilovsky
Although many variants of these approaches have been proposed, they can sometimes lag behind state-of-the-art adaptive optimizers for deep learning.
2 code implementations • 7 Mar 2023 • Boris Knyazev, Doha Hwang, Simon Lacoste-Julien
Pretraining a neural network on a large dataset is becoming a cornerstone in machine learning that is within the reach of only a few communities with large-resources.
1 code implementation • 29 Sep 2022 • Konstantin Schürholt, Boris Knyazev, Xavier Giró-i-Nieto, Damian Borth
Learning representations of neural network weights given a model zoo is an emerging and challenging area with many potential applications from model inspection, to neural architecture search or knowledge distillation.
1 code implementation • 29 Sep 2022 • Konstantin Schürholt, Diyar Taskiran, Boris Knyazev, Xavier Giró-i-Nieto, Damian Borth
With this work, we publish a novel dataset of model zoos containing systematically generated and diverse populations of NN models for further research.
1 code implementation • 22 Jul 2022 • Konstantin Schürholt, Boris Knyazev, Xavier Giró-i-Nieto, Damian Borth
Learning representations of neural network weights given a model zoo is an emerging and challenging area with many potential applications from model inspection, to neural architecture search or knowledge distillation.
2 code implementations • 20 Jul 2022 • Boris Knyazev
A recently released Graph HyperNetwork (GHN) trained this way on one million smaller ImageNet architectures is able to predict parameters for large unseen networks such as ResNet-50.
1 code implementation • ICLR 2022 • Rylee Thompson, Boris Knyazev, Elahe Ghalebi, Jungtaek Kim, Graham W. Taylor
While we focus on applying these metrics to GGM evaluation, in practice this enables the ability to easily compute the dissimilarity between any two sets of graphs regardless of domain.
no code implementations • NeurIPS 2021 • Hyunsoo Chung, Jungtaek Kim, Boris Knyazev, Jinhwi Lee, Graham W. Taylor, Jaesik Park, Minsu Cho
Discovering a solution in a combinatorial space is prevalent in many real-world problems but it is also challenging due to diverse complex constraints and the vast number of possible combinations.
1 code implementation • NeurIPS 2021 • Boris Knyazev, Michal Drozdzal, Graham W. Taylor, Adriana Romero-Soriano
We introduce a large-scale dataset of diverse computational graphs of neural architectures - DeepNets-1M - and use it to explore parameter prediction on CIFAR-10 and ImageNet.
Ranked #1 on
Parameter Prediction
on CIFAR10
1 code implementation • ICCV 2021 • Yichao Lu, Himanshu Rai, Jason Chang, Boris Knyazev, Guangwei Yu, Shashank Shekhar, Graham W. Taylor, Maksims Volkovs
In this task, the model needs to detect objects and predict visual relationships between them.
1 code implementation • ICCV 2021 • Boris Knyazev, Harm de Vries, Cătălina Cangea, Graham W. Taylor, Aaron Courville, Eugene Belilovsky
However, test images might contain zero- and few-shot compositions of objects and relationships, e. g. <cup, on, surfboard>.
1 code implementation • 17 May 2020 • Boris Knyazev, Harm de Vries, Cătălina Cangea, Graham W. Taylor, Aaron Courville, Eugene Belilovsky
We show that such models can suffer the most in their ability to generalize to rare compositions, evaluating two different models on the Visual Genome dataset and its more recent, improved version, GQA.
1 code implementation • 23 Sep 2019 • Boris Knyazev, Carolyn Augusta, Graham W. Taylor
We consider a common case in which edges can be short term interactions (e. g., messaging) or long term structural connections (e. g., friendship).
1 code implementation • 21 Jul 2019 • Boris Knyazev, Xiao Lin, Mohamed R. Amer, Graham W. Taylor
Graph Convolutional Networks (GCNs) are a class of general models that can learn from graph structured data.
2 code implementations • NeurIPS 2019 • Boris Knyazev, Graham W. Taylor, Mohamed R. Amer
We aim to better understand attention over nodes in graph neural networks (GNNs) and identify factors influencing its effectiveness.
Ranked #27 on
Graph Classification
on D&D
1 code implementation • 23 Nov 2018 • Boris Knyazev, Xiao Lin, Mohamed R. Amer, Graham W. Taylor
Spectral Graph Convolutional Networks (GCNs) are a generalization of convolutional networks to learning on graph-structured data.
Ranked #19 on
Graph Classification
on NCI109
no code implementations • 13 Nov 2017 • Boris Knyazev, Roman Shvetsov, Natalia Efremova, Artem Kuharenko
In this paper we describe a solution to our entry for the emotion recognition challenge EmotiW 2017.
2 code implementations • 2 Jun 2016 • Boris Knyazev, Erhardt Barth, Thomas Martinetz
In visual recognition tasks, such as image classification, unsupervised learning exploits cheap unlabeled data and can help to solve these tasks more efficiently.