Search Results for author:
Found 72 papers, 34 papers with code
Task Singular Vectors: Reducing Task Interference in Model Merging
In this paper, we study task vectors at the layer level, focusing on task layer matrices and their singular value decomposition.
Interpreting Temporal Graph Neural Networks with Koopman Theory
The second relies on sparse identification of nonlinear dynamics (SINDy), a popular method for discovering governing equations, which we use for the first time as a general tool for explainability.
How to Connect Speech Foundation Models and Large Language Models? What Matters and What Does Not
The remarkable performance achieved by Large Language Models (LLM) has driven research efforts to leverage them for a wide range of tasks and input modalities.
Topological Deep Learning with State-Space Models: A Mamba Approach for Simplicial Complexes
Graph Neural Networks based on the message-passing (MP) mechanism are a dominant approach for handling graph-structured data.
Adaptive Layer Selection for Efficient Vision Transformer Fine-Tuning
To this end, in this paper we introduce an efficient fine-tuning method for ViTs called $\textbf{ALaST}$ ($\textit{Adaptive Layer Selection Fine-Tuning for Vision Transformers}$) to speed up the fine-tuning process while reducing computational cost, memory load, and training time.
A Simple and Effective $L_2$ Norm-Based Strategy for KV Cache Compression
Existing approaches to reduce the KV cache size involve either fine-tuning the model to learn a compression strategy or leveraging attention scores to reduce the sequence length.
TopoBenchmarkX: A Framework for Benchmarking Topological Deep Learning
This work introduces TopoBenchmarkX, a modular open-source library designed to standardize benchmarking and accelerate research in Topological Deep Learning (TDL).
Alice's Adventures in a Differentiable Wonderland -- Volume I, A Tour of the Land
Neural networks surround us, in the form of large language models, speech transcription systems, molecular discovery algorithms, robotics, and much more.
Adaptive Semantic Token Selection for AI-native Goal-oriented Communications
In this paper, we propose a novel design for AI-native goal-oriented communications, exploiting transformer neural networks under dynamic inference constraints on bandwidth and computation.
Influence based explainability of brain tumors segmentation in multimodal Magnetic Resonance Imaging
In recent years Artificial Intelligence has emerged as a fundamental tool in medical applications.
Conditional computation in neural networks: principles and research trends
This article summarizes principles and ideas from the emerging area of applying \textit{conditional computation} methods to the design of neural networks.
Position: Topological Deep Learning is the New Frontier for Relational Learning
At the same time, this paper serves as an invitation to the scientific community to actively participate in TDL research to unlock the potential of this emerging field.
Goal-Oriented and Semantic Communication in 6G AI-Native Networks: The 6G-GOALS Approach
Recent advances in AI technologies have notably expanded device intelligence, fostering federation and cooperation among distributed AI agents.
TopoX: A Suite of Python Packages for Machine Learning on Topological Domains
We introduce TopoX, a Python software suite that provides reliable and user-friendly building blocks for computing and machine learning on topological domains that extend graphs: hypergraphs, simplicial, cellular, path and combinatorial complexes.
Class incremental learning with probability dampening and cascaded gated classifier
The latter is a gated incremental classifier, helping the model modify past predictions without directly interfering with them.
Adaptive Point Transformer
The recent surge in 3D data acquisition has spurred the development of geometric deep learning models for point cloud processing, boosted by the remarkable success of transformers in natural language processing.
NACHOS: Neural Architecture Search for Hardware Constrained Early Exit Neural Networks
To this end, this work presents Neural Architecture Search for Hardware Constrained Early Exit Neural Networks (NACHOS), the first NAS framework for the design of optimal EENNs satisfying constraints on the accuracy and the number of Multiply and Accumulate (MAC) operations performed by the EENNs at inference time.
Adaptive Computation Modules: Granular Conditional Computation For Efficient Inference
The computational cost of transformer models makes them inefficient in low-latency or low-power applications.
Hypergraph Neural Networks through the Lens of Message Passing: A Common Perspective to Homophily and Architecture Design
Most of the current hypergraph learning methodologies and benchmarking datasets in the hypergraph realm are obtained by lifting procedures from their graph analogs, leading to overshadowing specific characteristics of hypergraphs.
Exploiting Activation Sparsity with Dense to Dynamic-k Mixture-of-Experts Conversion
We demonstrate that the efficiency of the conversion can be significantly enhanced by a proper regularization of the activation sparsity of the base model.
ICML 2023 Topological Deep Learning Challenge : Design and Results
This paper presents the computational challenge on topological deep learning that was hosted within the ICML 2023 Workshop on Topology and Geometry in Machine Learning.
Probabilistic load forecasting with Reservoir Computing
For this reason, point forecasts are not enough hence it is necessary to adopt methods that provide an uncertainty quantification.
From Latent Graph to Latent Topology Inference: Differentiable Cell Complex Module
Latent Graph Inference (LGI) relaxed the reliance of Graph Neural Networks (GNNs) on a given graph topology by dynamically learning it.
GeoMultiTaskNet: remote sensing unsupervised domain adaptation using geographical coordinates
This approach is the first to use geographical metadata for UDA in semantic segmentation.
Combining Stochastic Explainers and Subgraph Neural Networks can Increase Expressivity and Interpretability
Subgraph-enhanced graph neural networks (SGNN) can increase the expressive power of the standard message-passing framework.
Drop Edges and Adapt: a Fairness Enforcing Fine-tuning for Graph Neural Networks
One novelty of DEA is that we can use a discrete yet learnable adjacency matrix in our fine-tuning.
EGG-GAE: scalable graph neural networks for tabular data imputation
Missing data imputation (MDI) is crucial when dealing with tabular datasets across various domains.
Explainability in subgraphs-enhanced Graph Neural Networks
Recently, subgraphs-enhanced Graph Neural Networks (SGNNs) have been introduced to enhance the expressive power of Graph Neural Networks (GNNs), which was proved to be not higher than the 1-dimensional Weisfeiler-Leman isomorphism test.
Centroids Matching: an efficient Continual Learning approach operating in the embedding space
In this paper, we propose a novel regularization method called Centroids Matching, that, inspired by meta-learning approaches, fights CF by operating in the feature space produced by the neural network, achieving good results while requiring a small memory footprint.
Inferring 3D change detection from bitemporal optical images
Our work goes one step further, proposing two novel networks, able to solve simultaneously the 2D and 3D CD tasks, and the 3DCD dataset, a novel and freely available dataset precisely designed for this multitask.
Continual Barlow Twins: continual self-supervised learning for remote sensing semantic segmentation
In the field of Earth Observation (EO), Continual Learning (CL) algorithms have been proposed to deal with large datasets by decomposing them into several subsets and processing them incrementally.
Engagement Detection with Multi-Task Training in E-Learning Environments
Recognition of user interaction, in particular engagement detection, became highly crucial for online working and learning environments, especially during the COVID-19 outbreak.
Learning Speech Emotion Representations in the Quaternion Domain
On the one hand, the classifier permits to optimize each latent axis of the embeddings for the classification of a specific emotion-related characteristic: valence, arousal, dominance and overall emotion.
Towards Self-Supervised Gaze Estimation
Recent joint embedding-based self-supervised methods have surpassed standard supervised approaches on various image recognition tasks such as image classification.
Continual Learning with Invertible Generative Models
We show that our method performs favorably with respect to state-of-the-art approaches in the literature, with bounded computational power and memory overheads.
Pixle: a fast and effective black-box attack based on rearranging pixels
Recent research has found that neural networks are vulnerable to several types of adversarial attacks, where the input samples are modified in such a way that the model produces a wrong prediction that misclassifies the adversarial sample.
A Meta-Learning Approach for Training Explainable Graph Neural Networks
Experiments on synthetic and real-world datasets for node and graph classification show that we can produce models that are consistently easier to explain by different algorithms.
Structured Ensembles: an Approach to Reduce the Memory Footprint of Ensemble Methods
In this paper, we propose a novel ensembling technique for deep neural networks, which is able to drastically reduce the required memory compared to alternative approaches.
FairDrop: Biased Edge Dropout for Enhancing Fairness in Graph Representation Learning
Furthermore, to better evaluate the gains, we propose a new dyadic group definition to measure the bias of a link prediction task when paired with group-based fairness metrics.
A New Class of Efficient Adaptive Filters for Online Nonlinear Modeling
In order to make this class of functional link adaptive filters (FLAFs) efficient, we propose low-complexity expansions and frequency-domain adaptation of the parameters.
Avalanche: an End-to-End Library for Continual Learning
Learning continually from non-stationary data streams is a long-standing goal and a challenging problem in machine learning.
Combined Sparse Regularization for Nonlinear Adaptive Filters
Nonlinear adaptive filters often show some sparse behavior due to the fact that not all the coefficients are equally useful for the modeling of any nonlinearity.
Distributed Training of Graph Convolutional Networks
After formulating the centralized GCN training problem, we first show how to make inference in a distributed scenario where the underlying data graph is split among different agents.
Pseudo-Rehearsal for Continual Learning with Normalizing Flows
We show that our method performs favorably with respect to state-of-the-art approaches in the literature, with bounded computational power and memory overheads.
Distributed Stochastic Nonconvex Optimization and Learning based on Successive Convex Approximation
We study distributed stochastic nonconvex optimization in multi-agent networks.
Why should we add early exits to neural networks?
Deep neural networks are generally designed as a stack of differentiable layers, in which a prediction is obtained only after running the full stack.
Bayesian Neural Networks With Maximum Mean Discrepancy Regularization
Bayesian Neural Networks (BNNs) are trained to optimize an entire distribution over their weights instead of a single set, having significant advantages in terms of, e. g., interpretability, multi-task learning, and calibration.
Deep Randomized Neural Networks
For both, we focus specifically on recent results in the domain of deep randomized systems, and (for recurrent models) their application to structured domains.
Adaptive Propagation Graph Convolutional Network
Graph convolutional networks (GCNs) are a family of neural network models that perform inference on graph data by interleaving vertex-wise operations and message-passing exchanges across nodes.
Efficient Continual Learning in Neural Networks with Embedding Regularization
Continual learning of deep neural networks is a key requirement for scaling them up to more complex applicative scenarios and for achieving real lifelong learning of these architectures.
A Multimodal Deep Network for the Reconstruction of T2W MR Images
In this paper, we present a deep learning method that is able to reconstruct subsampled MR images obtained by reducing the k-space data, while maintaining a high image quality that can be used to observe brain lesions.
Compressing deep quaternion neural networks with targeted regularization
To this end, we investigate two extensions of l1 and structured regularization to the quaternion domain.
Efficient data augmentation using graph imputation neural networks
Recently, data augmentation in the semi-supervised regime, where unlabeled data vastly outnumbers labeled data, has received a considerable attention.
Missing Data Imputation with Adversarially-trained Graph Convolutional Networks
We also explore a few extensions to the basic architecture involving the use of residual connections between layers, and of global statistics computed from the data set to improve the accuracy.
On the Stability and Generalization of Learning with Kernel Activation Functions
In this brief we investigate the generalization properties of a recently-proposed class of non-parametric activation functions, the kernel activation functions (KAFs).
Widely Linear Kernels for Complex-Valued Kernel Activation Functions
Complex-valued neural networks (CVNNs) have been shown to be powerful nonlinear approximators when the input data can be properly modeled in the complex domain.
Multikernel activation functions: formulation and a case study
In this paper we focus on the kernel activation function (KAF), a recently proposed framework wherein each function is modeled as a one-dimensional kernel model, whose weights are adapted through standard backpropagation-based optimization.
Quaternion Convolutional Neural Networks for Detection and Localization of 3D Sound Events
Learning from data in the quaternion domain enables us to exploit internal dependencies of 4D signals and treating them as a single entity.
Recurrent Neural Networks with Flexible Gates using Kernel Activation Functions
Gated recurrent neural networks have achieved remarkable results in the analysis of sequential data.
Reservoir computing approaches for representation and classification of multivariate time series
The architectures are compared to other MTS classifiers, including deep learning models and time series kernels.
Improving Graph Convolutional Networks with Non-Parametric Activation Functions
Graph neural networks (GNNs) are a class of neural networks that allow to efficiently perform inference on data that is associated to a graph structure, such as, e. g., citation networks or knowledge graphs.
Complex-valued Neural Networks with Non-parametric Activation Functions
Complex-valued neural networks (CVNNs) are a powerful modeling tool for domains where data can be naturally interpreted in terms of complex numbers.
Bidirectional deep-readout echo state networks
We propose a deep architecture for the classification of multivariate time series.
Kafnets: kernel-based non-parametric activation functions for neural networks
Neural networks are generally built by interleaving (adaptable) linear layers with (fixed) nonlinear activation functions.
Stochastic Training of Neural Networks via Successive Convex Approximations
Additionally, we show how the algorithm can be easily parallelized over multiple computational units without hindering its performance.
Recursive Multikernel Filters Exploiting Nonlinear Temporal Structure
In kernel methods, temporal information on the data is commonly included by using time-delayed embeddings as inputs.
Adaptation and learning over networks for nonlinear system modeling
In this chapter, we analyze nonlinear filtering problems in distributed environments, e. g., sensor networks or peer-to-peer protocols.
A Framework for Parallel and Distributed Training of Neural Networks
The aim of this paper is to develop a general framework for training neural networks (NNs) in a distributed environment, where training data is partitioned over a set of agents that communicate with each other through a sparse, possibly time-varying, connectivity pattern.
Distributed Supervised Learning using Neural Networks
Distributed learning is the problem of inferring a function in the case where training data is distributed among multiple geographically separated sources.
Group Sparse Regularization for Deep Neural Networks
In this paper, we consider the joint task of simultaneously optimizing (i) the weights of a deep neural network, (ii) the number of neurons for each hidden layer, and (iii) the subset of active input features (i. e., feature selection).
Effective Blind Source Separation Based on the Adam Algorithm
In this paper, we derive a modified InfoMax algorithm for the solution of Blind Signal Separation (BSS) problems by using advanced stochastic methods.
Learning activation functions from data using cubic spline interpolation
Neural networks require a careful design in order to perform properly on a given task.
