Search Results for author:
Found 28 papers, 14 papers with code
Symmetric Pruning of Large Language Models
Popular post-training pruning methods such as Wanda and RIA are known for their simple, yet effective, designs that have shown exceptional empirical performance.
Score-matching-based Structure Learning for Temporal Data on Networks
To enhance the scalability of score matching, we have developed a new parent-finding subroutine for leaf nodes in DAGs, significantly accelerating the most time-consuming part of the process: the pruning step.
MANTA: A Large-Scale Multi-View and Visual-Text Anomaly Detection Dataset for Tiny Objects
We present MANTA, a visual-text anomaly detection dataset for tiny objects.
A Regressor-Guided Graph Diffusion Model for Predicting Enzyme Mutations to Enhance Turnover Number
kcatDiffuser is a graph diffusion model guided by a regressor, enabling the prediction of amino acid mutations at multiple random positions simultaneously.
Cohort Squeeze: Beyond a Single Communication Round per Cohort in Cross-Device Federated Learning
Virtually all federated learning (FL) methods, including FedAvg, operate in the following manner: i) an orchestrating server sends the current model parameters to a cohort of clients selected via certain rule, ii) these clients then independently perform a local training procedure (e. g., via SGD or Adam) using their own training data, and iii) the resulting models are shipped to the server for aggregation.
Sparse-ProxSkip: Accelerated Sparse-to-Sparse Training in Federated Learning
In Federated Learning (FL), both client resource constraints and communication costs pose major problems for training large models.
PV-Tuning: Beyond Straight-Through Estimation for Extreme LLM Compression
In this work, we question the use of STE for extreme LLM compression, showing that it can be sub-optimal, and perform a systematic study of quantization-aware fine-tuning strategies for LLMs.
FedP3: Federated Personalized and Privacy-friendly Network Pruning under Model Heterogeneity
The interest in federated learning has surged in recent research due to its unique ability to train a global model using privacy-secured information held locally on each client.
DiffImpute: Tabular Data Imputation With Denoising Diffusion Probabilistic Model
Specifically, DiffImpute is trained on complete tabular datasets, ensuring that it can produce credible imputations for missing entries without undermining the authenticity of the existing data.
FedComLoc: Communication-Efficient Distributed Training of Sparse and Quantized Models
Federated Learning (FL) has garnered increasing attention due to its unique characteristic of allowing heterogeneous clients to process their private data locally and interact with a central server, while being respectful of privacy.
Continual Zero-Shot Learning through Semantically Guided Generative Random Walks
The GRW loss augments the training by continually encouraging the model to generate realistic and characterized samples to represent the unseen space.
Graph Denoising Diffusion for Inverse Protein Folding
In contrast, diffusion probabilistic models, as an emerging genre of generative approaches, offer the potential to generate a diverse set of sequence candidates for determined protein backbones.
Explicit Personalization and Local Training: Double Communication Acceleration in Federated Learning
Federated Learning is an evolving machine learning paradigm, in which multiple clients perform computations based on their individual private data, interspersed by communication with a remote server.
Accurate and Definite Mutational Effect Prediction with Lightweight Equivariant Graph Neural Networks
Directed evolution as a widely-used engineering strategy faces obstacles in finding desired mutants from the massive size of candidate modifications.
Variance Reduced ProxSkip: Algorithm, Theory and Application to Federated Learning
We study distributed optimization methods based on the {\em local training (LT)} paradigm: achieving communication efficiency by performing richer local gradient-based training on the clients before parameter averaging.
Approximate Equivariance SO(3) Needlet Convolution
This paper develops a rotation-invariant needlet convolution for rotation group SO(3) to distill multiscale information of spherical signals.
ACMP: Allen-Cahn Message Passing for Graph Neural Networks with Particle Phase Transition
Neural message passing is a basic feature extraction unit for graph-structured data considering neighboring node features in network propagation from one layer to the next.
EF-BV: A Unified Theory of Error Feedback and Variance Reduction Mechanisms for Biased and Unbiased Compression in Distributed Optimization
Our general approach works with a new, larger class of compressors, which has two parameters, the bias and the variance, and includes unbiased and biased compressors as particular cases.
Exploring Hierarchical Graph Representation for Large-Scale Zero-Shot Image Classification
The main question we address in this paper is how to scale up visual recognition of unseen classes, also known as zero-shot learning, to tens of thousands of categories as in the ImageNet-21K benchmark.
Domain-Aware Continual Zero-Shot Learning
Accordingly, we propose a Domain-Invariant Network (DIN) to learn factorized features for shifting domains and improved textual representation for unseen classes.
Disentangling semantic features of macromolecules in Cryo-Electron Tomography
Cryo-electron tomography (Cryo-ET) is a 3D imaging technique that enables the systemic study of shape, abundance, and distribution of macromolecular structures in single cells in near-atomic resolution.
Imaginative Walks: Generative Random Walk Deviation Loss for Improved Unseen Learning Representation
By generating representations of unseen classes based on their semantic descriptions, e. g., attributes or text, generative ZSL attempts to differentiate unseen from seen categories.
VisualGPT: Data-efficient Adaptation of Pretrained Language Models for Image Captioning
To the best of our knowledge, this is the first work that improves data efficiency of image captioning by utilizing LM pretrained on unimodal data.
CIZSL++: Creativity Inspired Generative Zero-Shot Learning
To improve the discriminative power of ZSL, we model the visual learning process of unseen categories with inspiration from the psychology of human creativity for producing novel art.
Experimental Analysis of Legendre Decomposition in Machine Learning
In this technical report, we analyze Legendre decomposition for non-negative tensor in theory and application.
Deep Learning Based Unsupervised and Semi-supervised Classification for Keratoconus
Currently, there is no cure for keratoconus other than corneal transplantation for advanced stage keratoconus or corneal cross-linking, which can only halt KC progression.
CosmoVAE: Variational Autoencoder for CMB Image Inpainting
The noise of the CMB map has a significant impact on the estimation precision for cosmological parameters.
Feature Selective Small Object Detection via Knowledge-based Recurrent Attentive Neural Network
At present, the performance of deep neural network in general object detection is comparable to or even surpasses that of human beings.
