Search Results for author: Parikshit Ram

Found 33 papers, 12 papers with code

Enhancing In-context Learning via Linear Probe Calibration

1 code implementation22 Jan 2024 Momin Abbas, Yi Zhou, Parikshit Ram, Nathalie Baracaldo, Horst Samulowitz, Theodoros Salonidis, Tianyi Chen

However, applying ICL in real cases does not scale with the number of samples, and lacks robustness to different prompt templates and demonstration permutations.

In-Context Learning

Compositional Program Generation for Few-Shot Systematic Generalization

1 code implementation28 Sep 2023 Tim Klinger, Luke Liu, Soham Dan, Maxwell Crouse, Parikshit Ram, Alexander Gray

Compositional generalization is a key ability of humans that enables us to learn new concepts from only a handful examples.

Systematic Generalization

End-to-end Differentiable Clustering with Associative Memories

1 code implementation5 Jun 2023 Bishwajit Saha, Dmitry Krotov, Mohammed J. Zaki, Parikshit Ram

Clustering is a widely used unsupervised learning technique involving an intensive discrete optimization problem.

Clustering

Model Sparsity Can Simplify Machine Unlearning

1 code implementation NeurIPS 2023 Jinghan Jia, Jiancheng Liu, Parikshit Ram, Yuguang Yao, Gaowen Liu, Yang Liu, Pranay Sharma, Sijia Liu

We show in both theory and practice that model sparsity can boost the multi-criteria unlearning performance of an approximate unlearner, closing the approximation gap, while continuing to be efficient.

Machine Unlearning Transfer Learning

What Is Missing in IRM Training and Evaluation? Challenges and Solutions

no code implementations4 Mar 2023 Yihua Zhang, Pranay Sharma, Parikshit Ram, Mingyi Hong, Kush Varshney, Sijia Liu

We propose a new IRM variant to address this limitation based on a novel viewpoint of ensemble IRM games as consensus-constrained bi-level optimization.

Out-of-Distribution Generalization

Toward Theoretical Guidance for Two Common Questions in Practical Cross-Validation based Hyperparameter Selection

no code implementations12 Jan 2023 Parikshit Ram, Alexander G. Gray, Horst C. Samulowitz, Gregory Bramble

We show, to our knowledge, the first theoretical treatments of two common questions in cross-validation based hyperparameter selection: (1) After selecting the best hyperparameter using a held-out set, we train the final model using {\em all} of the training data -- since this may or may not improve future generalization error, should one do this?

Navigating Ensemble Configurations for Algorithmic Fairness

no code implementations11 Oct 2022 Michael Feffer, Martin Hirzel, Samuel C. Hoffman, Kiran Kate, Parikshit Ram, Avraham Shinnar

Bias mitigators can improve algorithmic fairness in machine learning models, but their effect on fairness is often not stable across data splits.

Ensemble Learning Fairness +1

Advancing Model Pruning via Bi-level Optimization

1 code implementation8 Oct 2022 Yihua Zhang, Yuguang Yao, Parikshit Ram, Pu Zhao, Tianlong Chen, Mingyi Hong, Yanzhi Wang, Sijia Liu

To reduce the computation overhead, various efficient 'one-shot' pruning methods have been developed, but these schemes are usually unable to find winning tickets as good as IMP.

Min-Max Bilevel Multi-objective Optimization with Applications in Machine Learning

1 code implementation3 Mar 2022 Alex Gu, Songtao Lu, Parikshit Ram, Lily Weng

We consider a generic min-max multi-objective bilevel optimization problem with applications in robust machine learning such as representation learning and hyperparameter optimization.

BIG-bench Machine Learning Bilevel Optimization +4

FLoRA: Single-shot Hyper-parameter Optimization for Federated Learning

no code implementations15 Dec 2021 Yi Zhou, Parikshit Ram, Theodoros Salonidis, Nathalie Baracaldo, Horst Samulowitz, Heiko Ludwig

We address the relatively unexplored problem of hyper-parameter optimization (HPO) for federated learning (FL-HPO).

Federated Learning

Federated Nearest Neighbor Classification with a Colony of Fruit-Flies: With Supplement

1 code implementation14 Dec 2021 Parikshit Ram, Kaushik Sinha

The mathematical formalization of a neurological mechanism in the olfactory circuit of a fruit-fly as a locality sensitive hash (Flyhash) and bloom filter (FBF) has been recently proposed and "reprogrammed" for various machine learning tasks such as similarity search, outlier detection and text embeddings.

Federated Learning Outlier Detection

Sign-MAML: Efficient Model-Agnostic Meta-Learning by SignSGD

1 code implementation15 Sep 2021 Chen Fan, Parikshit Ram, Sijia Liu

The key enabling technique is to interpret MAML as a bilevel optimization (BLO) problem and leverage the sign-based SGD(signSGD) as a lower-level optimizer of BLO.

Bilevel Optimization Few-Shot Image Classification +1

Designing Machine Learning Pipeline Toolkit for AutoML Surrogate Modeling Optimization

no code implementations2 Jul 2021 Paulito P. Palmes, Akihiro Kishimoto, Radu Marinescu, Parikshit Ram, Elizabeth Daly

The pipeline optimization problem in machine learning requires simultaneous optimization of pipeline structures and parameter adaptation of their elements.

AutoML BIG-bench Machine Learning +1

Leveraging Theoretical Tradeoffs in Hyperparameter Selection for Improved Empirical Performance

no code implementations ICML Workshop AutoML 2021 Parikshit Ram, Alexander G. Gray, Horst Samulowitz

The tradeoffs in the excess risk incurred from data-driven learning of a single model has been studied by decomposing the excess risk into approximation, estimation and optimization errors.

Hyperparameter Optimization

Overcoming Catastrophic Forgetting via Direction-Constrained Optimization

1 code implementation25 Nov 2020 Yunfei Teng, Anna Choromanska, Murray Campbell, Songtao Lu, Parikshit Ram, Lior Horesh

We study the principal directions of the trajectory of the optimizer after convergence and show that traveling along a few top principal directions can quickly bring the parameters outside the cone but this is not the case for the remaining directions.

Continual Learning

Learning to Generate Image Source-Agnostic Universal Adversarial Perturbations

no code implementations29 Sep 2020 Pu Zhao, Parikshit Ram, Songtao Lu, Yuguang Yao, Djallel Bouneffouf, Xue Lin, Sijia Liu

The resulting scheme for meta-learning a UAP generator (i) has better performance (50% higher ASR) than baselines such as Projected Gradient Descent, (ii) has better performance (37% faster) than the vanilla L2O and MAML frameworks (when applicable), and (iii) is able to simultaneously handle UAP generation for different victim models and image data sources.

Adversarial Attack Bilevel Optimization +1

Neural Neighborhood Encoding for Classification

no code implementations19 Aug 2020 Kaushik Sinha, Parikshit Ram

Inspired by the fruit-fly olfactory circuit, the Fly Bloom Filter [Dasgupta et al., 2018] is able to efficiently summarize the data with a single pass and has been used for novelty detection.

Classification General Classification +2

Lale: Consistent Automated Machine Learning

1 code implementation4 Jul 2020 Guillaume Baudart, Martin Hirzel, Kiran Kate, Parikshit Ram, Avraham Shinnar

Automated machine learning makes it easier for data scientists to develop pipelines by searching over possible choices for hyperparameters, algorithms, and even pipeline topologies.

BIG-bench Machine Learning

Solving Constrained CASH Problems with ADMM

no code implementations17 Jun 2020 Parikshit Ram, Sijia Liu, Deepak Vijaykeerthi, Dakuo Wang, Djallel Bouneffouf, Greg Bramble, Horst Samulowitz, Alexander G. Gray

The CASH problem has been widely studied in the context of automated configurations of machine learning (ML) pipelines and various solvers and toolkits are available.

BIG-bench Machine Learning Fairness

Type-Driven Automated Learning with Lale

2 code implementations24 May 2019 Martin Hirzel, Kiran Kate, Avraham Shinnar, Subhrajit Roy, Parikshit Ram

Machine-learning automation tools, ranging from humble grid-search to hyperopt, auto-sklearn, and TPOT, help explore large search spaces of possible pipelines.

Time Series Time Series Analysis +1

An ADMM Based Framework for AutoML Pipeline Configuration

no code implementations1 May 2019 Sijia Liu, Parikshit Ram, Deepak Vijaykeerthy, Djallel Bouneffouf, Gregory Bramble, Horst Samulowitz, Dakuo Wang, Andrew Conn, Alexander Gray

We study the AutoML problem of automatically configuring machine learning pipelines by jointly selecting algorithms and their appropriate hyper-parameters for all steps in supervised learning pipelines.

AutoML Binary Classification

Plug-and-play dual-tree algorithm runtime analysis

no code implementations21 Jan 2015 Ryan R. Curtin, Dongryeol Lee, William B. March, Parikshit Ram

In this paper, we present a problem-independent runtime guarantee for any dual-tree algorithm using the cover tree, separating out the problem-dependent and the problem-independent elements.

Density Estimation

Which Space Partitioning Tree to Use for Search?

no code implementations NeurIPS 2013 Parikshit Ram, Alexander Gray

We consider the task of nearest-neighbor search with the class of binary-space-partitioning trees, which includes kd-trees, principal axis trees and random projection trees, and try to rigorously answer the question which tree to use for nearest-neighbor search?''

Quantization

Maximum Inner-Product Search using Tree Data-structures

1 code implementation28 Feb 2012 Parikshit Ram, Alexander G. Gray

Finally we present a new data structure for increasing the efficiency of the dual-tree algorithm.

Rank-Approximate Nearest Neighbor Search: Retaining Meaning and Speed in High Dimensions

no code implementations NeurIPS 2009 Parikshit Ram, Dongryeol Lee, Hua Ouyang, Alexander G. Gray

The long-standing problem of efficient nearest-neighbor (NN) search has ubiquitous applications ranging from astrophysics to MP3 fingerprinting to bioinformatics to movie recommendations.

Vocal Bursts Intensity Prediction

Linear-time Algorithms for Pairwise Statistical Problems

no code implementations NeurIPS 2009 Parikshit Ram, Dongryeol Lee, William March, Alexander G. Gray

Several key computational bottlenecks in machine learning involve pairwise distance computations, including all-nearest-neighbors (finding the nearest neighbor(s) for each point, e. g. in manifold learning) and kernel summations (e. g. in kernel density estimation or kernel machines).

BIG-bench Machine Learning Density Estimation

Cannot find the paper you are looking for? You can Submit a new open access paper.