Search Results for author:
Found 13 papers, 3 papers with code
Unveiling the Secret Recipe: A Guide For Supervised Fine-Tuning Small LLMs
The rise of large language models (LLMs) has created a significant disparity: industrial research labs with their computational resources, expert teams, and advanced infrastructures, can effectively fine-tune LLMs, while individual developers and small organizations face barriers due to limited resources.
Emergence of Abstractions: Concept Encoding and Decoding Mechanism for In-Context Learning in Transformers
In this paper, we propose concept encoding-decoding mechanism to explain ICL by studying how transformers form and use internal abstractions in their representations.
Value Augmented Sampling for Language Model Alignment and Personalization
Aligning Large Language Models (LLMs) to cater to different human preferences, learning new skills, and unlearning harmful behavior is an important problem.
Estimating the Density Ratio between Distributions with High Discrepancy using Multinomial Logistic Regression
We show that if these auxiliary densities are constructed such that they overlap with $p$ and $q$, then a multi-class logistic regression allows for estimating $\log p/q$ on the domain of any of the $K+2$ distributions and resolves the distribution shift problems of the current state-of-the-art methods.
Constructive Assimilation: Boosting Contrastive Learning Performance through View Generation Strategies
Recently, several attempts have been made to replace such domain-specific, human-designed transformations with generated views that are learned.
Multi-Symmetry Ensembles: Improving Diversity and Generalization via Opposing Symmetries
In this work, we present Multi-Symmetry Ensembles (MSE), a framework for constructing diverse ensembles by capturing the multiplicity of hypotheses along symmetry axes, which explore the hypothesis space beyond stochastic perturbations of model weights and hyperparameters.
On the Importance of Calibration in Semi-supervised Learning
State-of-the-art (SOTA) semi-supervised learning (SSL) methods have been highly successful in leveraging a mix of labeled and unlabeled data by combining techniques of consistency regularization and pseudo-labeling.
Equivariant Contrastive Learning
In state-of-the-art self-supervised learning (SSL) pre-training produces semantically good representations by encouraging them to be invariant under meaningful transformations prescribed from human knowledge.
Equivariant Self-Supervised Learning: Encouraging Equivariance in Representations
In state-of-the-art self-supervised learning (SSL) pre-training produces semantically good representations by encouraging them to be invariant under meaningful transformations prescribed from human knowledge.
Scaling Densities For Improved Density Ratio Estimation
As such, estimating density ratios accurately using only samples from $p$ and $q$ is of high significance and has led to a flurry of recent work in this direction.
not-so-big-GAN: Generating High-Fidelity Images on Small Compute with Wavelet-based Super-Resolution
First, we generate images in low-frequency bands by training a sampler in the wavelet domain.
not-so-BigGAN: Generating High-Fidelity Images on Small Compute with Wavelet-based Super-Resolution
First, we generate images in low-frequency bands by training a sampler in the wavelet domain.
Predicting the accuracy of neural networks from final and intermediate layer outputs
We show that information about whether a neural network's output will be correct or incorrect is present in the outputs of the network's intermediate layers.
