Search Results for author:
Found 25 papers, 13 papers with code
Large Language Models are Interpretable Learners
In the proposed LLM-based Symbolic Programs (LSPs), the pretrained LLM with natural language prompts provides a massive set of interpretable modules that can transform raw input into natural language concepts.
Gemini 1.5: Unlocking multimodal understanding across millions of tokens of context
In this report, we introduce the Gemini 1. 5 family of models, representing the next generation of highly compute-efficient multimodal models capable of recalling and reasoning over fine-grained information from millions of tokens of context, including multiple long documents and hours of video and audio.
Dual-Encoders for Extreme Multi-Label Classification
We propose decoupled softmax loss - a simple modification to the InfoNCE loss - that overcomes the limitations of existing contrastive losses.
EHI: End-to-end Learning of Hierarchical Index for Efficient Dense Retrieval
To facilitate the effective learning of this discrete structure, EHI introduces dense path embeddings that encodes the path traversed by queries and documents within the tree.
MatFormer: Nested Transformer for Elastic Inference
Foundation models are applied in a broad spectrum of settings with different inference constraints, from massive multi-accelerator clusters to resource-constrained standalone mobile devices.
Bayesian regularization of empirical MDPs
Due to noise, the policy learnedfrom the estimated model is often far from the optimal policy of the underlying model.
Positive Unlabeled Contrastive Learning
We extend this paradigm to the classical positive unlabeled (PU) setting, where the task is to learn a binary classifier given only a few labeled positive samples, and (often) a large amount of unlabeled samples (which could be positive or negative).
Accelerating Primal-dual Methods for Regularized Markov Decision Processes
Entropy regularized Markov decision processes have been widely used in reinforcement learning.
Extreme Zero-Shot Learning for Extreme Text Classification
To learn the semantic embeddings of instances and labels with raw text, we propose to pre-train Transformer-based encoders with self-supervised contrastive losses.
Multi Label Text Classification
Multi-Label Text Classification
+2
DRONE: Data-aware Low-rank Compression for Large NLP Models
In addition to compressing standard models, out method can also be used on distilled BERT models to further improve compression rate.
Approximate Newton policy gradient algorithms
This paper proposes an approximate Newton method for the policy gradient algorithm with entropy regularization.
Enterprise-Scale Search: Accelerating Inference for Sparse Extreme Multi-Label Ranking Trees
In industrial applications, these models operate at extreme scales, where every bit of performance is critical.
Top-$k$ eXtreme Contextual Bandits with Arm Hierarchy
We show that our algorithm has a regret guarantee of $O(k\sqrt{(A-k+1)T \log (|\mathcal{F}|T)})$, where $A$ is the total number of arms and $\mathcal{F}$ is the class containing the regression function, while only requiring $\tilde{O}(A)$ computation per time step.
Computational Efficiency
Extreme Multi-Label Classification
+3
Voting based ensemble improves robustness of defensive models
Several previous attempts tackled this problem by ensembling the soft-label prediction and have been proved vulnerable based on the latest attack methods.
On the Benefits of Multiple Gossip Steps in Communication-Constrained Decentralized Optimization
In this paper, we show that, in such compressed decentralized optimization settings, there are benefits to having {\em multiple} gossip steps between subsequent gradient iterations, even when the cost of doing so is appropriately accounted for e. g. by means of reducing the precision of compressed information.
Extreme Multi-label Classification from Aggregated Labels
Current XMC approaches are not built for such multi-instance multi-label (MIML) training data, and MIML approaches do not scale to XMC sizes.
Learning to Encode Position for Transformer with Continuous Dynamical Model
The main reason is that position information among input units is not inherently encoded, i. e., the models are permutation equivalent; this problem justifies why all of the existing models are accompanied by a sinusoidal encoding/embedding layer at the input.
Ranked #5 on
Semantic Textual Similarity
on MRPC
CAT: Customized Adversarial Training for Improved Robustness
Adversarial training has become one of the most effective methods for improving robustness of neural networks.
Think Globally, Act Locally: A Deep Neural Network Approach to High-Dimensional Time Series Forecasting
Forecasting high-dimensional time series plays a crucial role in many applications such as demand forecasting and financial predictions.
Taming Pretrained Transformers for Extreme Multi-label Text Classification
However, naively applying deep transformer models to the XMC problem leads to sub-optimal performance due to the large output space and the label sparsity issue.
Extreme Multi-Label Classification
General Classification
+4
Online Embedding Compression for Text Classification using Low Rank Matrix Factorization
Empirically, we show that the proposed method can achieve 90% compression with minimal impact in accuracy for sentence classification tasks, and outperforms alternative methods like fixed-point quantization or offline word embedding compression.
Kernel Ridge Regression via Partitioning
In this paper, we investigate a divide and conquer approach to Kernel Ridge Regression (KRR).
Structured Sparse Regression via Greedy Hard-Thresholding
Several learning applications require solving high-dimensional regression problems where the relevant features belong to a small number of (overlapping) groups.
Coordinate Descent Methods for Symmetric Nonnegative Matrix Factorization
Given a symmetric nonnegative matrix $A$, symmetric nonnegative matrix factorization (symNMF) is the problem of finding a nonnegative matrix $H$, usually with much fewer columns than $A$, such that $A \approx HH^T$.
NOMAD: Non-locking, stOchastic Multi-machine algorithm for Asynchronous and Decentralized matrix completion
One of the key features of NOMAD is that the ownership of a variable is asynchronously transferred between processors in a decentralized fashion.
Distributed, Parallel, and Cluster Computing
