Search Results for author:
Found 120 papers, 16 papers with code
Parameter-Efficient Sparsity for Large Language Models Fine-Tuning
With the dramatically increased number of parameters in language models, sparsity methods have received ever-increasing research focus to compress and accelerate the models.
Visual Adversarial Examples Jailbreak Aligned Large Language Models
Recently, there has been a surge of interest in integrating vision into Large Language Models (LLMs), exemplified by Visual Language Models (VLMs) such as Flamingo and GPT-4.
Picasso: A Sparse Learning Library for High Dimensional Data Analysis in R and Python
We describe a new library named picasso, which implements a unified framework of pathwise coordinate optimization for a variety of sparse learning problems (e. g., sparse linear regression, sparse logistic regression, sparse Poisson regression and scaled sparse linear regression) combined with efficient active set selection strategies.
You Only Compress Once: Towards Effective and Elastic BERT Compression via Exploit-Explore Stochastic Nature Gradient
Motivated by the necessity of efficient inference across various constraints on BERT, we propose a novel approach, YOCO-BERT, to achieve compress once and deploy everywhere.
1xN Pattern for Pruning Convolutional Neural Networks
We also provide a workflow of filter rearrangement that first rearranges the weight matrix in the output channel dimension to derive more influential blocks for accuracy improvements and then applies similar rearrangement to the next-layer weights in the input channel dimension to ensure correct convolutional operations.
Efficient Reinforcement Learning in Block MDPs: A Model-free Representation Learning Approach
We present BRIEE (Block-structured Representation learning with Interleaved Explore Exploit), an algorithm for efficient reinforcement learning in Markov Decision Processes with block-structured dynamics (i. e., Block MDPs), where rich observations are generated from a set of unknown latent states.
Provable Benefits of Representational Transfer in Reinforcement Learning
We study the problem of representational transfer in RL, where an agent first pretrains in a number of source tasks to discover a shared representation, which is subsequently used to learn a good policy in a \emph{target task}.
Contrastive Multi-document Question Generation
In the SL stage, a single-document question generator is trained.
Towards Compact CNNs via Collaborative Compression
Channel pruning and tensor decomposition have received extensive attention in convolutional neural network compression.
Deep Reinforcement Learning for Cost-Effective Medical Diagnosis
In this work, we use reinforcement learning (RL) to find a dynamic policy that selects lab test panels sequentially based on previous observations, ensuring accurate testing at a low cost.
Unified Off-Policy Learning to Rank: a Reinforcement Learning Perspective
Building upon this, we leverage offline RL techniques for off-policy LTR and propose the Click Model-Agnostic Unified Off-policy Learning to Rank (CUOLR) method, which could be easily applied to a wide range of click models.
Variance Reduced Value Iteration and Faster Algorithms for Solving Markov Decision Processes
Given a discounted Markov Decision Process (DMDP) with $|S|$ states, $|A|$ actions, discount factor $\gamma\in(0, 1)$, and rewards in the range $[-M, M]$, we show how to compute an $\epsilon$-optimal policy, with probability $1 - \delta$ in time \[ \tilde{O}\left( \left(|S|^2 |A| + \frac{|S| |A|}{(1 - \gamma)^3} \right) \log\left( \frac{M}{\epsilon} \right) \log\left( \frac{1}{\delta} \right) \right) ~ .
Near-Optimal Time and Sample Complexities for Solving Discounted Markov Decision Process with a Generative Model
In this paper we consider the problem of computing an $\epsilon$-optimal policy of a discounted Markov Decision Process (DMDP) provided we can only access its transition function through a generative sampling model that given any state-action pair samples from the transition function in $O(1)$ time.
Optimization and Control
Improved Sample Complexity for Stochastic Compositional Variance Reduced Gradient
Convex composition optimization is an emerging topic that covers a wide range of applications arising from stochastic optimal control, reinforcement learning and multi-stage stochastic programming.
TurboSVM-FL: Boosting Federated Learning through SVM Aggregation for Lazy Clients
Federated learning is a distributed collaborative machine learning paradigm that has gained strong momentum in recent years.
Dimensionality Reduction for Stationary Time Series via Stochastic Nonconvex Optimization
Specifically, our goal is to estimate the principle component of time series data with respect to the covariance matrix of the stationary distribution.
Scalable Bilinear $π$ Learning Using State and Action Features
In this work, we study a primal-dual formulation of the ALP, and develop a scalable, model-free algorithm called bilinear $\pi$ learning for reinforcement learning when a sampling oracle is provided.
Variance Reduction Methods for Sublinear Reinforcement Learning
There is a technical issue in the analysis that is not easily fixable.
Improved Oracle Complexity of Variance Reduced Methods for Nonsmooth Convex Stochastic Composition Optimization
We consider the nonsmooth convex composition optimization problem where the objective is a composition of two finite-sum functions and analyze stochastic compositional variance reduced gradient (SCVRG) methods for them.
Estimation of Markov Chain via Rank-Constrained Likelihood
This paper studies the estimation of low-rank Markov chains from empirical trajectories.
Spectral State Compression of Markov Processes
Model reduction of Markov processes is a basic problem in modeling state-transition systems.
Online Factorization and Partition of Complex Networks From Random Walks
We formulate this into a nonconvex stochastic factorization problem and propose an efficient and scalable stochastic generalized Hebbian algorithm.
Deep Primal-Dual Reinforcement Learning: Accelerating Actor-Critic using Bellman Duality
We believe that the primal-dual updates to the value and policy functions would expedite the learning process.
Primal-Dual $π$ Learning: Sample Complexity and Sublinear Run Time for Ergodic Markov Decision Problems
Consider the problem of approximating the optimal policy of a Markov decision process (MDP) by sampling state transitions.
Near-Optimal Stochastic Approximation for Online Principal Component Estimation
We prove for the first time a nearly optimal finite-sample error bound for the online PCA algorithm.
Lower Bound On the Computational Complexity of Discounted Markov Decision Problems
We study the computational complexity of the infinite-horizon discounted-reward Markov Decision Problem (MDP) with a finite state space $|\mathcal{S}|$ and a finite action space $|\mathcal{A}|$.
Stochastic Primal-Dual Methods and Sample Complexity of Reinforcement Learning
We study the online estimation of the optimal policy of a Markov decision process (MDP).
Accelerating Stochastic Composition Optimization
The ASC-PG is the first proximal gradient method for the stochastic composition problem that can deal with nonsmooth regularization penalty.
Random Multi-Constraint Projection: Stochastic Gradient Methods for Convex Optimization with Many Constraints
Consider convex optimization problems subject to a large number of constraints.
Stochastic Compositional Gradient Descent: Algorithms for Minimizing Compositions of Expected-Value Functions
For smooth convex problems, the SCGD can be accelerated to converge at a rate of $O(k^{-2/7})$ in the general case and $O(k^{-4/5})$ in the strongly convex case.
Diffusion Approximations for Online Principal Component Estimation and Global Convergence
In this paper, we propose to adopt the diffusion approximation tools to study the dynamics of Oja's iteration which is an online stochastic gradient descent method for the principal component analysis.
Adaptive Low-Nonnegative-Rank Approximation for State Aggregation of Markov Chains
The efficiency and statistical properties of our approach are illustrated on synthetic data.
Towards Coherent and Cohesive Long-form Text Generation
Generating coherent and cohesive long-form texts is a challenging task.
State Aggregation Learning from Markov Transition Data
Our proposed method is a simple two-step algorithm: The first step is spectral decomposition of empirical transition matrix, and the second step conducts a linear transformation of singular vectors to find their approximate convex hull.
Graph-Adaptive Pruning for Efficient Inference of Convolutional Neural Networks
In this method, the network is viewed as a computational graph, in which the vertices denote the computation nodes and edges represent the information flow.
Near-Optimal Time and Sample Complexities for Solving Markov Decision Processes with a Generative Model
In this paper we consider the problem of computing an $\epsilon$-optimal policy of a discounted Markov Decision Process (DMDP) provided we can only access its transition function through a generative sampling model that given any state-action pair samples from the transition function in $O(1)$ time.
Scalable Bilinear Pi Learning Using State and Action Features
In this work, we study a primal-dual formulation of the ALP, and develop a scalable, model-free algorithm called bilinear $\pi$ learning for reinforcement learning when a sampling oracle is provided.
Sample-Optimal Parametric Q-Learning Using Linearly Additive Features
Consider a Markov decision process (MDP) that admits a set of state-action features, which can linearly express the process's probabilistic transition model.
Learning to Control in Metric Space with Optimal Regret
We study online reinforcement learning for finite-horizon deterministic control systems with {\it arbitrary} state and action spaces.
Reinforcement Learning in Feature Space: Matrix Bandit, Kernels, and Regret Bound
In this case, the kernelized MatrixRL satisfies a regret bound ${O}\big(H^2\widetilde{d}\log T\sqrt{T}\big)$, where $\widetilde{d}$ is the effective dimension of the kernel space.
Learning low-dimensional state embeddings and metastable clusters from time series data
This paper studies how to find compact state embeddings from high-dimensional Markov state trajectories, where the transition kernel has a small intrinsic rank.
Feature-Based Q-Learning for Two-Player Stochastic Games
Consider a two-player zero-sum stochastic game where the transition function can be embedded in a given feature space.
RL4health: Crowdsourcing Reinforcement Learning for Knee Replacement Pathway Optimization
Joint replacement is the most common inpatient surgical treatment in the US.
Learning Markov models via low-rank optimization
We show that one can estimate the full transition model accurately using a trajectory of length that is proportional to the total number of states.
Voting-Based Multi-Agent Reinforcement Learning for Intelligent IoT
The recent success of single-agent reinforcement learning (RL) in Internet of things (IoT) systems motivates the study of multi-agent reinforcement learning (MARL), which is more challenging but more useful in large-scale IoT.
Solving Discounted Stochastic Two-Player Games with Near-Optimal Time and Sample Complexity
In this paper, we settle the sampling complexity of solving discounted two-player turn-based zero-sum stochastic games up to polylogarithmic factors.
Characterizing Deep Learning Training Workloads on Alibaba-PAI
One critical issue for efficiently operating practical AI clouds, is to characterize the computing and data transfer demands of these workloads, and more importantly, the training performance given the underlying software framework and hardware configurations.
Continuous Control with Contexts, Provably
To our knowledge, this is first provably efficient algorithm to build a decoder in the continuous control setting.
Sketching Transformed Matrices with Applications to Natural Language Processing
We show that our approach obtains small error and is efficient in both space and time.
Minimax-Optimal Off-Policy Evaluation with Linear Function Approximation
We prove that this method is information-theoretically optimal and has nearly minimal estimation error.
Cautious Reinforcement Learning via Distributional Risk in the Dual Domain
To ameliorate this issue, we propose a new definition of risk, which we call caution, as a penalty function added to the dual objective of the linear programming (LP) formulation of reinforcement learning.
Concept Annotation for Intelligent Textbooks
The outcomes of our work include a validated knowledge engineering procedure, a code-book for technical concept annotation, and a set of concept annotations for the target textbook, which could be used as gold standard in further research.
Model-Based Reinforcement Learning with Value-Targeted Regression
We propose a model based RL algorithm that is based on optimism principle: In each episode, the set of models that are `consistent' with the data collected is constructed.
Variational Policy Gradient Method for Reinforcement Learning with General Utilities
Analogously to the Policy Gradient Theorem \cite{sutton2000policy} available for RL with cumulative rewards, we derive a new Variational Policy Gradient Theorem for RL with general utilities, which establishes that the parametrized policy gradient may be obtained as the solution of a stochastic saddle point problem involving the Fenchel dual of the utility function.
Generalized Leverage Score Sampling for Neural Networks
Leverage score sampling is a powerful technique that originates from theoretical computer science, which can be used to speed up a large number of fundamental questions, e. g. linear regression, linear programming, semi-definite programming, cutting plane method, graph sparsification, maximum matching and max-flow.
On Function Approximation in Reinforcement Learning: Optimism in the Face of Large State Spaces
The classical theory of reinforcement learning (RL) has focused on tabular and linear representations of value functions.
High-Dimensional Sparse Linear Bandits
Stochastic linear bandits with high-dimensional sparse features are a practical model for a variety of domains, including personalized medicine and online advertising.
Sparse Feature Selection Makes Batch Reinforcement Learning More Sample Efficient
To evaluate a new target policy, we analyze a Lasso fitted Q-evaluation method and establish a finite-sample error bound that has no polynomial dependence on the ambient dimension.
Online Sparse Reinforcement Learning
First, we provide a lower bound showing that linear regret is generally unavoidable in this case, even if there exists a policy that collects well-conditioned data.
Provably Efficient Reinforcement Learning with Kernel and Neural Function Approximations
Reinforcement learning (RL) algorithms combined with modern function approximators such as kernel functions and deep neural networks have achieved significant empirical successes in large-scale application problems with a massive number of states.
Bootstrapping Fitted Q-Evaluation for Off-Policy Inference
Bootstrapping provides a flexible and effective approach for assessing the quality of batch reinforcement learning, yet its theoretical property is less understood.
On the Convergence and Sample Efficiency of Variance-Reduced Policy Gradient Method
By assuming the overparameterizaiton of policy and exploiting the hidden convexity of the problem, we further show that TSIVR-PG converges to global $\epsilon$-optimal policy with $\tilde{\mathcal{O}}(\epsilon^{-2})$ samples.
Learning Good State and Action Representations via Tensor Decomposition
The transition kernel of a continuous-state-action Markov decision process (MDP) admits a natural tensor structure.
MARL with General Utilities via Decentralized Shadow Reward Actor-Critic
DSAC augments the classic critic step by requiring agents to (i) estimate their local occupancy measure in order to (ii) estimate the derivative of the local utility with respect to their occupancy measure, i. e., the "shadow reward".
On the Sample Complexity and Metastability of Heavy-tailed Policy Search in Continuous Control
To close this gap, we step towards persistent exploration in continuous space through policy parameterizations defined by distributions of heavier tails defined by tail-index parameter alpha, which increases the likelihood of jumping in state space.
Boosting the Convergence of Reinforcement Learning-based Auto-pruning Using Historical Data
Reinforcement learning (RL)-based auto-pruning has been further proposed to automate the DNN pruning process to avoid expensive hand-crafted work.
Optimal policy evaluation using kernel-based temporal difference methods
Whereas existing worst-case theory predicts cubic scaling ($H^3$) in the effective horizon, our theory reveals that there is in fact a much wider range of scalings, depending on the kernel, the stationary distribution, and the variance of the Bellman residual error.
Cell2State: Learning Cell State Representations From Barcoded Single-Cell Gene-Expression Transitions
Genetic barcoding coupled with single-cell sequencing technology enables direct measurement of cell-to-cell transitions and gene-expression evolution over a long timespan.
A bird's eye view on coherence, and a worm's eye view on cohesion
Generating coherent and cohesive long-form texts is a challenging problem in natural language generation.
PROVABLY BENEFITS OF DEEP HIERARCHICAL RL
Modern complex sequential decision-making problem often both low-level policy and high-level planning.
A Duality Approach for Regret Minimization in Average-Award Ergodic Markov Decision Processes
In light of the Bellman duality, we propose a novel value-policy gradient algorithm to explore and act in infinite-horizon Average-reward Markov Decision Process (AMDP) and show that it has sublinear regret.
Optimal Estimation of Off-Policy Policy Gradient via Double Fitted Iteration
Policy gradient (PG) estimation becomes a challenge when we are not allowed to sample with the target policy but only have access to a dataset generated by some unknown behavior policy.
Off-Policy Fitted Q-Evaluation with Differentiable Function Approximators: Z-Estimation and Inference Theory
We approach this problem using the Z-estimation theory and establish the following results: The FQE estimation error is asymptotically normal with explicit variance determined jointly by the tangent space of the function class at the ground truth, the reward structure, and the distribution shift due to off-policy learning; The finite-sample FQE error bound is dominated by the same variance term, and it can also be bounded by function class-dependent divergence, which measures how the off-policy distribution shift intertwines with the function approximator.
Near-optimal Offline Reinforcement Learning with Linear Representation: Leveraging Variance Information with Pessimism
However, a precise understanding of the statistical limits with function representations, remains elusive, even when such a representation is linear.
Byzantine-Robust Online and Offline Distributed Reinforcement Learning
We consider a distributed reinforcement learning setting where multiple agents separately explore the environment and communicate their experiences through a central server.
Bandit Theory and Thompson Sampling-Guided Directed Evolution for Sequence Optimization
We propose a Thompson Sampling-guided Directed Evolution (TS-DE) framework for sequence optimization, where the sequence-to-function mapping is unknown and querying a single value is subject to costly and noisy measurements.
Sample Complexity of Nonparametric Off-Policy Evaluation on Low-Dimensional Manifolds using Deep Networks
We consider the off-policy evaluation problem of reinforcement learning using deep convolutional neural networks.
Offline Stochastic Shortest Path: Learning, Evaluation and Towards Optimality
Goal-oriented Reinforcement Learning, where the agent needs to reach the goal state while simultaneously minimizing the cost, has received significant attention in real-world applications.
Communication Efficient Distributed Learning for Kernelized Contextual Bandits
We tackle the communication efficiency challenge of learning kernelized contextual bandits in a distributed setting.
Decentralized Gossip-Based Stochastic Bilevel Optimization over Communication Networks
This paper studies the problem of distributed bilevel optimization over a network where agents can only communicate with neighbors, including examples from multi-task, multi-agent learning and federated learning.
Provably Efficient Reinforcement Learning for Online Adaptive Influence Maximization
Online influence maximization aims to maximize the influence spread of a content in a social network with unknown network model by selecting a few seed nodes.
Model-based Reinforcement Learning
reinforcement-learning
+1
Offline Reinforcement Learning with Differentiable Function Approximation is Provably Efficient
Offline reinforcement learning, which aims at optimizing sequential decision-making strategies with historical data, has been extensively applied in real-life applications.
Representation Learning for General-sum Low-rank Markov Games
To our best knowledge, this is the first sample-efficient algorithm for multi-agent general-sum Markov games that incorporates (non-linear) function approximation.
Energy System Digitization in the Era of AI: A Three-Layered Approach towards Carbon Neutrality
The transition towards carbon-neutral electricity is one of the biggest game changers in addressing climate change since it addresses the dual challenges of removing carbon emissions from the two largest sectors of emitters: electricity and transportation.
Near Sample-Optimal Reduction-based Policy Learning for Average Reward MDP
This work considers the sample complexity of obtaining an $\varepsilon$-optimal policy in an average reward Markov Decision Process (AMDP), given access to a generative model (simulator).
STEERING: Stein Information Directed Exploration for Model-Based Reinforcement Learning
Directed Exploration is a crucial challenge in reinforcement learning (RL), especially when rewards are sparse.
Model-based Reinforcement Learning
reinforcement-learning
+1
Score Approximation, Estimation and Distribution Recovery of Diffusion Models on Low-Dimensional Data
Furthermore, the generated distribution based on the estimated score function captures the data geometric structures and converges to a close vicinity of the data distribution.
ChipGPT: How far are we from natural language hardware design
As large language models (LLMs) like ChatGPT exhibited unprecedented machine intelligence, it also shows great performance in assisting hardware engineers to realize higher-efficiency logic design via natural language interaction.
Eye-tracked Virtual Reality: A Comprehensive Survey on Methods and Privacy Challenges
Latest developments in computer hardware, sensor technologies, and artificial intelligence can make virtual reality (VR) and virtual spaces an important part of human everyday life.
Reinforcement Learning with Human Feedback: Learning Dynamic Choices via Pessimism
In this paper, we study offline Reinforcement Learning with Human Feedback (RLHF) where we aim to learn the human's underlying reward and the MDP's optimal policy from a set of trajectories induced by human choices.
Adversarial Attacks on Online Learning to Rank with Stochastic Click Models
We propose the first study of adversarial attacks on online learning to rank.
Efficient Reinforcement Learning with Impaired Observability: Learning to Act with Delayed and Missing State Observations
We present algorithms and establish near-optimal regret upper and lower bounds, of the form $\tilde{\mathcal{O}}(\sqrt{{\rm poly}(H) SAK})$, for RL in the delayed and missing observation settings.
Provably Efficient Representation Learning with Tractable Planning in Low-Rank POMDP
In this paper, we study representation learning in partially observable Markov Decision Processes (POMDPs), where the agent learns a decoder function that maps a series of high-dimensional raw observations to a compact representation and uses it for more efficient exploration and planning.
Effective Minkowski Dimension of Deep Nonparametric Regression: Function Approximation and Statistical Theories
Existing theories on deep nonparametric regression have shown that when the input data lie on a low-dimensional manifold, deep neural networks can adapt to the intrinsic data structures.
Nonparametric Classification on Low Dimensional Manifolds using Overparameterized Convolutional Residual Networks
Convolutional residual neural networks (ConvResNets), though overparameterized, can achieve remarkable prediction performance in practice, which cannot be well explained by conventional wisdom.
Scaling In-Context Demonstrations with Structured Attention
However, their capabilities of in-context learning are limited by the model architecture: 1) the use of demonstrations is constrained by a maximum sentence length due to positional embeddings; 2) the quadratic complexity of attention hinders users from using more demonstrations efficiently; 3) LLMs are shown to be sensitive to the order of the demonstrations.
Sample-Efficient Learning of POMDPs with Multiple Observations In Hindsight
This paper studies the sample-efficiency of learning in Partially Observable Markov Decision Processes (POMDPs), a challenging problem in reinforcement learning that is known to be exponentially hard in the worst-case.
Provable Benefits of Policy Learning from Human Preferences in Contextual Bandit Problems
A popular approach is to utilize human feedback to learn a reward function for training.
Actions Speak What You Want: Provably Sample-Efficient Reinforcement Learning of the Quantal Stackelberg Equilibrium from Strategic Feedbacks
The goal of the leader is to find her optimal policy, which yields the optimal expected total return, by interacting with the follower and learning from data.
PARL: A Unified Framework for Policy Alignment in Reinforcement Learning
We present a novel unified bilevel optimization-based framework, \textsf{PARL}, formulated to address the recently highlighted critical issue of policy alignment in reinforcement learning using utility or preference-based feedback.
Deep Reinforcement Learning for Efficient and Fair Allocation of Health Care Resources
Scarcity of health care resources could result in the unavoidable consequence of rationing.
Sample Complexity of Neural Policy Mirror Descent for Policy Optimization on Low-Dimensional Manifolds
As a result, by properly choosing the network size and hyperparameters, NPMD can find an $\epsilon$-optimal policy with $\widetilde{O}(\epsilon^{-\frac{d}{\alpha}-2})$ samples in expectation, where $\alpha\in(0, 1]$ indicates the smoothness of environment.
A 5' UTR Language Model for Decoding Untranslated Regions of mRNA and Function Predictions
The model outperformed the best-known benchmark by up to 42% for predicting the Mean Ribosome Loading, and by up to 60% for predicting the Translation Efficiency and the mRNA Expression Level.
Federated Multi-Level Optimization over Decentralized Networks
Multi-level optimization has gained increasing attention in recent years, as it provides a powerful framework for solving complex optimization problems that arise in many fields, such as meta-learning, multi-player games, reinforcement learning, and nested composition optimization.
Sample Complexity of Preference-Based Nonparametric Off-Policy Evaluation with Deep Networks
In fact, human preference data are now used with classic reinforcement learning algorithms such as actor-critic methods, which involve evaluating an intermediate policy over a reward learned from human preference data with distribution shift, known as off-policy evaluation (OPE).
Is Inverse Reinforcement Learning Harder than Standard Reinforcement Learning? A Theoretical Perspective
Inverse Reinforcement Learning (IRL) -- the problem of learning reward functions from demonstrations of an \emph{expert policy} -- plays a critical role in developing intelligent systems.
Scalable Normalizing Flows Enable Boltzmann Generators for Macromolecules
We demonstrate that standard architectures and training strategies, such as maximum likelihood alone, fail while our novel architecture and multi-stage training strategy are able to model the conformational distributions of protein G and HP35.
Tree Search-Based Evolutionary Bandits for Protein Sequence Optimization
To enhance the efficiency of such a process, we propose a tree search-based bandit learning method, which expands a tree starting from the initial sequence with the guidance of a bandit machine learning model.
Embedding Large Language Models into Extended Reality: Opportunities and Challenges for Inclusion, Engagement, and Privacy
Lastly, we speculate that combining the information provided to LLM-powered environments by the users and the biometric data obtained through the sensors might lead to novel privacy invasions.
Assessing the Brittleness of Safety Alignment via Pruning and Low-Rank Modifications
We develop methods to identify critical regions that are vital for safety guardrails, and that are disentangled from utility-relevant regions at both the neuron and rank levels.
MaxMin-RLHF: Towards Equitable Alignment of Large Language Models with Diverse Human Preferences
Reinforcement Learning from Human Feedback (RLHF) aligns language models to human preferences by employing a singular reward model derived from preference data.
Double Duality: Variational Primal-Dual Policy Optimization for Constrained Reinforcement Learning
Designing algorithms for a constrained convex MDP faces several challenges, including (1) handling the large state space, (2) managing the exploration/exploitation tradeoff, and (3) solving the constrained optimization where the objective and the constraint are both nonlinear functions of the visitation measure.
Theoretical Insights for Diffusion Guidance: A Case Study for Gaussian Mixture Models
Diffusion models benefit from instillation of task-specific information into the score function to steer the sample generation towards desired properties.
Regularized DeepIV with Model Selection
In this paper, we study nonparametric estimation of instrumental variable (IV) regressions.
Data is all you need: Finetuning LLMs for Chip Design via an Automated design-data augmentation framework
Our 13B model (ChipGPT-FT) has a pass rate improvement compared with GPT-3. 5 in Verilog generation and outperforms in EDA script (i. e., SiliconCompiler) generation with only 200 EDA script data.
Offline Multitask Representation Learning for Reinforcement Learning
We study offline multitask representation learning in reinforcement learning (RL), where a learner is provided with an offline dataset from different tasks that share a common representation and is asked to learn the shared representation.
Unveil Conditional Diffusion Models with Classifier-free Guidance: A Sharp Statistical Theory
Conditional diffusion models serve as the foundation of modern image synthesis and find extensive application in fields like computational biology and reinforcement learning.
Embodied LLM Agents Learn to Cooperate in Organized Teams
Large Language Models (LLMs) have emerged as integral tools for reasoning, planning, and decision-making, drawing upon their extensive world knowledge and proficiency in language-related tasks.
Diffusion Model for Data-Driven Black-Box Optimization
In this paper, we focus on diffusion models, a powerful generative AI technology, and investigate their potential for black-box optimization over complex structured variables.
An Overview of Diffusion Models: Applications, Guided Generation, Statistical Rates and Optimization
In this paper, we review emerging applications of diffusion models, understanding their sample generation under various controls.
