Search Results for author:
Found 100 papers, 29 papers with code
Doubly Robust Off-policy Value Evaluation for Reinforcement Learning
We study the problem of off-policy value evaluation in reinforcement learning (RL), where one aims to estimate the value of a new policy based on data collected by a different policy.
Scale Match for Tiny Person Detection
In this paper, we introduce a new benchmark, referred to as TinyPerson, opening up a promising directionfor tiny object detection in a long distance and with mas-sive backgrounds.
The 1st Tiny Object Detection Challenge:Methods and Results
The 1st Tiny Object Detection (TOD) Challenge aims to encourage research in developing novel and accurate methods for tiny object detection in images which have wide views, with a current focus on tiny person detection.
Anti-UAV: A Large Multi-Modal Benchmark for UAV Tracking
The releasing of such a large-scale dataset could be a useful initial step in research of tracking UAVs.
Adversarially Trained Actor Critic for Offline Reinforcement Learning
We propose Adversarially Trained Actor Critic (ATAC), a new model-free algorithm for offline reinforcement learning (RL) under insufficient data coverage, based on the concept of relative pessimism.
Quantifying Spatial Homogeneity of Urban Road Networks via Graph Neural Networks
We apply the method to 11, 790 urban road networks across 30 cities worldwide to measure the spatial homogeneity of road networks within each city and across different cities.
Full-Body Articulated Human-Object Interaction
By learning the geometrical relationships in HOI, we devise the very first model that leverage human pose estimation to tackle the estimation of articulated object poses and shapes during whole-body interactions.
Empirical Study of Off-Policy Policy Evaluation for Reinforcement Learning
We offer an experimental benchmark and empirical study for off-policy policy evaluation (OPE) in reinforcement learning, which is a key problem in many safety critical applications.
RouterBench: A Benchmark for Multi-LLM Routing System
To bridge this gap, we present RouterBench, a novel evaluation framework designed to systematically assess the efficacy of LLM routing systems, along with a comprehensive dataset comprising over 405k inference outcomes from representative LLMs to support the development of routing strategies.
CURE: Code-Aware Neural Machine Translation for Automatic Program Repair
Finally, CURE uses a subword tokenization technique to generate a smaller search space that contains more correct fixes.
Provably efficient RL with Rich Observations via Latent State Decoding
We study the exploration problem in episodic MDPs with rich observations generated from a small number of latent states.
KNOD: Domain Knowledge Distilled Tree Decoder for Automated Program Repair
KNOD has two major novelties, including (1) a novel three-stage tree decoder, which directly generates Abstract Syntax Trees of patched code according to the inherent tree structure, and (2) a novel domain-rule distillation, which leverages syntactic and semantic rules and teacher-student distributions to explicitly inject the domain knowledge into the decoding procedure during both the training and inference phases.
How Effective Are Neural Networks for Fixing Security Vulnerabilities
The results call for innovations to enhance automated Java vulnerability repair such as creating larger vulnerability repair training data, tuning LLMs with such data, and applying code simplification transformation to facilitate vulnerability repair.
When Counterpoint Meets Chinese Folk Melodies
An interaction reward model is trained on the duets formed from outer parts of Bach chorales to model counterpoint interaction, while a style reward model is trained on monophonic melodies of Chinese folk songs to model melodic patterns.
Language Generation via Combinatorial Constraint Satisfaction: A Tree Search Enhanced Monte-Carlo Approach
Generating natural language under complex constraints is a principled formulation towards controllable text generation.
Batch Value-function Approximation with Only Realizability
We make progress in a long-standing problem of batch reinforcement learning (RL): learning $Q^\star$ from an exploratory and polynomial-sized dataset, using a realizable and otherwise arbitrary function class.
Towards Hyperparameter-free Policy Selection for Offline Reinforcement Learning
How to select between policies and value functions produced by different training algorithms in offline reinforcement learning (RL) -- which is crucial for hyperpa-rameter tuning -- is an important open question.
Hand Segmentation for Hand-Object Interaction from Depth map
Thus, we propose hand segmentation method for hand-object interaction using only a depth map.
Future-Dependent Value-Based Off-Policy Evaluation in POMDPs
Finally, we extend our methods to learning of dynamics and establish the connection between our approach and the well-known spectral learning methods in POMDPs.
A Minimax Learning Approach to Off-Policy Evaluation in Confounded Partially Observable Markov Decision Processes
In this work, we first propose novel identification methods for OPE in POMDPs with latent confounders, by introducing bridge functions that link the target policy's value and the observed data distribution.
From Importance Sampling to Doubly Robust Policy Gradient
We show that on-policy policy gradient (PG) and its variance reduction variants can be derived by taking finite difference of function evaluations supplied by estimators from the importance sampling (IS) family for off-policy evaluation (OPE).
Learning Markov Random Fields for Combinatorial Structures via Sampling through Lovász Local Lemma
Contrastive divergence is then applied to separate these samples from those in the training set.
Ultrasonic Image's Annotation Removal: A Self-supervised Noise2Noise Approach
Accurately annotated ultrasonic images are vital components of a high-quality medical report.
Is attention required for ICL? Exploring the Relationship Between Model Architecture and In-Context Learning Ability
We also measure each architecture's predisposition towards in-context learning when presented with the option to memorize rather than leverage in-context examples.
Tiered Reinforcement Learning: Pessimism in the Face of Uncertainty and Constant Regret
We propose a new learning framework that captures the tiered structure of many real-world user-interaction applications, where the users can be divided into two groups based on their different tolerance on exploration risks and should be treated separately.
Explaining RL Decisions with Trajectories
To do so, we encode trajectories in offline training data individually as well as collectively (encoding a set of trajectories).
Hierarchical Imitation and Reinforcement Learning
We study how to effectively leverage expert feedback to learn sequential decision-making policies.
On Oracle-Efficient PAC RL with Rich Observations
We study the computational tractability of PAC reinforcement learning with rich observations.
Image Classification Based on Quantum KNN Algorithm
However, the usually used classification method --- the K Nearest-Neighbor algorithm has high complexity, because its two main processes: similarity computing and searching are time-consuming.
Optimizing Recurrent Neural Networks Architectures under Time Constraints
A greedy algorithm with bounds is suggested to solve the transformed problem.
Neural Network Architecture Optimization through Submodularity and Supermodularity
Deep learning models' architectures, including depth and width, are key factors influencing models' performance, such as test accuracy and computation time.
Markov Decision Processes with Continuous Side Information
Because our lower bound has an exponential dependence on the dimension, we consider a tractable linear setting where the context is used to create linear combinations of a finite set of MDPs.
Repeated Inverse Reinforcement Learning
We introduce a novel repeated Inverse Reinforcement Learning problem: the agent has to act on behalf of a human in a sequence of tasks and wishes to minimize the number of tasks that it surprises the human by acting suboptimally with respect to how the human would have acted.
Contextual Decision Processes with Low Bellman Rank are PAC-Learnable
Our first contribution is a complexity measure, the Bellman rank, that we show enables tractable learning of near-optimal behavior in these processes and is naturally small for many well-studied reinforcement learning settings.
Word Embedding based Correlation Model for Question/Answer Matching
With the development of community based question answering (Q&A) services, a large scale of Q&A archives have been accumulated and are an important information and knowledge resource on the web.
Why Do Neural Response Generation Models Prefer Universal Replies?
Recent advances in sequence-to-sequence learning reveal a purely data-driven approach to the response generation task.
Model-based RL in Contextual Decision Processes: PAC bounds and Exponential Improvements over Model-free Approaches
We study the sample complexity of model-based reinforcement learning (henceforth RL) in general contextual decision processes that require strategic exploration to find a near-optimal policy.
LSDSCC: a Large Scale Domain-Specific Conversational Corpus for Response Generation with Diversity Oriented Evaluation Metrics
The experimental results have shown that our proposed corpus can be taken as a new benchmark dataset for the NRG task, and the presented metrics are promising to guide the optimization of NRG models by quantifying the diversity of the generated responses reasonably.
Completing State Representations using Spectral Learning
A central problem in dynamical system modeling is state discovery—that is, finding a compact summary of the past that captures the information needed to predict the future.
Unifying Spatial and Attribute Selection for Distracter-Resilient Tracking
This paper presents a novel method to jointly determine the best spatial location and the optimal metric.
Information-Theoretic Considerations in Batch Reinforcement Learning
Value-function approximation methods that operate in batch mode have foundational importance to reinforcement learning (RL).
Provably Efficient Q-Learning with Low Switching Cost
We take initial steps in studying PAC-MDP algorithms with limited adaptivity, that is, algorithms that change its exploration policy as infrequently as possible during regret minimization.
On Value Functions and the Agent-Environment Boundary
When function approximation is deployed in reinforcement learning (RL), the same problem may be formulated in different ways, often by treating a pre-processing step as a part of the environment or as part of the agent.
Sample Complexity of Reinforcement Learning using Linearly Combined Model Ensembles
As an extension, we also consider the more challenging problem of model selection, where the state features are unknown and can be chosen from a large candidate set.
Minimax Weight and Q-Function Learning for Off-Policy Evaluation
We provide theoretical investigations into off-policy evaluation in reinforcement learning using function approximators for (marginalized) importance weights and value functions.
Quantum Communication between Multiplexed Atomic Quantum Memories
The use of multiplexed atomic quantum memories (MAQM) can significantly enhance the efficiency to establish entanglement in a quantum network.
Quantum Physics
Minimax Value Interval for Off-Policy Evaluation and Policy Optimization
By slightly altering the derivation of previous methods (one from each style; Uehara et al., 2020), we unify them into a single value interval that comes with a special type of double robustness: when either the value-function or the importance-weight class is well specified, the interval is valid and its length quantifies the misspecification of the other class.
RL-Duet: Online Music Accompaniment Generation Using Deep Reinforcement Learning
We cast this as a reinforcement learning problem, where the generation agent learns a policy to generate a musical note (action) based on previously generated context (state).
Q* Approximation Schemes for Batch Reinforcement Learning: A Theoretical Comparison
We prove performance guarantees of two algorithms for approximating $Q^\star$ in batch reinforcement learning.
A Question Type Driven and Copy Loss Enhanced Frameworkfor Answer-Agnostic Neural Question Generation
The answer-agnostic question generation is a significant and challenging task, which aims to automatically generate questions for a given sentence but without an answer.
Improved Worst-Case Regret Bounds for Randomized Least-Squares Value Iteration
This paper studies regret minimization with randomized value functions in reinforcement learning.
A Variant of the Wang-Foster-Kakade Lower Bound for the Discounted Setting
Recently, Wang et al. (2020) showed a highly intriguing hardness result for batch reinforcement learning (RL) with linearly realizable value function and good feature coverage in the finite-horizon case.
Analysis of Random Access in NB-IoT Networks with Three Coverage Enhancement Groups: A Stochastic Geometry Approach
NarrowBand-Internet of Things (NB-IoT) is a new 3GPP radio access technology designed to provide better coverage for Low Power Wide Area (LPWA) networks.
Experimental demonstration of memory-enhanced scaling for entanglement connection of quantum repeater segments
The experimental realization of entanglement connection of two quantum repeater segments with an efficient memory-enhanced scaling demonstrates a key advantage of the quantum repeater protocol, which makes a cornerstone towards future large-scale quantum networks.
Quantum Physics
On Query-efficient Planning in MDPs under Linear Realizability of the Optimal State-value Function
We consider local planning in fixed-horizon MDPs with a generative model under the assumption that the optimal value function lies close to the span of a feature map.
Finite Sample Analysis of Minimax Offline Reinforcement Learning: Completeness, Fast Rates and First-Order Efficiency
We offer a theoretical characterization of off-policy evaluation (OPE) in reinforcement learning using function approximation for marginal importance weights and $q$-functions when these are estimated using recent minimax methods.
SM+: Refined Scale Match for Tiny Person Detection
Detecting tiny objects ( e. g., less than 20 x 20 pixels) in large-scale images is an important yet open problem.
Model-free Representation Learning and Exploration in Low-rank MDPs
In this work, we present the first model-free representation learning algorithms for low rank MDPs.
Minimax Model Learning
We present a novel off-policy loss function for learning a transition model in model-based reinforcement learning.
On the Convergence Rate of Off-Policy Policy Optimization Methods with Density-Ratio Correction
In this paper, we study the convergence properties of off-policy policy improvement algorithms with state-action density ratio correction under function approximation setting, where the objective function is formulated as a max-max-min optimization problem.
Policy Finetuning: Bridging Sample-Efficient Offline and Online Reinforcement Learning
This offline result is the first that matches the sample complexity lower bound in this setting, and resolves a recent open question in offline RL.
Bellman-consistent Pessimism for Offline Reinforcement Learning
The use of pessimism, when reasoning about datasets lacking exhaustive exploration has recently gained prominence in offline reinforcement learning.
A Spectral Approach to Off-Policy Evaluation for POMDPs
We consider off-policy evaluation (OPE) in Partially Observable Markov Decision Processes, where the evaluation policy depends only on observable variables but the behavior policy depends on latent states (Tennenholtz et al. (2020a)).
A Fast Randomized Algorithm for Massive Text Normalization
In addition, FLAN does not require any annotated data or supervised learning.
Offline Reinforcement Learning with Realizability and Single-policy Concentrability
Sample-efficiency guarantees for offline reinforcement learning (RL) often rely on strong assumptions on both the function classes (e. g., Bellman-completeness) and the data coverage (e. g., all-policy concentrability).
Towards Deployment-Efficient Reinforcement Learning: Lower Bound and Optimality
Deployment efficiency is an important criterion for many real-world applications of reinforcement learning (RL).
VRConvMF: Visual Recurrent Convolutional Matrix Factorization for Movie Recommendation
Fortunately, context-aware recommender systems can alleviate the sparsity problem by making use of some auxiliary information, such as the information of both the users and items.
Offline Reinforcement Learning Under Value and Density-Ratio Realizability: The Power of Gaps
We consider a challenging theoretical problem in offline reinforcement learning (RL): obtaining sample-efficiency guarantees with a dataset lacking sufficient coverage, under only realizability-type assumptions for the function approximators.
Adaptable Semantic Compression and Resource Allocation for Task-Oriented Communications
To solve the problem, both compression ratio and resource allocation are optimized for the task-oriented communication system to maximize the success probability of tasks.
Interaction-Grounded Learning with Action-inclusive Feedback
Consider the problem setting of Interaction-Grounded Learning (IGL), in which a learner's goal is to optimally interact with the environment with no explicit reward to ground its policies.
On the Statistical Efficiency of Reward-Free Exploration in Non-Linear RL
We study reward-free reinforcement learning (RL) under general non-linear function approximation, and establish sample efficiency and hardness results under various standard structural assumptions.
A Few Expert Queries Suffices for Sample-Efficient RL with Resets and Linear Value Approximation
Towards establishing the minimal amount of expert queries needed, we show that, in the same setting, any learner whose exploration budget is polynomially-bounded (in terms of $d, H,$ and $|\mathcal{A}|$) will require at least $\tilde\Omega(\sqrt{d})$ oracle calls to recover a policy competing with the expert's value function.
On the Value of Behavioral Representations for Dense Retrieval
We consider text retrieval within dense representational space in real-world settings such as e-commerce search where (a) document popularity and (b) diversity of queries associated with a document have a skewed distribution.
The Role of Coverage in Online Reinforcement Learning
Coverage conditions -- which assert that the data logging distribution adequately covers the state space -- play a fundamental role in determining the sample complexity of offline reinforcement learning.
Beyond the Return: Off-policy Function Estimation under User-specified Error-measuring Distributions
Off-policy evaluation often refers to two related tasks: estimating the expected return of a policy and estimating its value function (or other functions of interest, such as density ratios).
ARMOR: A Model-based Framework for Improving Arbitrary Baseline Policies with Offline Data
We propose a new model-based offline RL framework, called Adversarial Models for Offline Reinforcement Learning (ARMOR), which can robustly learn policies to improve upon an arbitrary baseline policy regardless of data coverage.
Toward Multi-Service Edge-Intelligence Paradigm: Temporal-Adaptive Prediction for Time-Critical Control over Wireless
The paper articulates some of the key system design aspects of multi-service edge-intelligence.
Information Bottleneck-Inspired Type Based Multiple Access for Remote Estimation in IoT Systems
Type-based multiple access (TBMA) is a semantics-aware multiple access protocol for remote inference.
Offline Learning in Markov Games with General Function Approximation
We study offline multi-agent reinforcement learning (RL) in Markov games, where the goal is to learn an approximate equilibrium -- such as Nash equilibrium and (Coarse) Correlated Equilibrium -- from an offline dataset pre-collected from the game.
Multi-agent Reinforcement Learning
Reinforcement Learning (RL)
Reinforcement Learning in Low-Rank MDPs with Density Features
As a central technical challenge, the additive error of occupancy estimation is incompatible with the multiplicative definition of data coverage.
Adversarial Model for Offline Reinforcement Learning
We propose a novel model-based offline Reinforcement Learning (RL) framework, called Adversarial Model for Offline Reinforcement Learning (ARMOR), which can robustly learn policies to improve upon an arbitrary reference policy regardless of data coverage.
LM-Switch: Lightweight Language Model Conditioning in Word Embedding Space
As pre-training and fine-tuning are costly and might negatively impact model performance, it is desired to efficiently adapt an existing model to different conditions such as styles, sentiments or narratives, when facing different audiences or scenarios.
Symbolic Regression via Control Variable Genetic Programming
CVGP starts by fitting simple expressions involving a small set of independent variables using genetic programming, under controlled experiments where other variables are held as constants.
The Optimal Approximation Factors in Misspecified Off-Policy Value Function Estimation
Theoretical guarantees in reinforcement learning (RL) are known to suffer multiplicative blow-up factors with respect to the misspecification error of function approximation.
Marginalized Importance Sampling for Off-Environment Policy Evaluation
This paper proposes a new approach to evaluate the real-world performance of agent policies prior to deploying them in the real world.
Mitigating the Alignment Tax of RLHF
Building on the analysis and the observation that averaging different layers of the transformer leads to significantly different reward-tax trade-offs, we propose Adaptive Model Averaging (AMA) to adaptively find various combination ratios of model layers.
Racing Control Variable Genetic Programming for Symbolic Regression
A selection scheme similar to that used in selecting good symbolic equations in the genetic programming process is implemented to ensure that promising experiment schedules eventually win over the average ones.
Solving Satisfiability Modulo Counting for Symbolic and Statistical AI Integration With Provable Guarantees
Solving SMC is challenging because of its highly intractable nature($\text{NP}^{\text{PP}}$-complete), incorporating statistical inference and symbolic reasoning.
Single-view 3D Scene Reconstruction with High-fidelity Shape and Texture
Reconstructing detailed 3D scenes from single-view images remains a challenging task due to limitations in existing approaches, which primarily focus on geometric shape recovery, overlooking object appearances and fine shape details.
Nova$^+$: Generative Language Models for Binaries
We build Nova$^+$ to further boost Nova using two new pre-training tasks, i. e., optimization generation and optimization level prediction, which are designed to learn binary optimization and align equivalent binaries.
Iterative Preference Learning from Human Feedback: Bridging Theory and Practice for RLHF under KL-Constraint
This includes an iterative version of the Direct Preference Optimization (DPO) algorithm for online settings, and a multi-step rejection sampling strategy for offline scenarios.
Vertical Symbolic Regression
The first few steps in vertical discovery are significantly cheaper than the horizontal path, as their search is in reduced hypothesis spaces involving a small set of variables.
Harnessing Density Ratios for Online Reinforcement Learning
The theories of offline and online reinforcement learning, despite having evolved in parallel, have begun to show signs of the possibility for a unification, with algorithms and analysis techniques for one setting often having natural counterparts in the other.
Vertical Symbolic Regression via Deep Policy Gradient
We propose Vertical Symbolic Regression using Deep Policy Gradient (VSR-DPG) and demonstrate that VSR-DPG can recover ground-truth equations involving multiple input variables, significantly beyond both deep reinforcement learning-based approaches and previous VSR variants.
A Theoretical Analysis of Nash Learning from Human Feedback under General KL-Regularized Preference
In this work, we provide theoretical insights for a recently proposed learning paradigm, Nash learning from human feedback (NLHF), which considered a general preference model and formulated the alignment process as a game between two competitive LLMs.
On the Curses of Future and History in Future-dependent Value Functions for Off-policy Evaluation
We study off-policy evaluation (OPE) in partially observable environments with complex observations, with the goal of developing estimators whose guarantee avoids exponential dependence on the horizon.
Scaling Up Dynamic Human-Scene Interaction Modeling
Confronting the challenges of data scarcity and advanced motion synthesis in human-scene interaction modeling, we introduce the TRUMANS dataset alongside a novel HSI motion synthesis method.
STMGF: An Effective Spatial-Temporal Multi-Granularity Framework for Traffic Forecasting
Accurate Traffic Prediction is a challenging task in intelligent transportation due to the spatial-temporal aspects of road networks.
Towards Effective Next POI Prediction: Spatial and Semantic Augmentation with Remote Sensing Data
The next point-of-interest (POI) prediction is a significant task in location-based services, yet its complexity arises from the consolidation of spatial and semantic intent.
A Note on Loss Functions and Error Compounding in Model-based Reinforcement Learning
This note clarifies some confusions (and perhaps throws out more) around model-based reinforcement learning and their theoretical understanding in the context of deep RL.
