Finite Sample System Identification: Improved Rates and the Role of Regularization

This paper studies low-order linear system identification via regularized regression. The nuclear norm of the system’s Hankel matrix is added as a regularizer to the least-squares cost function due to the following advantages. The regularized problem is (1) often easier to tune, (2) has lower sample complexity, and (3) returns a Hankel matrix with a clear singular value gap, which robustly recovers a low-order linear system from noisy output observations. Recently, the performance of the unregularized least-squares formulations have been studied statistically in terms of finite sample complexity and recovery errors; however, no results are known for the regularized approach. In this work, we provide a novel statistical analysis of the regularized algorithm. Our analysis leads to new bounds on estimating the impulse response and the Hankel matrix associated with the system while using smaller number of observations than the least-squares estimator.