Discovering a correlation from one variable to another variable is of
fundamental scientific and practical interest. While existing correlation
measures are suitable for discovering average correlation, they fail to
discover hidden or potential correlations...
To bridge this gap, (i) we postulate
a set of natural axioms that we expect a measure of potential correlation to
satisfy; (ii) we show that the rate of information bottleneck, i.e., the
hypercontractivity coefficient, satisfies all the proposed axioms; (iii) we
provide a novel estimator to estimate the hypercontractivity coefficient from
samples; and (iv) we provide numerical experiments demonstrating that this
proposed estimator discovers potential correlations among various indicators of
WHO datasets, is robust in discovering gene interactions from gene expression
time series data, and is statistically more powerful than the estimators for
other correlation measures in binary hypothesis testing of canonical examples
of potential correlations.