no code implementations • 22 Nov 2023 • Jarek Duda
Working with multiple variables they usually contain difficult to control complex dependencies.
no code implementations • 24 Apr 2023 • Jarek Duda
Electroencephalography (EEG) signals are resultants of extremely complex brain activity.
no code implementations • 6 Apr 2023 • Jarek Duda
To avoid such bias, we will focus on recently proposed agnostic philosophy of moving estimator: in time $t$ finding parameters optimizing e. g. $F_t=\sum_{\tau<t} (1-\eta)^{t-\tau} \ln(\rho_\theta (x_\tau))$ moving log-likelihood, evolving in time.
no code implementations • 13 Sep 2022 • Jarek Duda
We are mostly interested in the best drug in their batch to be tested - proper optimization of their selection for extreme statistics requires knowledge of the entire probability distributions, which for distributions of drug properties among cell lines often turn out binomial, e. g. depending on corresponding gene.
no code implementations • 21 Jul 2022 • Jarek Duda, Sabina Podlewska
While there is a general focus on predictions of values, mathematically more appropriate is prediction of probability distributions: with additional possibilities like prediction of uncertainty, higher moments and quantiles.
no code implementations • 13 Jun 2022 • Jarek Duda
While there is a general focus on prediction of values, real data often only allows to predict conditional probability distributions, with capabilities bounded by conditional entropy $H(Y|X)$.
no code implementations • 18 Apr 2022 • Jarek Duda
SVD (singular value decomposition) is one of the basic tools of machine learning, allowing to optimize basis for a given matrix.
no code implementations • 6 Apr 2020 • Jarek Duda
The presented simple inexpensive general methodology can be also used for different types of data like DCT coefficients in lossy image compression.
no code implementations • 4 Mar 2020 • Jarek Duda
While standard estimation assumes that all datapoints are from probability distribution of the same fixed parameters $\theta$, we will focus on maximum likelihood (ML) adaptive estimation for nonstationary time series: separately estimating parameters $\theta_T$ for each time $T$ based on the earlier values $(x_t)_{t<T}$ using (exponential) moving ML estimator $\theta_T=\arg\max_\theta l_T$ for $l_T=\sum_{t<T} \eta^{T-t} \ln(\rho_\theta (x_t))$ and some $\eta\in(0, 1]$.
1 code implementation • 16 Jul 2019 • Jarek Duda
It is done by estimating linear trend of gradients $\vec{g}=\nabla F(\vec{\theta})$ in $\hat{v}$ direction: such that $g(\vec{\theta}_\bot+\theta\hat{v})\approx \lambda (\theta -p)$ for $\theta = \vec{\theta}\cdot \hat{v}$, $g= \vec{g}\cdot \hat{v}$, $\vec{\theta}_\bot=\vec{\theta}-\theta\hat{v}$.
1 code implementation • 31 Jan 2019 • Jarek Duda
Deep neural networks are usually trained with stochastic gradient descent (SGD), which minimizes objective function using very rough approximations of gradient, only averaging to the real gradient.
no code implementations • 19 Dec 2018 • Jarek Duda, Adam Szulc
In situations like tax declarations or analyzes of household budgets we would like to automatically evaluate credibility of exogenous variable (declared income) based on some available (endogenous) variables - we want to build a model and train it on provided data sample to predict (conditional) probability distribution of exogenous variable based on values of endogenous variables.
no code implementations • 12 Nov 2018 • Jarek Duda
The original Variational AutoEncoder (VAE) uses randomness in encoder - causing problematic distortion, and overlaps in latent space for distinct inputs.
no code implementations • 11 Jul 2018 • Jarek Duda
While we are usually focused on forecasting future values of time series, it is often valuable to additionally predict their entire probability distributions, e. g. to evaluate risk, Monte Carlo simulations.
no code implementations • 17 Apr 2018 • Jarek Duda
Machine learning often needs to model density from a multidimensional data sample, including correlations between coordinates.
no code implementations • 3 Jan 2018 • Jarek Duda
One of basic difficulties of machine learning is handling unknown rotations of objects, for example in image recognition.
no code implementations • 7 Feb 2017 • Jarek Duda
There will be discussed inexpensive density estimation, for example literally fitting a polynomial (or Fourier series) to the sample, which coefficients are calculated by just averaging monomials (or sine/cosine) over the sample.
no code implementations • 20 May 2015 • Jarek Duda
This approach requires a priori knowledge of the final damage level of every packet - insufficient redundancy leads to packet loss, overprotection means suboptimal channel rate.
Information Theory Information Theory
8 code implementations • 11 Nov 2013 • Jarek Duda
The latter uses nearly exact probabilities - easily approaching theoretical compression rate limit (Shannon entropy), but at cost of much larger computational cost.
Information Theory Information Theory
1 code implementation • 7 Nov 2012 • Jarek Duda
We will discuss a general problem of using codes with chosen statistical constrains, for example reproducing given grayscale picture using halftone technique.
Information Theory Cryptography and Security Multimedia Information Theory
1 code implementation • 2 Feb 2009 • Jarek Duda
It has some similarities to Range Coding but instead of encoding symbol in choosing a range, we spread these ranges uniformly over the whole interval.
Information Theory Cryptography and Security General Mathematics Information Theory