Hierarchical correlation reconstruction - between statistics and ML


dr Jarosław Duda; Institute of Computer Science and Computer Mathematics, Jagiellonian University, Cracow, Poland


27/03/2019 - 13:00
While machine learning techniques are very powerful, they have some weaknesses, like iterative optimization with many local minimums, large freedom of parameters, lack of their interpretability and accuracy control. From the other side we have classical statistics based on moments not having these issues, but providing only a rough description. I will talk about approach which combines their advantages: with MSE-optimal moment-like coefficients, but designed such that we can directly translate them into probability density. For multivariate case such basis of mixed moments asymptotically allows to accurately reconstruct any joint distribution, each coefficient can be independently and cheaply estimated, has a clear interpretation, and we have some control of its accuracy. I will also present its two applications: systematic enhancement of ARMA/ARCH-like modeling for any mixed moments and non-stationary time series, and for credibility evaluation of income data: modeling continuous conditional probability distribution from a large number of variables of various types. Slides: https://www.dropbox.com/s/7u6f2zpreph6j8o/rapid.pdf