NUMML 2009 Numerical Mathematics in Machine Learning

ICML 2009 Workshop

Most machine learning (ML) algorithms rely fundamentally on concepts of numerical mathematics. Standard reductions to black-box computational primitives do not usually meet real-world demands and have to be modified at all levels. The increasing complexity of ML problems requires layered approaches, where algorithms are components rather than stand-alone tools fitted individually with much human effort. In this modern context, predictable run-time and numerical stability behavior of algorithms become fundamental. Unfortunately, these aspects are widely ignored today by ML researchers, which limits the applicability of ML algorithms to complex problems.

Our workshop aims to address these shortcomings, by trying to distill a compromise between inadequate black-box reductions and highly involved complete numerical analyses. We will invite speakers with interest in *both* numerical methodology *and* real problems in applications close to machine learning. While numerical software packages of ML interest will be pointed out, our focus will rather be on how to best bridge the gaps between ML requirements and these computational libraries. A subordinate goal will be to address the role of parallel numerical computation in ML. Examples of machine learning founded on numerical methods include low level computer vision and image processing, non-Gaussian approximate inference, Gaussian filtering / smoothing, state space models, approximations to kernel methods, and many more.

We will call the community's attention to the increasingly critical issue of numerical considerations in algorithm design and implementation. A set of essential rules for how to use and modify numerical software in ML is required, for which we aim to lay the groundwork in this workshop. These efforts should lead to an awareness of the problems, as well as increased focus on efficient and stable ML implementations. We will encourage speakers to point out useful software packages, together with their caveats, asking them to focus on examples of ML interest. Raising awareness about the increasing importance of stability and predictable run-time behaviour of numerical machine learning algorithms and primitives. Establishing a code of conduct for how to best select and modify existing numerical mathematics code for machine learning problems. Learning about developments in current numerical mathematics, a major backbone of most machine learning methods.