Literature

This page contains a non-exhaustive list of probabilistic numerics research, sorted by problem type.

Note

If you would like your publication to be featured in this list, please open a pull request on GitHub with the corresponding bibtex entry.

General and Foundational

Lar72

F. M. Larkin. Gaussian measure in Hilbert space and applications in numerical analysis. Rocky Mountain Journal of Mathematics, 2(3):379–422, 1972.

Dia88

Persi Diaconis. Bayesian numerical analysis. Statistical decision theory and related topics IV, 1:163–175, 1988.

OHagan92

Anthony O’Hagan. Some Bayesian numerical analysis. Bayesian Statistics, 4(345–363):4–2, 1992.

HOG15

Philipp Hennig, Michael A. Osborne, and Mark Girolami. Probabilistic numerics and uncertainty in computations. Proceedings of the Royal Society of London A: Mathematical, Physical and Engineering Sciences, 2015.

OS16

Houman Owhadi and Clint Scovel. Toward machine Wald. In Springer Handbook of Uncertainty Quantification, pages 1–35. Springer, 2016.

CockayneOatesSullivanGirolami17

J. Cockayne, C. Oates, T. Sullivan, and M. Girolami. Bayesian Probabilistic Numerical Methods. ArXiv e-prints, February 2017.

OS17

Houman Owhadi and Clint Scovel. Universal Scalable Robust Solvers from Computational Information Games and fast eigenspace adapted Multiresolution Analysis. arXiv:1703.10761 [math, stat], March 2017. URL: http://arxiv.org/abs/1703.10761 (visited on 2017-09-10).

OS15

Houman Owhadi and Clint Scovel. Conditioning Gaussian measure on Hilbert space. arXiv:1506.04208 [math], June 2015. arXiv: 1506.04208. URL: http://arxiv.org/abs/1506.04208 (visited on 2017-09-10).

OatesSullivan19

C. J. Oates and T. J. Sullivan. A Modern Retrospective on Probabilistic Numerics. arXiv e-prints, Jan 2019.

OatesCockaynePrangle+19

Chris. J. Oates, Jon Cockayne, Dennis Prangle, T. J. Sullivan, and Mark Girolami. Optimality Criteria for Probabilistic Numerical Methods. arXiv e-prints, Jan 2019.

TFB+21

Onur Teymur, Christopher N Foley, Philip G Breen, Toni Karvonen, and Chris J Oates. Black box probabilistic numerics. arXiv e-prints, 2021.

Quadrature

OHagan87

A. O'Hagan. Monte carlo is fundamentally unsound. Journal of the Royal Statistical Society. Series D (The Statistician), 36(2/3):pp. 247–249, 1987.

KOHagan96

MC Kennedy and A O’Hagan. Iterative rescaling for bayesian quadrature. Bayesian Statistics, 5:639–645, 1996.

Ken98

Marc Kennedy. Bayesian quadrature with non-normal approximating functions. Statistics and Computing, 8(4):365–375, 1998.

OHagan91

A. O'Hagan. Bayes–Hermite quadrature. Journal of statistical planning and inference, 29(3):245–260, 1991.

Min00

T.P. Minka. Deriving quadrature rules from Gaussian processes. Technical Report, Statistics Department, Carnegie Mellon University, 2000.

GR02

Zoubin Ghahramani and Carl E Rasmussen. Bayesian Monte Carlo. In Advances in neural information processing systems, 489–496. 2002.

KMM+03

Augustine Kong, Peter McCullagh, Xiao-Li Meng, Dan L. Nicolae, and Zhiquiang Tan. A theory of statistical models for Monte Carlo integration. Journal of the Royal Statistical Society, Series B (Statistical Methodology), 65(3):585–618, 2003.

Tan04

Zhiqiang Tan. On a Likelihood Approach for Monte Carlo Integration. Journal of the American Statistical Association, 99(468):1027–1036, 2004.

KMMN07

Augustine Kong, Peter McCullagh, Xiao-Li Meng, and Dan L. Nicolae. Further Explorations of Likelihood Theory for Monte Carlo Integration. Advances in Statistical Modelling and Inference, pages 563–592, 2007.

OGR+12

M.A. Osborne, R. Garnett, S.J. Roberts, C. Hart, S. Aigrain, and N. Gibson. Bayesian quadrature for ratios. In International Conference on Artificial Intelligence and Statistics, 832–840. 2012.

ODG+12

M.A. Osborne, D.K. Duvenaud, R. Garnett, C.E. Rasmussen, S.J. Roberts, and Z. Ghahramani. Active Learning of Model Evidence Using Bayesian Quadrature. In Advances in Neural Information Processing Systems (NIPS), 46–54. 2012.

SarkkaHSS14

Simo Särkkä, Jouni Hartikainen, Lennart Svensson, and Fredrik Sandblom. Gaussian process quadratures in nonlinear sigma-point filtering and smoothing. In FUSION. 2014.

GOG+14

Tom Gunter, Michael A. Osborne, Roman Garnett, Philipp Hennig, and Stephen Roberts. Sampling for inference in probabilistic models with fast bayesian quadrature. In C. Cortes and N. Lawrence, editors, Advances in Neural Information Processing Systems (NIPS). 2014.

SarkkaHartikainenSvenssonSandblom15

S. Särkkä, J. Hartikainen, L. Svensson, and F. Sandblom. On the relation between Gaussian process quadratures and sigma-point methods. arXiv preprint stat.ME 1504.05994, April 2015.

ETKW14

S. M. Ali Eslami, Daniel Tarlow, Pushmeet Kohli, and John Winn. Just-in-time learning for fast and flexible inference. In Z. Ghahramani, M. Welling, C. Cortes, N.D. Lawrence, and K.Q. Weinberger, editors, Advances in Neural Information Processing Systems (NIPS) 27, 154–162. 2014.

JitkrittumGrettonHeess+15

W. Jitkrittum, A. Gretton, N. Heess, S. M. A. Eslami, B. Lakshminarayanan, D. Sejdinovic, and Z. Szabó. Kernel-Based Just-In-Time Learning for Passing Expectation Propagation Messages. In Uncertainty in Artificial Intelligence (UAI) 31. 2015.

BOGO15

François-Xavier Briol, Chris J. Oates, Mark Girolami, and Michael A. Osborne. Frank-Wolfe Bayesian Quadrature: Probabilistic Integration with Theoretical Guarantees. In Advances in Neural Information Processing Systems (NIPS). 2015. URL: http://arxiv.org/abs/1506.02681.

Bac15

Francis Bach. On the equivalence between quadrature rules and random features. arXiv preprint arXiv:1502.06800, 2015.

BOG+15

François-Xavier Briol, Chris J. Oates, Mark Girolami, Michael A. Osborne, and Dino Sejdinovic. Probabilistic Integration: a role for statisticians in numerical analysis? arXiv:1512.00933 [cs, math, stat], 2015. arXiv: 1512.00933. URL: http://arxiv.org/abs/1512.00933 (visited on 2015-07-22).

PruhervSimandl16

Jakub Prüher and Miroslav Šimandl. Bayesian Quadrature Variance in Sigma-Point Filtering. In Joaquim Filipe, Kurosh Madani, Oleg Gusikhin, and Jurek Sasiadek, editors, International Conference on Informatics in Control, Automation and Robotics (ICINCO) Revised Selected Papers, volume 12, pages 355–370. Springer International Publishing, Colmar, France, 2016.

KSF16

Motonobu Kanagawa, Bharath K. Sriperumbudur, and Kenji Fukumizu. Convergence guarantees for kernel-based quadrature rules in misspecified settings. In D. D. Lee, M. Sugiyama, U. V. Luxburg, I. Guyon, and R. Garnett, editors, Advances in Neural Information Processing Systems 29, pages 3288–3296. Curran Associates, Inc., 2016.

KS17

Toni Karvonen and Simo Särkkä. Classical quadrature rules via gaussian processes. In 2017 IEEE 27th International Workshop on Machine Learning for Signal Processing (MLSP), volume, 1–6. 2017. doi:10.1109/MLSP.2017.8168195.

KSarkka17

Toni Karvonen and Simo Särkkä. Fully symmetric kernel quadrature. arXiv:1703.06359 [cs, math, stat], March 2017. arXiv: 1703.06359. URL: http://arxiv.org/abs/1703.06359 (visited on 2017-09-11).

BOC+17

François-Xavier Briol, Chris J. Oates, Jon Cockayne, Wilson Ye Chen, and Mark Girolami. On the Sampling Problem for Kernel Quadrature. In Thirty-fourth International Conference on Machine Learning (ICML 2017). June 2017. arXiv: 1706.03369. URL: http://arxiv.org/abs/1706.03369 (visited on 2017-09-11).

ONL+16

Chris J. Oates, Steven Niederer, Angela Lee, François-Xavier Briol, and Mark Girolami. Probabilistic Models for Integration Error in the Assessment of Functional Cardiac Models. arXiv:1606.06841 [stat], June 2016. arXiv: 1606.06841. URL: https://arxiv.org/pdf/1606.06841.pdf (visited on 2017-09-11).

KSF17

Motonobu Kanagawa, Bharath K. Sriperumbudur, and Kenji Fukumizu. Convergence Analysis of Deterministic Kernel-Based Quadrature Rules in Misspecified Settings. arXiv:1709.00147 [cs, math, stat], September 2017. arXiv: 1709.00147. URL: http://arxiv.org/abs/1709.00147 (visited on 2017-09-12).

XiBriolGirolami18

X. Xi, F.-X. Briol, and M. Girolami. Bayesian Quadrature for Multiple Related Integrals. ArXiv e-prints, January 2018. arXiv:1801.04153.

EhlerGraefOates17

M. Ehler, M. Graef, and C. J. Oates. Optimal Monte Carlo integration on closed manifolds. ArXiv e-prints, July 2017. arXiv:1707.04723.

FOPT20

Matthew Fisher, Chris Oates, Catherine Powell, and Aretha Teckentrup. A locally adaptive bayesian cubature method. In International Conference on Artificial Intelligence and Statistics, 1265–1275. PMLR, 2020.

Linear Algebra

SOEnsslin12

Marco Selig, Niels Oppermann, and Torsten A. Enßlin. Improving stochastic estimates with inference methods: calculating matrix diagonals. Phys. Rev. E, 85:021134, Feb 2012.

Hennig15

P. Hennig. Probabilistic Interpretation of Linear Solvers. SIAM J on Optimization, January 2015.

DEnsslin15

Sebastian Dorn and Torsten A. Enßlin. Stochastic determination of matrix determinants. Phys. Rev. E, 92:013302, Jul 2015.

FCO+17

Jack Fitzsimons, Kurt Cutajar, Michael Osborne, Stephen Roberts, and Maurizio Filippone. Bayesian Inference of Log Determinants. In Uncertainty in Artificial Intelligence. 2017. URL: https://arxiv.org/abs/1704.01445 (visited on 2017-06-21).

Bar16

Probabilistic Approximate Least-Squares, volume 51 of JMLR Workshop and Conference Proceedings, 2016.

SSO17

Florian Schäfer, T. J. Sullivan, and Houman Owhadi. Compression, inversion, and approximate PCA of dense kernel matrices at near-linear computational complexity. arXiv:1706.02205 [cs, math], June 2017. arXiv: 1706.02205. URL: http://arxiv.org/abs/1706.02205 (visited on 2017-09-10).

COIG19

Jon Cockayne, Chris J Oates, Ilse CF Ipsen, and Mark Girolami. A bayesian conjugate gradient method (with discussion). Bayesian Analysis, 14(3):937–1012, 2019.

CIOR20

Jon Cockayne, Ilse CF Ipsen, Chris J Oates, and Tim W Reid. Probabilistic iterative methods for linear systems. arXiv preprint arXiv:2012.12615, 2020.

RICO20

Tim W Reid, Ilse CF Ipsen, Jon Cockayne, and Chris J Oates. A probabilistic numerical extension of the conjugate gradient method. arXiv preprint arXiv:2008.03225, 2020.

WH20

Jonathan Wenger and Philipp Hennig. Probabilistic linear solvers for machine learning. In Advances in Neural Information Processing Systems (NeurIPS). 2020.

Global Optimization

MTZ78

Jonas Mockus, Vytautas Tiesis, and Antanas Zilinskas. The application of bayesian methods for seeking the extremum. Towards global optimization, 2(117-129):2, 1978.

HS12

P. Hennig and CJ. Schuler. Entropy search for information-efficient global optimization. Journal of Machine Learning Research, 13:1809–1837, June 2012.

Local Optimization

HK12

P. Hennig and M. Kiefel. Quasi-Newton methods – a new direction. In International Conference on Machine Learning (ICML). 2012.

HK13

P. Hennig and M. Kiefel. Quasi-Newton methods – a new direction. Journal of Machine Learning Research, 14:834–865, March 2013.

Hen13

P. Hennig. Fast Probabilistic Optimization from Noisy Gradients. In International Conference on Machine Learning (ICML). 2013.

MH15

Maren Mahsereci and Philipp Hennig. Probabilistic line searches for stochastic optimization. In C. Cortes, N. D. Lawrence, D. D. Lee, M. Sugiyama, and R. Garnett, editors, Advances in Neural Information Processing Systems 28, pages 181–189. Curran Associates, Inc., 2015.

DenizAkyildizElviraMiguez18

Ö. Deniz Akyildiz, V. Elvira, and J. Miguez. The Incremental Proximal Method: A Probabilistic Perspective. ArXiv e-prints, July 2018.

Ordinary Differential Equations

HS66

T. E. Hull and J. R. Swenson. Test of Probabilistic Models for the Propagation of Roundoff Errors. Communications of the ACM, 9(2):108–113, 1966.

KC73

H. Kuki and W. J. Cody. A Statistical Study Of The Accuracy Of Floating Point Number Systems. Communications of the ACM, 16(1):223–230, 1973.

Ski91

J. Skilling. Bayesian solution of ordinary differential equations. Maximum Entropy and Bayesian Methods, Seattle, 1991.

Gra03

Thore Graepel. Solving noisy linear operator equations by gaussian processes: application to ordinary and partial differential equations. In ICML, 234–241. 2003.

MT09

Sebastian Mosbach and Amanda G. Turner. A quantitative probabilistic investigation into the accumulation of rounding errors in numerical ODE solution. Computers & Mathematics with Applications, 57(7):1157–1167, 2009.

RSGEnsslin13

Tiago Ramalho, Marco Selig, Ulrich Gerland, and Torsten A. Enßlin. Simulation of stochastic network dynamics via entropic matching. Phys. Rev. E, 87:022719, Feb 2013. doi:10.1103/PhysRevE.87.022719.

HH14

Philipp Hennig and Søren Hauberg. Probabilistic Solutions to Differential Equations and their Application to Riemannian Statistics. In Proc. of the 17th int. Conf. on Artificial Intelligence and Statistics (AISTATS), volume 33. JMLR, W&CP, 2014.

CCGC13

O. Chkrebtii, D.A. Campbell, M.A. Girolami, and B. Calderhead. Bayesian uncertainty quantification for differential equations. Bayesin Analysis (discussion paper), pages in press, 2013.

SKF+14

Michael Schober, Niklas Kasenburg, Aasa Feragen, Philipp Hennig, and Søren Hauberg. Probabilistic shortest path tractography in DTI using Gaussian Process ODE solvers. In Polina Golland, Nobuhiko Hata, Christian Barillot, Joachim Hornegger, and Robert Howe, editors, Medical Image Computing and Computer-Assisted Intervention – MICCAI 2014, volume 8675 of Lecture Notes in Computer Science, 265–272. Springer, 2014.

SDH14

Michael Schober, David K Duvenaud, and Philipp Hennig. Probabilistic ODE solvers with Runge-Kutta means. In Z. Ghahramani, M. Welling, C. Cortes, N.D. Lawrence, and K.Q. Weinberger, editors, Advances in Neural Information Processing Systems 27, pages 739–747. Curran Associates, Inc., 2014.

Barber14

D. Barber. On solving Ordinary Differential Equations using Gaussian Processes. ArXiv pre-print 1408.3807, August 2014.

HSL+15

Søren Hauberg, Michael Schober, Matthew Liptrot, Philipp Hennig, and Aasa Feragen. A random riemannian metric for probabilistic shortest-path tractography. In Medical Image Computing and Computer-Assisted Intervention (MICCAI), volume 18. Munich, Germany, September 2015.

KH16

Hans P. Kersting and Philipp Hennig. Active uncertainty calibration in Bayesian ODE solvers. In Janzing and Ihlers, editors, Uncertainty in Artificial Intelligence (UAI), volume 32. 2016.

SchoberSarkkaHennig16

M. Schober, S. Särkkä, and P. Hennig. A probabilistic model for the numerical solution of initial value problems. ArXiv e-prints, October 2016. arXiv:1610.05261.

AbdulleGaregnani18

A. Abdulle and G. Garegnani. Random time step probabilistic methods for uncertainty quantification in chaotic and geometric numerical integration. ArXiv e-prints, 1 2018.

KerstingSullivanHennig18

H. Kersting, T. J. Sullivan, and P. Hennig. Convergence Rates of Gaussian ODE Filters. ArXiv e-prints, 7 2018.

TZC16

Onur Teymur, Kostas Zygalakis, and Ben Calderhead. Probabilistic Linear Multistep Methods. In D. D. Lee, M. Sugiyama, U. V. Luxburg, I. Guyon, and R. Garnett, editors, Advances in Neural Information Processing Systems 29, pages 4314–4321. Curran Associates, Inc., 2016.

TLSC18

Onur Teymur, Han Cheng Lie, Tim Sullivan, and Ben Calderhead. Implicit Probabilistic Integrators for ODEs. In Advances in Neural Information Processing Systems 31. Curran Associates, Inc., 2018.

WCO20

Junyang Wang, Jon Cockayne, and Chris J Oates. A role for symmetry in the Bayesian solution of differential equations. Bayesian Analysis, 15(4):1057–1085, 2020.

Partial Differential Equations

Ensslin13

Torsten A. Enßlin. Information field dynamics for simulation scheme construction. Phys. Rev. E, 87:013308, Jan 2013. doi:10.1103/PhysRevE.87.013308.

Owh15

Houman Owhadi. Bayesian numerical homogenization. Multiscale Modeling & Simulation, 13(3):812–828, 2015.

CGS+15

Patrick R. Conrad, Mark Girolami, Simo Särkkä, Andrew Stuart, and Konstantinos Zygalakis. Probability Measures for Numerical Solutions of Differential Equations. arXiv:1506.04592 [stat], June 2015. arXiv: 1506.04592.

OCA17

Chris J. Oates, Jon Cockayne, and Robert G. Aykroyd. Bayesian Probabilistic Numerical Methods for Industrial Process Monitoring. arXiv:1707.06107 [stat], July 2017. arXiv: 1707.06107. URL: http://arxiv.org/abs/1707.06107 (visited on 2017-07-20).

Owhadi15

H. Owhadi. Multigrid with rough coefficients and multiresolution operator decomposition from hierarchical information games. ArXiv, March 2015.

CockayneOatesSullivanGirolami16

J. Cockayne, C. Oates, T. Sullivan, and M. Girolami. Probabilistic meshless methods for partial differential equations and Bayesian inverse problems. ArXiv, may 2016.

OwhadiZhang16

H. Owhadi and L. Zhang. Gamblets for opening the complexity-bottleneck of implicit schemes for hyperbolic and parabolic ODEs/PDEs with rough coefficients. ArXiv e-prints, June 2016.

OZ17

Houman Owhadi and Lei Zhang. Gamblets for opening the complexity-bottleneck of implicit schemes for hyperbolic and parabolic ODEs/PDEs with rough coefficients. Journal of Computational Physics, 347:99–128, October 2017. arXiv: 1606.07686. URL: http://arxiv.org/abs/1606.07686 (visited on 2017-09-10), doi:10.1016/j.jcp.2017.06.037.

DupontEnsslin18

M. Dupont and T. Enßlin. Consistency and convergence of simulation schemes in information field dynamics. ArXiv e-prints, February 2018. arXiv:1802.00971.

LEnsslin18

Reimar H. Leike and Torsten A. Enßlin. Towards information-optimal simulation of partial differential equations. Phys. Rev. E, 97:033314, Mar 2018. doi:10.1103/PhysRevE.97.033314.

WCC+21

Junyang Wang, Jon Cockayne, Oksana Chkrebtii, Timothy John Sullivan, Chris Oates, and others. Bayesian numerical methods for nonlinear partial differential equations. Statistics and Computing, 2021.

Historical Influences

The following papers are not on probabilistic numerics per se, but are often cited as early works on the interplay of uncertainty and deterministic computations.

Poincare96

H. Poincaré. Calcul des probabilités. Gauthier-Villars, Paris, 1896.

ErdosK40

P. Erdös and M. Kac. The gaussian law of errors in the theory of additive number theoretic functions. American Journal of Mathematics, pages 738–742, 1940.

Kac49

M. Kac. On distributions of certain Wiener functionals. Transactions of the AMS, 65(1):1–13, 1949.

Suldin59

Al'bert Valentinovich Sul'din. Wiener measure and its applications to approximation methods. i. Izvestiya Vysshikh Uchebnykh Zavedenii. Matematika, pages 145–158, 1959.

Suldin60

Al'bert Valentinovich Sul'din. Wiener measure and its applications to approximation methods. ii. Izvestiya Vysshikh Uchebnykh Zavedenii. Matematika, pages 165–179, 1960.

AD60

Björn Ajna and Tore Dalenius. Några tillämpningar av statistiska idéer på numerisk integration. Nordisk Matematisk Tidskrift, pages 145–152, 1960.

Sar63

Arthur Sard. Linear approximation. American Mathematical Society, Providence, R.I., 1963.

KW70

George S Kimeldorf and Grace Wahba. A correspondence between bayesian estimation on stochastic processes and smoothing by splines. The Annals of Mathematical Statistics, 41(2):495–502, 1970.

KW85

J. B. Kadane and G. W. Wasilkowski. Average case epsilon-complexity in computer science: A Bayesian view. In Bayesian Statistics 2, Proceedings of the Second Valencia International Meeting, number July, 361–374. 1985.

Kad85

J. B. Kadane. Parallel and Sequential Computation: A Statistician's View. Journal of Complexity, 1:256–263, 1985.

Kop94

P Kopanov. Probabilistic analysis of methods for numerical integration. Dokladi na Bulgarskata Akademia na Naukite, 47(4):17–20, 1994.

Kop95

P Kopanov. On the optimality of the trapezoidal method when integrating the wiener process. Journal of Applied Statistical Science, 2(4):397–403, 1995.

Kop96

P Kopanov. Rate of convergence for the approximate integration of the wiener process. Mathematica Balkanica, 10(1):83–88, 1996.