Literature¶
This page contains a non-exhaustive list of probabilistic numerics research, sorted by problem type.
Note
If you would like your publication to be featured in this list, please open a pull request on GitHub with the corresponding bibtex
entry.
General and Foundational¶
- Lar72
F. M. Larkin. Gaussian measure in Hilbert space and applications in numerical analysis. Rocky Mountain Journal of Mathematics, 2(3):379–422, 1972.
- Dia88
Persi Diaconis. Bayesian numerical analysis. Statistical decision theory and related topics IV, 1:163–175, 1988.
- OHagan92
Anthony O’Hagan. Some Bayesian numerical analysis. Bayesian Statistics, 4(345–363):4–2, 1992.
- HOG15
Philipp Hennig, Michael A. Osborne, and Mark Girolami. Probabilistic numerics and uncertainty in computations. Proceedings of the Royal Society of London A: Mathematical, Physical and Engineering Sciences, 2015.
- OS16
Houman Owhadi and Clint Scovel. Toward machine Wald. In Springer Handbook of Uncertainty Quantification, pages 1–35. Springer, 2016.
- CockayneOatesSullivanGirolami17
J. Cockayne, C. Oates, T. Sullivan, and M. Girolami. Bayesian Probabilistic Numerical Methods. ArXiv e-prints, February 2017.
- OS17
Houman Owhadi and Clint Scovel. Universal Scalable Robust Solvers from Computational Information Games and fast eigenspace adapted Multiresolution Analysis. arXiv:1703.10761 [math, stat], March 2017. URL: http://arxiv.org/abs/1703.10761 (visited on 2017-09-10).
- OS15
Houman Owhadi and Clint Scovel. Conditioning Gaussian measure on Hilbert space. arXiv:1506.04208 [math], June 2015. arXiv: 1506.04208. URL: http://arxiv.org/abs/1506.04208 (visited on 2017-09-10).
- OatesSullivan19
C. J. Oates and T. J. Sullivan. A Modern Retrospective on Probabilistic Numerics. arXiv e-prints, Jan 2019.
- OatesCockaynePrangle+19
Chris. J. Oates, Jon Cockayne, Dennis Prangle, T. J. Sullivan, and Mark Girolami. Optimality Criteria for Probabilistic Numerical Methods. arXiv e-prints, Jan 2019.
- TFB+21
Onur Teymur, Christopher N Foley, Philip G Breen, Toni Karvonen, and Chris J Oates. Black box probabilistic numerics. arXiv e-prints, 2021.
Quadrature¶
- OHagan87
A. O'Hagan. Monte carlo is fundamentally unsound. Journal of the Royal Statistical Society. Series D (The Statistician), 36(2/3):pp. 247–249, 1987.
- KOHagan96
MC Kennedy and A O’Hagan. Iterative rescaling for bayesian quadrature. Bayesian Statistics, 5:639–645, 1996.
- Ken98
Marc Kennedy. Bayesian quadrature with non-normal approximating functions. Statistics and Computing, 8(4):365–375, 1998.
- OHagan91
A. O'Hagan. Bayes–Hermite quadrature. Journal of statistical planning and inference, 29(3):245–260, 1991.
- Min00
T.P. Minka. Deriving quadrature rules from Gaussian processes. Technical Report, Statistics Department, Carnegie Mellon University, 2000.
- GR02
Zoubin Ghahramani and Carl E Rasmussen. Bayesian Monte Carlo. In Advances in neural information processing systems, 489–496. 2002.
- KMM+03
Augustine Kong, Peter McCullagh, Xiao-Li Meng, Dan L. Nicolae, and Zhiquiang Tan. A theory of statistical models for Monte Carlo integration. Journal of the Royal Statistical Society, Series B (Statistical Methodology), 65(3):585–618, 2003.
- Tan04
Zhiqiang Tan. On a Likelihood Approach for Monte Carlo Integration. Journal of the American Statistical Association, 99(468):1027–1036, 2004.
- KMMN07
Augustine Kong, Peter McCullagh, Xiao-Li Meng, and Dan L. Nicolae. Further Explorations of Likelihood Theory for Monte Carlo Integration. Advances in Statistical Modelling and Inference, pages 563–592, 2007.
- OGR+12
M.A. Osborne, R. Garnett, S.J. Roberts, C. Hart, S. Aigrain, and N. Gibson. Bayesian quadrature for ratios. In International Conference on Artificial Intelligence and Statistics, 832–840. 2012.
- ODG+12
M.A. Osborne, D.K. Duvenaud, R. Garnett, C.E. Rasmussen, S.J. Roberts, and Z. Ghahramani. Active Learning of Model Evidence Using Bayesian Quadrature. In Advances in Neural Information Processing Systems (NIPS), 46–54. 2012.
- SarkkaHSS14
Simo Särkkä, Jouni Hartikainen, Lennart Svensson, and Fredrik Sandblom. Gaussian process quadratures in nonlinear sigma-point filtering and smoothing. In FUSION. 2014.
- GOG+14
Tom Gunter, Michael A. Osborne, Roman Garnett, Philipp Hennig, and Stephen Roberts. Sampling for inference in probabilistic models with fast bayesian quadrature. In C. Cortes and N. Lawrence, editors, Advances in Neural Information Processing Systems (NIPS). 2014.
- SarkkaHartikainenSvenssonSandblom15
S. Särkkä, J. Hartikainen, L. Svensson, and F. Sandblom. On the relation between Gaussian process quadratures and sigma-point methods. arXiv preprint stat.ME 1504.05994, April 2015.
- ETKW14
S. M. Ali Eslami, Daniel Tarlow, Pushmeet Kohli, and John Winn. Just-in-time learning for fast and flexible inference. In Z. Ghahramani, M. Welling, C. Cortes, N.D. Lawrence, and K.Q. Weinberger, editors, Advances in Neural Information Processing Systems (NIPS) 27, 154–162. 2014.
- JitkrittumGrettonHeess+15
W. Jitkrittum, A. Gretton, N. Heess, S. M. A. Eslami, B. Lakshminarayanan, D. Sejdinovic, and Z. Szabó. Kernel-Based Just-In-Time Learning for Passing Expectation Propagation Messages. In Uncertainty in Artificial Intelligence (UAI) 31. 2015.
- BOGO15
François-Xavier Briol, Chris J. Oates, Mark Girolami, and Michael A. Osborne. Frank-Wolfe Bayesian Quadrature: Probabilistic Integration with Theoretical Guarantees. In Advances in Neural Information Processing Systems (NIPS). 2015. URL: http://arxiv.org/abs/1506.02681.
- Bac15
Francis Bach. On the equivalence between quadrature rules and random features. arXiv preprint arXiv:1502.06800, 2015.
- BOG+15
François-Xavier Briol, Chris J. Oates, Mark Girolami, Michael A. Osborne, and Dino Sejdinovic. Probabilistic Integration: a role for statisticians in numerical analysis? arXiv:1512.00933 [cs, math, stat], 2015. arXiv: 1512.00933. URL: http://arxiv.org/abs/1512.00933 (visited on 2015-07-22).
- PruhervSimandl16
Jakub Prüher and Miroslav Šimandl. Bayesian Quadrature Variance in Sigma-Point Filtering. In Joaquim Filipe, Kurosh Madani, Oleg Gusikhin, and Jurek Sasiadek, editors, International Conference on Informatics in Control, Automation and Robotics (ICINCO) Revised Selected Papers, volume 12, pages 355–370. Springer International Publishing, Colmar, France, 2016.
- KSF16
Motonobu Kanagawa, Bharath K. Sriperumbudur, and Kenji Fukumizu. Convergence guarantees for kernel-based quadrature rules in misspecified settings. In D. D. Lee, M. Sugiyama, U. V. Luxburg, I. Guyon, and R. Garnett, editors, Advances in Neural Information Processing Systems 29, pages 3288–3296. Curran Associates, Inc., 2016.
- KS17
Toni Karvonen and Simo Särkkä. Classical quadrature rules via gaussian processes. In 2017 IEEE 27th International Workshop on Machine Learning for Signal Processing (MLSP), volume, 1–6. 2017. doi:10.1109/MLSP.2017.8168195.
- KSarkka17
Toni Karvonen and Simo Särkkä. Fully symmetric kernel quadrature. arXiv:1703.06359 [cs, math, stat], March 2017. arXiv: 1703.06359. URL: http://arxiv.org/abs/1703.06359 (visited on 2017-09-11).
- BOC+17
François-Xavier Briol, Chris J. Oates, Jon Cockayne, Wilson Ye Chen, and Mark Girolami. On the Sampling Problem for Kernel Quadrature. In Thirty-fourth International Conference on Machine Learning (ICML 2017). June 2017. arXiv: 1706.03369. URL: http://arxiv.org/abs/1706.03369 (visited on 2017-09-11).
- ONL+16
Chris J. Oates, Steven Niederer, Angela Lee, François-Xavier Briol, and Mark Girolami. Probabilistic Models for Integration Error in the Assessment of Functional Cardiac Models. arXiv:1606.06841 [stat], June 2016. arXiv: 1606.06841. URL: https://arxiv.org/pdf/1606.06841.pdf (visited on 2017-09-11).
- KSF17
Motonobu Kanagawa, Bharath K. Sriperumbudur, and Kenji Fukumizu. Convergence Analysis of Deterministic Kernel-Based Quadrature Rules in Misspecified Settings. arXiv:1709.00147 [cs, math, stat], September 2017. arXiv: 1709.00147. URL: http://arxiv.org/abs/1709.00147 (visited on 2017-09-12).
- XiBriolGirolami18
X. Xi, F.-X. Briol, and M. Girolami. Bayesian Quadrature for Multiple Related Integrals. ArXiv e-prints, January 2018. arXiv:1801.04153.
- EhlerGraefOates17
M. Ehler, M. Graef, and C. J. Oates. Optimal Monte Carlo integration on closed manifolds. ArXiv e-prints, July 2017. arXiv:1707.04723.
- FOPT20
Matthew Fisher, Chris Oates, Catherine Powell, and Aretha Teckentrup. A locally adaptive bayesian cubature method. In International Conference on Artificial Intelligence and Statistics, 1265–1275. PMLR, 2020.
Linear Algebra¶
- SOEnsslin12
Marco Selig, Niels Oppermann, and Torsten A. Enßlin. Improving stochastic estimates with inference methods: calculating matrix diagonals. Phys. Rev. E, 85:021134, Feb 2012.
- Hennig15
P. Hennig. Probabilistic Interpretation of Linear Solvers. SIAM J on Optimization, January 2015.
- DEnsslin15
Sebastian Dorn and Torsten A. Enßlin. Stochastic determination of matrix determinants. Phys. Rev. E, 92:013302, Jul 2015.
- FCO+17
Jack Fitzsimons, Kurt Cutajar, Michael Osborne, Stephen Roberts, and Maurizio Filippone. Bayesian Inference of Log Determinants. In Uncertainty in Artificial Intelligence. 2017. URL: https://arxiv.org/abs/1704.01445 (visited on 2017-06-21).
- Bar16
Probabilistic Approximate Least-Squares, volume 51 of JMLR Workshop and Conference Proceedings, 2016.
- SSO17
Florian Schäfer, T. J. Sullivan, and Houman Owhadi. Compression, inversion, and approximate PCA of dense kernel matrices at near-linear computational complexity. arXiv:1706.02205 [cs, math], June 2017. arXiv: 1706.02205. URL: http://arxiv.org/abs/1706.02205 (visited on 2017-09-10).
- COIG19
Jon Cockayne, Chris J Oates, Ilse CF Ipsen, and Mark Girolami. A bayesian conjugate gradient method (with discussion). Bayesian Analysis, 14(3):937–1012, 2019.
- CIOR20
Jon Cockayne, Ilse CF Ipsen, Chris J Oates, and Tim W Reid. Probabilistic iterative methods for linear systems. arXiv preprint arXiv:2012.12615, 2020.
- RICO20
Tim W Reid, Ilse CF Ipsen, Jon Cockayne, and Chris J Oates. A probabilistic numerical extension of the conjugate gradient method. arXiv preprint arXiv:2008.03225, 2020.
- WH20
Jonathan Wenger and Philipp Hennig. Probabilistic linear solvers for machine learning. In Advances in Neural Information Processing Systems (NeurIPS). 2020.
Global Optimization¶
- MTZ78
Jonas Mockus, Vytautas Tiesis, and Antanas Zilinskas. The application of bayesian methods for seeking the extremum. Towards global optimization, 2(117-129):2, 1978.
- HS12
P. Hennig and CJ. Schuler. Entropy search for information-efficient global optimization. Journal of Machine Learning Research, 13:1809–1837, June 2012.
Local Optimization¶
- HK12
P. Hennig and M. Kiefel. Quasi-Newton methods – a new direction. In International Conference on Machine Learning (ICML). 2012.
- HK13
P. Hennig and M. Kiefel. Quasi-Newton methods – a new direction. Journal of Machine Learning Research, 14:834–865, March 2013.
- Hen13
P. Hennig. Fast Probabilistic Optimization from Noisy Gradients. In International Conference on Machine Learning (ICML). 2013.
- MH15
Maren Mahsereci and Philipp Hennig. Probabilistic line searches for stochastic optimization. In C. Cortes, N. D. Lawrence, D. D. Lee, M. Sugiyama, and R. Garnett, editors, Advances in Neural Information Processing Systems 28, pages 181–189. Curran Associates, Inc., 2015.
- DenizAkyildizElviraMiguez18
Ö. Deniz Akyildiz, V. Elvira, and J. Miguez. The Incremental Proximal Method: A Probabilistic Perspective. ArXiv e-prints, July 2018.
Ordinary Differential Equations¶
- HS66
T. E. Hull and J. R. Swenson. Test of Probabilistic Models for the Propagation of Roundoff Errors. Communications of the ACM, 9(2):108–113, 1966.
- KC73
H. Kuki and W. J. Cody. A Statistical Study Of The Accuracy Of Floating Point Number Systems. Communications of the ACM, 16(1):223–230, 1973.
- Ski91
J. Skilling. Bayesian solution of ordinary differential equations. Maximum Entropy and Bayesian Methods, Seattle, 1991.
- Gra03
Thore Graepel. Solving noisy linear operator equations by gaussian processes: application to ordinary and partial differential equations. In ICML, 234–241. 2003.
- MT09
Sebastian Mosbach and Amanda G. Turner. A quantitative probabilistic investigation into the accumulation of rounding errors in numerical ODE solution. Computers & Mathematics with Applications, 57(7):1157–1167, 2009.
- RSGEnsslin13
Tiago Ramalho, Marco Selig, Ulrich Gerland, and Torsten A. Enßlin. Simulation of stochastic network dynamics via entropic matching. Phys. Rev. E, 87:022719, Feb 2013. doi:10.1103/PhysRevE.87.022719.
- HH14
Philipp Hennig and Søren Hauberg. Probabilistic Solutions to Differential Equations and their Application to Riemannian Statistics. In Proc. of the 17th int. Conf. on Artificial Intelligence and Statistics (AISTATS), volume 33. JMLR, W&CP, 2014.
- CCGC13
O. Chkrebtii, D.A. Campbell, M.A. Girolami, and B. Calderhead. Bayesian uncertainty quantification for differential equations. Bayesin Analysis (discussion paper), pages in press, 2013.
- SKF+14
Michael Schober, Niklas Kasenburg, Aasa Feragen, Philipp Hennig, and Søren Hauberg. Probabilistic shortest path tractography in DTI using Gaussian Process ODE solvers. In Polina Golland, Nobuhiko Hata, Christian Barillot, Joachim Hornegger, and Robert Howe, editors, Medical Image Computing and Computer-Assisted Intervention – MICCAI 2014, volume 8675 of Lecture Notes in Computer Science, 265–272. Springer, 2014.
- SDH14
Michael Schober, David K Duvenaud, and Philipp Hennig. Probabilistic ODE solvers with Runge-Kutta means. In Z. Ghahramani, M. Welling, C. Cortes, N.D. Lawrence, and K.Q. Weinberger, editors, Advances in Neural Information Processing Systems 27, pages 739–747. Curran Associates, Inc., 2014.
- Barber14
D. Barber. On solving Ordinary Differential Equations using Gaussian Processes. ArXiv pre-print 1408.3807, August 2014.
- HSL+15
Søren Hauberg, Michael Schober, Matthew Liptrot, Philipp Hennig, and Aasa Feragen. A random riemannian metric for probabilistic shortest-path tractography. In Medical Image Computing and Computer-Assisted Intervention (MICCAI), volume 18. Munich, Germany, September 2015.
- KH16
Hans P. Kersting and Philipp Hennig. Active uncertainty calibration in Bayesian ODE solvers. In Janzing and Ihlers, editors, Uncertainty in Artificial Intelligence (UAI), volume 32. 2016.
- SchoberSarkkaHennig16
M. Schober, S. Särkkä, and P. Hennig. A probabilistic model for the numerical solution of initial value problems. ArXiv e-prints, October 2016. arXiv:1610.05261.
- AbdulleGaregnani18
A. Abdulle and G. Garegnani. Random time step probabilistic methods for uncertainty quantification in chaotic and geometric numerical integration. ArXiv e-prints, 1 2018.
- KerstingSullivanHennig18
H. Kersting, T. J. Sullivan, and P. Hennig. Convergence Rates of Gaussian ODE Filters. ArXiv e-prints, 7 2018.
- TZC16
Onur Teymur, Kostas Zygalakis, and Ben Calderhead. Probabilistic Linear Multistep Methods. In D. D. Lee, M. Sugiyama, U. V. Luxburg, I. Guyon, and R. Garnett, editors, Advances in Neural Information Processing Systems 29, pages 4314–4321. Curran Associates, Inc., 2016.
- TLSC18
Onur Teymur, Han Cheng Lie, Tim Sullivan, and Ben Calderhead. Implicit Probabilistic Integrators for ODEs. In Advances in Neural Information Processing Systems 31. Curran Associates, Inc., 2018.
- WCO20
Junyang Wang, Jon Cockayne, and Chris J Oates. A role for symmetry in the Bayesian solution of differential equations. Bayesian Analysis, 15(4):1057–1085, 2020.
Partial Differential Equations¶
- Ensslin13
Torsten A. Enßlin. Information field dynamics for simulation scheme construction. Phys. Rev. E, 87:013308, Jan 2013. doi:10.1103/PhysRevE.87.013308.
- Owh15
Houman Owhadi. Bayesian numerical homogenization. Multiscale Modeling & Simulation, 13(3):812–828, 2015.
- CGS+15
Patrick R. Conrad, Mark Girolami, Simo Särkkä, Andrew Stuart, and Konstantinos Zygalakis. Probability Measures for Numerical Solutions of Differential Equations. arXiv:1506.04592 [stat], June 2015. arXiv: 1506.04592.
- OCA17
Chris J. Oates, Jon Cockayne, and Robert G. Aykroyd. Bayesian Probabilistic Numerical Methods for Industrial Process Monitoring. arXiv:1707.06107 [stat], July 2017. arXiv: 1707.06107. URL: http://arxiv.org/abs/1707.06107 (visited on 2017-07-20).
- Owhadi15
H. Owhadi. Multigrid with rough coefficients and multiresolution operator decomposition from hierarchical information games. ArXiv, March 2015.
- CockayneOatesSullivanGirolami16
J. Cockayne, C. Oates, T. Sullivan, and M. Girolami. Probabilistic meshless methods for partial differential equations and Bayesian inverse problems. ArXiv, may 2016.
- OwhadiZhang16
H. Owhadi and L. Zhang. Gamblets for opening the complexity-bottleneck of implicit schemes for hyperbolic and parabolic ODEs/PDEs with rough coefficients. ArXiv e-prints, June 2016.
- OZ17
Houman Owhadi and Lei Zhang. Gamblets for opening the complexity-bottleneck of implicit schemes for hyperbolic and parabolic ODEs/PDEs with rough coefficients. Journal of Computational Physics, 347:99–128, October 2017. arXiv: 1606.07686. URL: http://arxiv.org/abs/1606.07686 (visited on 2017-09-10), doi:10.1016/j.jcp.2017.06.037.
- DupontEnsslin18
M. Dupont and T. Enßlin. Consistency and convergence of simulation schemes in information field dynamics. ArXiv e-prints, February 2018. arXiv:1802.00971.
- LEnsslin18
Reimar H. Leike and Torsten A. Enßlin. Towards information-optimal simulation of partial differential equations. Phys. Rev. E, 97:033314, Mar 2018. doi:10.1103/PhysRevE.97.033314.
- WCC+21
Junyang Wang, Jon Cockayne, Oksana Chkrebtii, Timothy John Sullivan, Chris Oates, and others. Bayesian numerical methods for nonlinear partial differential equations. Statistics and Computing, 2021.
Historical Influences¶
The following papers are not on probabilistic numerics per se, but are often cited as early works on the interplay of uncertainty and deterministic computations.
- Poincare96
H. Poincaré. Calcul des probabilités. Gauthier-Villars, Paris, 1896.
- ErdosK40
P. Erdös and M. Kac. The gaussian law of errors in the theory of additive number theoretic functions. American Journal of Mathematics, pages 738–742, 1940.
- Kac49
M. Kac. On distributions of certain Wiener functionals. Transactions of the AMS, 65(1):1–13, 1949.
- Suldin59
Al'bert Valentinovich Sul'din. Wiener measure and its applications to approximation methods. i. Izvestiya Vysshikh Uchebnykh Zavedenii. Matematika, pages 145–158, 1959.
- Suldin60
Al'bert Valentinovich Sul'din. Wiener measure and its applications to approximation methods. ii. Izvestiya Vysshikh Uchebnykh Zavedenii. Matematika, pages 165–179, 1960.
- AD60
Björn Ajna and Tore Dalenius. Några tillämpningar av statistiska idéer på numerisk integration. Nordisk Matematisk Tidskrift, pages 145–152, 1960.
- Sar63
Arthur Sard. Linear approximation. American Mathematical Society, Providence, R.I., 1963.
- KW70
George S Kimeldorf and Grace Wahba. A correspondence between bayesian estimation on stochastic processes and smoothing by splines. The Annals of Mathematical Statistics, 41(2):495–502, 1970.
- KW85
J. B. Kadane and G. W. Wasilkowski. Average case epsilon-complexity in computer science: A Bayesian view. In Bayesian Statistics 2, Proceedings of the Second Valencia International Meeting, number July, 361–374. 1985.
- Kad85
J. B. Kadane. Parallel and Sequential Computation: A Statistician's View. Journal of Complexity, 1:256–263, 1985.
- Kop94
P Kopanov. Probabilistic analysis of methods for numerical integration. Dokladi na Bulgarskata Akademia na Naukite, 47(4):17–20, 1994.
- Kop95
P Kopanov. On the optimality of the trapezoidal method when integrating the wiener process. Journal of Applied Statistical Science, 2(4):397–403, 1995.
- Kop96
P Kopanov. Rate of convergence for the approximate integration of the wiener process. Mathematica Balkanica, 10(1):83–88, 1996.