CovarianceFunction

class probnum.randprocs.covfuncs.CovarianceFunction(*, input_shape_0=None, input_shape_1=None, input_shape=None, output_shape_0=(), output_shape_1=())

Bases: ABC

(Cross-)covariance function.

A cross-covariance function

\begin{equation} k \colon \mathbb{X}_0 \times \mathbb{X}_1 \to \mathbb{R}^{d^\text{out}_0 \times d^\text{out}_1}, (x_0, x_1) \mapsto \operatorname{Cov}[f_0(x_0), f_1(x_1)] \end{equation}

is a function of two arguments \(x_0 \in \mathbb{X}_0\) and \(x_1 \in \ \mathbb{X}_1\) (often \(\mathbb{X}_i \subset \mathbb{R}^{d^\text{in}_i}\)), whose output corresponds to the covariance (matrix) between two evaluations \(f_0(x_0) \in \mathbb{R}^{d^\text{out}_0}\) and \(f_1(x_1) \in \ \mathbb{R}^{d^\text{out}_1}\) of two (vector-valued) RandomProcesses \(f_0\) and \(f_1\). If \(f_0 = f_1\), then a cross-covariance function is also referred to as a covariance function or a kernel, in which case it must be symmetric and positive (semi-)definite.

Parameters

Examples

>>> from probnum.randprocs.covfuncs import Linear
>>> D = 3
>>> k = Linear(input_shape=D)
>>> k.input_shape_0
(3,)
>>> k.input_shape_1
(3,)
>>> k.output_shape_0
()
>>> k.output_shape_1
()

Generate some input data.

>>> N = 4
>>> xs = np.linspace(0, 1, N * D).reshape(N, D)
>>> xs.shape
(4, 3)
>>> xs
array([[0.        , 0.09090909, 0.18181818],
       [0.27272727, 0.36363636, 0.45454545],
       [0.54545455, 0.63636364, 0.72727273],
       [0.81818182, 0.90909091, 1.        ]])

We can compute covariance matrices of multiple evaluations like so.

>>> k.matrix(xs)
array([[0.04132231, 0.11570248, 0.19008264, 0.26446281],
       [0.11570248, 0.41322314, 0.7107438 , 1.00826446],
       [0.19008264, 0.7107438 , 1.23140496, 1.75206612],
       [0.26446281, 1.00826446, 1.75206612, 2.49586777]])

The __call__() method is vectorized over the “batch shapes” of the inputs, applying standard NumPy broadcasting.

>>> k(xs[:, None], xs[None, :])  # same as `.matrix`
array([[0.04132231, 0.11570248, 0.19008264, 0.26446281],
       [0.11570248, 0.41322314, 0.7107438 , 1.00826446],
       [0.19008264, 0.7107438 , 1.23140496, 1.75206612],
       [0.26446281, 1.00826446, 1.75206612, 2.49586777]])

No broadcasting is applied if both inputs have the same shape. For instance, one can efficiently compute the marginal variance of a set of data points via

>>> k(xs, xs)
array([0.04132231, 0.41322314, 1.23140496, 2.49586777])
>>> k(xs, None)  # x1 = None is an efficient way to set x1 == x0
array([0.04132231, 0.41322314, 1.23140496, 2.49586777])

CovarianceFunctions support basic arithmetic operations. For example, we can model independent measurement noise as follows:

>>> from probnum.randprocs.covfuncs import WhiteNoise
>>> k_noise = k + 0.1 * WhiteNoise(input_shape=D)
>>> k_noise.matrix(xs)
array([[0.14132231, 0.11570248, 0.19008264, 0.26446281],
       [0.11570248, 0.51322314, 0.7107438 , 1.00826446],
       [0.19008264, 0.7107438 , 1.33140496, 1.75206612],
       [0.26446281, 1.00826446, 1.75206612, 2.59586777]])

Attributes Summary

input_ndim

Syntactic sugar for len(input_shape).

input_ndim_0

Syntactic sugar for len(input_shape_0).

input_ndim_1

Syntactic sugar for len(input_shape_1).

input_shape

Shorthand for the input shape of a covariance function with input_shape_0 == input_shape_1.

input_shape_0

input_shape of the RandomProcess \(f_0\).

input_shape_1

input_shape of the RandomProcess \(f_1\).

input_size

Syntactic sugar for the product of all entries in input_shape.

input_size_0

Syntactic sugar for the product of all entries in input_shape_0.

input_size_1

Syntactic sugar for the product of all entries in input_shape_1.

output_ndim_0

Syntactic sugar for len(output_shape_0).

output_ndim_1

Syntactic sugar for len(output_shape_1).

output_shape_0

output_shape of the RandomProcess \(f_0\).

output_shape_1

output_shape of the RandomProcess \(f_1\).

output_size_0

Syntactic sugar for the product of all entries in output_shape_0.

output_size_1

Syntactic sugar for the product of all entries in output_shape_1.

Methods Summary

__call__(x0, x1)

Evaluate the (cross-)covariance function.

linop(x0[, x1])

LinearOperator representing the pairwise covariances of evaluations of \(f_0\) and \(f_1\) at the given input points.

matrix(x0[, x1])

Matrix containing the pairwise covariances of evaluations of \(f_0\) and \(f_1\) at the given input points.

Attributes Documentation

input_ndim

Syntactic sugar for len(input_shape).

input_ndim_0

Syntactic sugar for len(input_shape_0).

input_ndim_1

Syntactic sugar for len(input_shape_1).

input_shape

Shorthand for the input shape of a covariance function with input_shape_0 == input_shape_1.

Raises

ValueError – If the input shapes of the CovarianceFunction are not equal.

input_shape_0

input_shape of the RandomProcess \(f_0\). This defines the shape of a single, i.e. non-batched, first argument \(x_0\) of the CovarianceFunction.

input_shape_1

input_shape of the RandomProcess \(f_1\). This defines the shape of a single, i.e. non-batched, second argument \(x_1\) of the CovarianceFunction.

input_size

Syntactic sugar for the product of all entries in input_shape.

input_size_0

Syntactic sugar for the product of all entries in input_shape_0.

input_size_1

Syntactic sugar for the product of all entries in input_shape_1.

output_ndim_0

Syntactic sugar for len(output_shape_0).

output_ndim_1

Syntactic sugar for len(output_shape_1).

output_shape_0

output_shape of the RandomProcess \(f_0\).

This defines the first part of the shape of a single, i.e. non-batched, return value of __call__().

output_shape_1

output_shape of the RandomProcess \(f_1\).

This defines the second part of the shape of a single, i.e. non-batched, return value of __call__().

output_size_0

Syntactic sugar for the product of all entries in output_shape_0.

output_size_1

Syntactic sugar for the product of all entries in output_shape_1.

Methods Documentation

__call__(x0, x1)[source]

Evaluate the (cross-)covariance function.

The evaluation of the (cross-covariance) function is vectorized over the batch shapes of the arguments, applying standard NumPy broadcasting.

Parameters
Returns

shape= bcast_batch_shape + output_shape_0 + output_shape_1 – The (cross-)covariance function evaluated at (x0, x1). Since the function is vectorized over the batch shapes of the inputs, the output array contains the following entries:

k_x0_x1[batch_idx] = k(x0[batch_idx, ...], x1[batch_idx, ...])

where we assume that the batch shapes of x0 and x1 have been broadcast to a common shape bcast_batch_shape, and where batch_idx is an index compatible with bcast_batch_shape.

Return type

k_x0_x1

Raises

See also

matrix

Convenience function computing the full covariance matrix of evaluations at two given sets of input points.

Examples

See documentation of class CovarianceFunction.

linop(x0, x1=None)[source]

LinearOperator representing the pairwise covariances of evaluations of \(f_0\) and \(f_1\) at the given input points.

Representing the resulting covariance matrix as a matrix-free LinearOperator is often more efficient than a representation as a ndarray, both in terms of memory and computation time, particularly when using iterative methods to solve the associated linear systems.

For instance, covariance matrices induced by separable covariance functions (e.g. tensor products of covariance functions or separable multi-output kernels) can often be represented as KroneckerProducts of smaller covariance matrices and frameworks like pykeops can be used to implement efficient matrix-vector products with covariance matrices without needing to construct the entire matrix in memory.

By default, a KeOps-based matrix-free implementation will be used if available. If there is no KeOps-based implementation, the standard implementation will be used as a fallback.

Parameters
Returns

shape= (output_size_0 * N0, output_size_1 * N1) with N0 = prod(batch_shape_0) and N1 = prod(batch_shape_1)LinearOperator representing the covariance matrix corresponding to the given batches of input points. The order of the rows and columns of the covariance matrix corresponds to the order of entries obtained by flattening ndarrays with shapes output_shape_0 + batch_shape_0 and output_shape_1 + batch_shape_1 in “C-order”.

Return type

k_x0_x1

Raises
matrix(x0, x1=None)[source]

Matrix containing the pairwise covariances of evaluations of \(f_0\) and \(f_1\) at the given input points.

Parameters
Returns

shape= (output_size_0 * N0, output_size_1 * N1) with N0 = prod(batch_shape_0) and N1 = prod(batch_shape_1) – The covariance matrix corresponding to the given batches of input points. The order of the rows and columns of the covariance matrix corresponds to the order of entries obtained by flattening ndarrays with shapes output_shape_0 + batch_shape_0 and output_shape_0 + batch_shape_1 in “C-order”.

Return type

k_x0_x1

Raises