CovarianceFunction¶

class probnum.randprocs.covfuncs.CovarianceFunction(*, input_shape_0=None, input_shape_1=None, input_shape=None, output_shape_0=(), output_shape_1=())¶

Bases: ABC

(Cross-)covariance function.

A cross-covariance function

\begin{equation} k \colon \mathbb{X}_0 \times \mathbb{X}_1 \to \mathbb{R}^{d^\text{out}_0 \times d^\text{out}_1}, (x_0, x_1) \mapsto \operatorname{Cov}[f_0(x_0), f_1(x_1)] \end{equation}

is a function of two arguments \(x_0 \in \mathbb{X}_0\) and \(x_1 \in \ \mathbb{X}_1\) (often \(\mathbb{X}_i \subset \mathbb{R}^{d^\text{in}_i}\)), whose output corresponds to the covariance (matrix) between two evaluations \(f_0(x_0) \in \mathbb{R}^{d^\text{out}_0}\) and \(f_1(x_1) \in \ \mathbb{R}^{d^\text{out}_1}\) of two (vector-valued) RandomProcesses \(f_0\) and \(f_1\). If \(f_0 = f_1\), then a cross-covariance function is also referred to as a covariance function or a kernel, in which case it must be symmetric and positive (semi-)definite.

Parameters:

input_shape_0 (Optional[ShapeLike]) – input_shape of the RandomProcess \(f_0\). This defines the shape of the CovarianceFunction‘s first input \(x_0\).
input_shape_1 (Optional[ShapeLike]) – input_shape of the RandomProcess \(f_1\). This defines the shape of the CovarianceFunction‘s second input \(x_1\).
input_shape (Optional[ShapeLike]) – Convenience argument, which can be used to set input_shape_0 == input_shape_1 == input_shape. If input_shape is specified, then input_shape_{0,1} must be set to None.
output_shape_0 (ShapeLike) – output_shape of the RandomProcess \(f_0\).
output_shape_1 (ShapeLike) – output_shape of the RandomProcess \(f_1\).

Examples

>>> from probnum.randprocs.covfuncs import Linear
>>> D = 3
>>> k = Linear(input_shape=D)
>>> k.input_shape_0
(3,)
>>> k.input_shape_1
(3,)
>>> k.output_shape_0
()
>>> k.output_shape_1
()

Generate some input data.

>>> N = 4
>>> xs = np.linspace(0, 1, N * D).reshape(N, D)
>>> xs.shape
(4, 3)
>>> xs
array([[0.        , 0.09090909, 0.18181818],
       [0.27272727, 0.36363636, 0.45454545],
       [0.54545455, 0.63636364, 0.72727273],
       [0.81818182, 0.90909091, 1.        ]])

We can compute covariance matrices of multiple evaluations like so.

>>> k.matrix(xs)
array([[0.04132231, 0.11570248, 0.19008264, 0.26446281],
       [0.11570248, 0.41322314, 0.7107438 , 1.00826446],
       [0.19008264, 0.7107438 , 1.23140496, 1.75206612],
       [0.26446281, 1.00826446, 1.75206612, 2.49586777]])

The __call__() method is vectorized over the “batch shapes” of the inputs, applying standard NumPy broadcasting.

>>> k(xs[:, None], xs[None, :])  # same as `.matrix`
array([[0.04132231, 0.11570248, 0.19008264, 0.26446281],
       [0.11570248, 0.41322314, 0.7107438 , 1.00826446],
       [0.19008264, 0.7107438 , 1.23140496, 1.75206612],
       [0.26446281, 1.00826446, 1.75206612, 2.49586777]])

No broadcasting is applied if both inputs have the same shape. For instance, one can efficiently compute the marginal variance of a set of data points via

>>> k(xs, xs)
array([0.04132231, 0.41322314, 1.23140496, 2.49586777])
>>> k(xs, None)  # x1 = None is an efficient way to set x1 == x0
array([0.04132231, 0.41322314, 1.23140496, 2.49586777])

CovarianceFunctions support basic arithmetic operations. For example, we can model independent measurement noise as follows:

>>> from probnum.randprocs.covfuncs import WhiteNoise
>>> k_noise = k + 0.1 * WhiteNoise(input_shape=D)
>>> k_noise.matrix(xs)
array([[0.14132231, 0.11570248, 0.19008264, 0.26446281],
       [0.11570248, 0.51322314, 0.7107438 , 1.00826446],
       [0.19008264, 0.7107438 , 1.33140496, 1.75206612],
       [0.26446281, 1.00826446, 1.75206612, 2.59586777]])

Attributes Summary

`input_ndim`	Syntactic sugar for `len(input_shape)`.
`input_ndim_0`	Syntactic sugar for `len(input_shape_0)`.
`input_ndim_1`	Syntactic sugar for `len(input_shape_1)`.
`input_shape`	Shorthand for the input shape of a covariance function with `input_shape_0` `==` `input_shape_1`.
`input_shape_0`	`input_shape` of the `RandomProcess` \(f_0\).
`input_shape_1`	`input_shape` of the `RandomProcess` \(f_1\).
`input_size`	Syntactic sugar for the product of all entries in `input_shape`.
`input_size_0`	Syntactic sugar for the product of all entries in `input_shape_0`.
`input_size_1`	Syntactic sugar for the product of all entries in `input_shape_1`.
`output_ndim_0`	Syntactic sugar for `len(output_shape_0)`.
`output_ndim_1`	Syntactic sugar for `len(output_shape_1)`.
`output_shape_0`	`output_shape` of the `RandomProcess` \(f_0\).
`output_shape_1`	`output_shape` of the `RandomProcess` \(f_1\).
`output_size_0`	Syntactic sugar for the product of all entries in `output_shape_0`.
`output_size_1`	Syntactic sugar for the product of all entries in `output_shape_1`.

Methods Summary

`__call__`(x0, x1)	Evaluate the (cross-)covariance function.
`linop`(x0[, x1])	`LinearOperator` representing the pairwise covariances of evaluations of \(f_0\) and \(f_1\) at the given input points.
`matrix`(x0[, x1])	Matrix containing the pairwise covariances of evaluations of \(f_0\) and \(f_1\) at the given input points.

Attributes Documentation

input_ndim¶: Syntactic sugar for len(input_shape).

input_ndim_0¶: Syntactic sugar for len(input_shape_0).

input_ndim_1¶: Syntactic sugar for len(input_shape_1).

input_shape¶

Shorthand for the input shape of a covariance function with input_shape_0 == input_shape_1.

Raises:: ValueError – If the input shapes of the CovarianceFunction are not equal.

input_shape_0¶: input_shape of the RandomProcess \(f_0\). This defines the shape of a single, i.e. non-batched, first argument \(x_0\) of the CovarianceFunction.

input_shape_1¶: input_shape of the RandomProcess \(f_1\). This defines the shape of a single, i.e. non-batched, second argument \(x_1\) of the CovarianceFunction.

input_size¶: Syntactic sugar for the product of all entries in input_shape.

input_size_0¶: Syntactic sugar for the product of all entries in input_shape_0.

input_size_1¶: Syntactic sugar for the product of all entries in input_shape_1.

output_ndim_0¶: Syntactic sugar for len(output_shape_0).

output_ndim_1¶: Syntactic sugar for len(output_shape_1).

output_shape_0¶

output_shape of the RandomProcess \(f_0\).

This defines the first part of the shape of a single, i.e. non-batched, return value of __call__().

output_shape_1¶

output_shape of the RandomProcess \(f_1\).

This defines the second part of the shape of a single, i.e. non-batched, return value of __call__().

output_size_0¶: Syntactic sugar for the product of all entries in output_shape_0.

output_size_1¶: Syntactic sugar for the product of all entries in output_shape_1.

Methods Documentation

__call__(x0, x1)[source]¶

Evaluate the (cross-)covariance function.

The evaluation of the (cross-covariance) function is vectorized over the batch shapes of the arguments, applying standard NumPy broadcasting.

Parameters:

x0 (ArrayLike) – shape= batch_shape_0 + input_shape_0 – (Batch of) input(s) for the first argument of the CovarianceFunction.
x1 (ArrayLike | None) – shape= batch_shape_1 + input_shape_1 – (Batch of) input(s) for the second argument of the CovarianceFunction. Can also be set to None, in which case the function will behave as if x1 = x0 (but it is implemented more efficiently).

Returns:

shape= bcast_batch_shape + output_shape_0 + output_shape_1 – The (cross-)covariance function evaluated at (x0, x1). Since the function is vectorized over the batch shapes of the inputs, the output array contains the following entries:

k_x0_x1[batch_idx] = k(x0[batch_idx, ...], x1[batch_idx, ...])

where we assume that the batch shapes of x0 and x1 have been broadcast to a common shape bcast_batch_shape, and where batch_idx is an index compatible with bcast_batch_shape.

Return type:

k_x0_x1

Raises:

ValueError – If the shape of \(x_0\) is not of the form batch_shape_0 + input_shape_0.
ValueError – If the shape of \(x_1\) is not of the form batch_shape_1 + input_shape_1.
ValueError – If the inputs can not be broadcast to a common shape.