CovarianceFunction¶
- class probnum.randprocs.covfuncs.CovarianceFunction(*, input_shape_0=None, input_shape_1=None, input_shape=None, output_shape_0=(), output_shape_1=())¶
Bases:
ABC
(Cross-)covariance function.
A cross-covariance function
\begin{equation} k \colon \mathbb{X}_0 \times \mathbb{X}_1 \to \mathbb{R}^{d^\text{out}_0 \times d^\text{out}_1}, (x_0, x_1) \mapsto \operatorname{Cov}[f_0(x_0), f_1(x_1)] \end{equation}is a function of two arguments \(x_0 \in \mathbb{X}_0\) and \(x_1 \in \ \mathbb{X}_1\) (often \(\mathbb{X}_i \subset \mathbb{R}^{d^\text{in}_i}\)), whose output corresponds to the covariance (matrix) between two evaluations \(f_0(x_0) \in \mathbb{R}^{d^\text{out}_0}\) and \(f_1(x_1) \in \ \mathbb{R}^{d^\text{out}_1}\) of two (vector-valued)
RandomProcess
es \(f_0\) and \(f_1\). If \(f_0 = f_1\), then a cross-covariance function is also referred to as a covariance function or a kernel, in which case it must be symmetric and positive (semi-)definite.- Parameters
input_shape_0 (Optional[ShapeLike]) –
input_shape
of theRandomProcess
\(f_0\). This defines the shape of theCovarianceFunction
‘s first input \(x_0\).input_shape_1 (Optional[ShapeLike]) –
input_shape
of theRandomProcess
\(f_1\). This defines the shape of theCovarianceFunction
‘s second input \(x_1\).input_shape (Optional[ShapeLike]) – Convenience argument, which can be used to set
input_shape_0 == input_shape_1 == input_shape
. Ifinput_shape
is specified, theninput_shape_{0,1}
must be set toNone
.output_shape_0 (ShapeLike) –
output_shape
of theRandomProcess
\(f_0\).output_shape_1 (ShapeLike) –
output_shape
of theRandomProcess
\(f_1\).
Examples
>>> from probnum.randprocs.covfuncs import Linear >>> D = 3 >>> k = Linear(input_shape=D) >>> k.input_shape_0 (3,) >>> k.input_shape_1 (3,) >>> k.output_shape_0 () >>> k.output_shape_1 ()
Generate some input data.
>>> N = 4 >>> xs = np.linspace(0, 1, N * D).reshape(N, D) >>> xs.shape (4, 3) >>> xs array([[0. , 0.09090909, 0.18181818], [0.27272727, 0.36363636, 0.45454545], [0.54545455, 0.63636364, 0.72727273], [0.81818182, 0.90909091, 1. ]])
We can compute covariance matrices of multiple evaluations like so.
>>> k.matrix(xs) array([[0.04132231, 0.11570248, 0.19008264, 0.26446281], [0.11570248, 0.41322314, 0.7107438 , 1.00826446], [0.19008264, 0.7107438 , 1.23140496, 1.75206612], [0.26446281, 1.00826446, 1.75206612, 2.49586777]])
The
__call__()
method is vectorized over the “batch shapes” of the inputs, applying standard NumPy broadcasting.>>> k(xs[:, None], xs[None, :]) # same as `.matrix` array([[0.04132231, 0.11570248, 0.19008264, 0.26446281], [0.11570248, 0.41322314, 0.7107438 , 1.00826446], [0.19008264, 0.7107438 , 1.23140496, 1.75206612], [0.26446281, 1.00826446, 1.75206612, 2.49586777]])
No broadcasting is applied if both inputs have the same shape. For instance, one can efficiently compute the marginal variance of a set of data points via
>>> k(xs, xs) array([0.04132231, 0.41322314, 1.23140496, 2.49586777]) >>> k(xs, None) # x1 = None is an efficient way to set x1 == x0 array([0.04132231, 0.41322314, 1.23140496, 2.49586777])
CovarianceFunction
s support basic arithmetic operations. For example, we can model independent measurement noise as follows:>>> from probnum.randprocs.covfuncs import WhiteNoise >>> k_noise = k + 0.1 * WhiteNoise(input_shape=D) >>> k_noise.matrix(xs) array([[0.14132231, 0.11570248, 0.19008264, 0.26446281], [0.11570248, 0.51322314, 0.7107438 , 1.00826446], [0.19008264, 0.7107438 , 1.33140496, 1.75206612], [0.26446281, 1.00826446, 1.75206612, 2.59586777]])
Attributes Summary
Syntactic sugar for
len(
input_shape
)
.Syntactic sugar for
len(
input_shape_0
)
.Syntactic sugar for
len(
input_shape_1
)
.Shorthand for the input shape of a covariance function with
input_shape_0
==
input_shape_1
.input_shape
of theRandomProcess
\(f_0\).input_shape
of theRandomProcess
\(f_1\).Syntactic sugar for the product of all entries in
input_shape
.Syntactic sugar for the product of all entries in
input_shape_0
.Syntactic sugar for the product of all entries in
input_shape_1
.Syntactic sugar for
len(
output_shape_0
)
.Syntactic sugar for
len(
output_shape_1
)
.output_shape
of theRandomProcess
\(f_0\).output_shape
of theRandomProcess
\(f_1\).Syntactic sugar for the product of all entries in
output_shape_0
.Syntactic sugar for the product of all entries in
output_shape_1
.Methods Summary
__call__
(x0, x1)Evaluate the (cross-)covariance function.
linop
(x0[, x1])LinearOperator
representing the pairwise covariances of evaluations of \(f_0\) and \(f_1\) at the given input points.matrix
(x0[, x1])Matrix containing the pairwise covariances of evaluations of \(f_0\) and \(f_1\) at the given input points.
Attributes Documentation
- input_ndim¶
Syntactic sugar for
len(
input_shape
)
.
- input_ndim_0¶
Syntactic sugar for
len(
input_shape_0
)
.
- input_ndim_1¶
Syntactic sugar for
len(
input_shape_1
)
.
- input_shape¶
Shorthand for the input shape of a covariance function with
input_shape_0
==
input_shape_1
.- Raises
ValueError – If the input shapes of the
CovarianceFunction
are not equal.
- input_shape_0¶
input_shape
of theRandomProcess
\(f_0\). This defines the shape of a single, i.e. non-batched, first argument \(x_0\) of theCovarianceFunction
.
- input_shape_1¶
input_shape
of theRandomProcess
\(f_1\). This defines the shape of a single, i.e. non-batched, second argument \(x_1\) of theCovarianceFunction
.
- input_size¶
Syntactic sugar for the product of all entries in
input_shape
.
- input_size_0¶
Syntactic sugar for the product of all entries in
input_shape_0
.
- input_size_1¶
Syntactic sugar for the product of all entries in
input_shape_1
.
- output_ndim_0¶
Syntactic sugar for
len(
output_shape_0
)
.
- output_ndim_1¶
Syntactic sugar for
len(
output_shape_1
)
.
- output_shape_0¶
output_shape
of theRandomProcess
\(f_0\).This defines the first part of the shape of a single, i.e. non-batched, return value of
__call__()
.
- output_shape_1¶
output_shape
of theRandomProcess
\(f_1\).This defines the second part of the shape of a single, i.e. non-batched, return value of
__call__()
.
- output_size_0¶
Syntactic sugar for the product of all entries in
output_shape_0
.
- output_size_1¶
Syntactic sugar for the product of all entries in
output_shape_1
.
Methods Documentation
- __call__(x0, x1)[source]¶
Evaluate the (cross-)covariance function.
The evaluation of the (cross-covariance) function is vectorized over the batch shapes of the arguments, applying standard NumPy broadcasting.
- Parameters
x0 (ArrayLike) – shape=
batch_shape_0 +
input_shape_0
– (Batch of) input(s) for the first argument of theCovarianceFunction
.x1 (Optional[ArrayLike]) – shape=
batch_shape_1 +
input_shape_1
– (Batch of) input(s) for the second argument of theCovarianceFunction
. Can also be set toNone
, in which case the function will behave as ifx1 = x0
(but it is implemented more efficiently).
- Returns
shape=
bcast_batch_shape +
output_shape_0
+
output_shape_1
– The (cross-)covariance function evaluated at(x0, x1)
. Since the function is vectorized over the batch shapes of the inputs, the output array contains the following entries:k_x0_x1[batch_idx] = k(x0[batch_idx, ...], x1[batch_idx, ...])
where we assume that the batch shapes of
x0
andx1
have been broadcast to a common shapebcast_batch_shape
, and wherebatch_idx
is an index compatible withbcast_batch_shape
.- Return type
k_x0_x1
- Raises
ValueError – If the shape of \(x_0\) is not of the form
batch_shape_0 +
input_shape_0
.ValueError – If the shape of \(x_1\) is not of the form
batch_shape_1 +
input_shape_1
.ValueError – If the inputs can not be broadcast to a common shape.
See also
matrix
Convenience function computing the full covariance matrix of evaluations at two given sets of input points.
Examples
See documentation of class
CovarianceFunction
.
- linop(x0, x1=None)[source]¶
LinearOperator
representing the pairwise covariances of evaluations of \(f_0\) and \(f_1\) at the given input points.Representing the resulting covariance matrix as a matrix-free
LinearOperator
is often more efficient than a representation as andarray
, both in terms of memory and computation time, particularly when using iterative methods to solve the associated linear systems.For instance, covariance matrices induced by separable covariance functions (e.g. tensor products of covariance functions or separable multi-output kernels) can often be represented as
KroneckerProduct
s of smaller covariance matrices and frameworks likepykeops
can be used to implement efficient matrix-vector products with covariance matrices without needing to construct the entire matrix in memory.By default, a KeOps-based matrix-free implementation will be used if available. If there is no KeOps-based implementation, the standard implementation will be used as a fallback.
- Parameters
x0 (ArrayLike) – shape=
batch_shape_0 +
input_shape_0
– (Batch of) input(s) for the first argument of theCovarianceFunction
.x1 (Optional[ArrayLike]) – shape=
batch_shape_1 +
input_shape_1
– (Batch of) input(s) for the second argument of theCovarianceFunction
. Can also be set toNone
, in which case the function will behave as ifx1 == x0
(potentially using a more efficient implementation for this particular case).
- Returns
shape=
(
output_size_0
* N0,
output_size_1
* N1)
withN0 = prod(batch_shape_0)
andN1 = prod(batch_shape_1)
–LinearOperator
representing the covariance matrix corresponding to the given batches of input points. The order of the rows and columns of the covariance matrix corresponds to the order of entries obtained by flatteningndarray
s with shapesoutput_shape_0
+ batch_shape_0
andoutput_shape_1
+ batch_shape_1
in “C-order”.- Return type
k_x0_x1
- Raises
ValueError – If the shape of \(x_0\) is not of the form
batch_shape_0 +
input_shape_0
.ValueError – If the shape of \(x_1\) is not of the form
batch_shape_1 +
input_shape_1
.
- matrix(x0, x1=None)[source]¶
Matrix containing the pairwise covariances of evaluations of \(f_0\) and \(f_1\) at the given input points.
- Parameters
x0 (ArrayLike) – shape=
batch_shape_0 +
input_shape_0
– (Batch of) input(s) for the first argument of theCovarianceFunction
.x1 (Optional[ArrayLike]) – shape=
batch_shape_1 +
input_shape_1
– (Batch of) input(s) for the second argument of theCovarianceFunction
. Can also be set toNone
, in which case the function will behave as ifx1 == x0
(potentially using a more efficient implementation for this particular case).
- Returns
shape=
(
output_size_0
* N0,
output_size_1
* N1)
withN0 = prod(batch_shape_0)
andN1 = prod(batch_shape_1)
– The covariance matrix corresponding to the given batches of input points. The order of the rows and columns of the covariance matrix corresponds to the order of entries obtained by flatteningndarray
s with shapesoutput_shape_0
+ batch_shape_0
andoutput_shape_0
+ batch_shape_1
in “C-order”.- Return type
k_x0_x1
- Raises
ValueError – If the shape of \(x_0\) is not of the form
batch_shape_0 +
input_shape_0
.ValueError – If the shape of \(x_1\) is not of the form
batch_shape_1 +
input_shape_1
.