# Transition¶

class probnum.randprocs.markov.Transition(input_dim, output_dim)

Bases: ABC

Interface for Markov transitions in discrete and continuous time.

This framework describes transition probabilities

$p(\mathcal{G}_t[x(t)] \,|\, x(t))$

for some operator $$\mathcal{G}: \mathbb{R}^d \rightarrow \mathbb{R}^m$$, which are used to describe the evolution of Markov processes

$p(x(t+\Delta t) \,|\, x(t))$

both in discrete time (Markov chains) and in continuous time (Markov processes). In continuous time, Markov processes are modelled as the solution of a stochastic differential equation (SDE)

$d x(t) = f(t, x(t)) d t + d w(t)$

driven by a Wiener process $$w$$. In discrete time, Markov chain are described by a transformation

$x({t + \Delta t}) \,|\, x(t) \sim p(x({t + \Delta t}) \,|\, x(t)).$

Sometimes, these can be equivalent. For example, linear, time-invariant SDEs have a mild solution that can be written as a discrete transition. In ProbNum, we also use discrete-time transition objects to describe observation models,

$z_k \,|\, x(t_k) \sim p(z_k \,|\, x(t_k))$

for some $$k=0,...,K$$. All three building blocks are used heavily in filtering and smoothing, as well as solving ODEs.

SDE

Markov-processes in continuous time.

NonlinearGaussian

Markov-chains and general discrete-time transitions (likelihoods).

Methods Summary

 backward_realization(realization_obtained, rv) Backward-pass of a realisation of a state, according to the transition. backward_rv(rv_obtained, rv[, rv_forwarded, ...]) Backward-pass of a state, according to the transition. forward_realization(realization, t[, dt, ...]) Forward-pass of a realization of a state, according to the transition. forward_rv(rv, t[, dt, compute_gain, ...]) Forward-pass of a state, according to the transition. Transform samples from a base measure into joint backward samples from a list of random variables. Transform samples from a base measure into joint backward samples from a list of random variables. smooth_list(rv_list, locations, _diffusion_list) Apply smoothing to a list of random variables, according to the present transition.

Methods Documentation

Parameters:
abstract backward_realization(realization_obtained, rv, rv_forwarded=None, gain=None, t=None, dt=None, _diffusion=1.0, _linearise_at=None)[source]

Backward-pass of a realisation of a state, according to the transition. In other words, return a description of

$p(x(t) \,|\, {\mathcal{G}_t(x(t)) = \xi})$

for an observed realization $$\xi$$ of $${\mathcal{G}_t}(x(t))$$. For example, this function is called in a Kalman update step.

Parameters:
• realization_obtained – Observed realization $$\xi$$ as an array.

• rv – “Current” distribution $$p(x(t))$$ as a RandomVariable.

• rv_forwarded – “Forwarded” distribution (think: $$p(\mathcal{G}_t(x(t)) \,|\, x(t))$$) as a RandomVariable. Optional. If provided (in conjunction with gain), computation might be more efficient, because most backward passes require the solution of a forward pass. If rv_forwarded is not provided, forward_rv() might be called internally (depending on the object) which is skipped if rv_forwarded has been provided

• gain – Expected gain. Optional. If provided (in conjunction with rv_forwarded), some additional computations may be avoided (depending on the object).

• t – Current time point.

• dt – Increment $$\Delta t$$. Ignored for discrete-time transitions.

• _diffusion – Special diffusion of the driving stochastic process, which is used internally.

• _linearise_at – Specific point of linearisation for approximate forward passes (think: extended Kalman filtering). Used internally for iterated filtering and smoothing.

Returns:

• RandomVariable – New state, after applying the backward-pass.

• Dict – Information about the backward-pass.

abstract backward_rv(rv_obtained, rv, rv_forwarded=None, gain=None, t=None, dt=None, _diffusion=1.0, _linearise_at=None)[source]

Backward-pass of a state, according to the transition. In other words, return a description of

$p(x(t) \,|\, z_{\mathcal{G}_t}) = \int p(x(t) \,|\, z_{\mathcal{G}_t}, \mathcal{G}_t(x(t))) p(\mathcal{G}_t(x(t)) \,|\, z_{\mathcal{G}_t})) d \mathcal{G}_t(x(t)),$

for observations $$z_{\mathcal{G}_t}$$ of $${\mathcal{G}_t}(x(t))$$. For example, this function is called in a Rauch-Tung-Striebel smoothing step, which computes a Gaussian distribution

$p(x(t) \,|\, z_{\leq t+\Delta t}) = \int p(x(t) \,|\, z_{\leq t+\Delta t}, x(t+\Delta t)) p(x(t+\Delta t) \,|\, z_{\leq t+\Delta t})) d x(t+\Delta t),$

from filtering distribution $$p(x(t) \,|\, z_{\leq t})$$ and smoothing distribution $$p(x(t+\Delta t) \,|\, z_{\leq t+\Delta t})$$, where $$z_{\leq t + \Delta t}$$ contains both $$z_{\leq t}$$ and $$z_{t + \Delta t}$$.

Parameters:
• rv_obtained – “Incoming” distribution (think: $$p(x(t+\Delta t) \,|\, z_{\leq t+\Delta t})$$) as a RandomVariable.

• rv – “Current” distribution (think: $$p(x(t) \,|\, z_{\leq t})$$) as a RandomVariable.

• rv_forwarded – “Forwarded” distribution (think: $$p(x(t+\Delta t) \,|\, z_{\leq t})$$) as a RandomVariable. Optional. If provided (in conjunction with gain), computation might be more efficient, because most backward passes require the solution of a forward pass. If rv_forwarded is not provided, forward_rv() might be called internally (depending on the object) which is skipped if rv_forwarded has been provided

• gain – Expected gain from “observing states at time $$t+\Delta t$$ from time $$t$$). Optional. If provided (in conjunction with rv_forwarded), some additional computations may be avoided (depending on the object).

• t – Current time point.

• dt – Increment $$\Delta t$$. Ignored for discrete-time transitions.

• _diffusion – Special diffusion of the driving stochastic process, which is used internally.

• _linearise_at – Specific point of linearisation for approximate forward passes (think: extended Kalman filtering). Used internally for iterated filtering and smoothing.

Returns:

• RandomVariable – New state, after applying the backward-pass.

• Dict – Information about the backward-pass.

abstract forward_realization(realization, t, dt=None, compute_gain=False, _diffusion=1.0, _linearise_at=None)[source]

Forward-pass of a realization of a state, according to the transition. In other words, return a description of

$p(\mathcal{G}_t[x(t)] \,|\, x(t)=\xi),$

for some realization $$\xi$$.

Parameters:
• realization – Realization $$\xi$$ of the random variable $$x(t)$$ that describes the current state.

• t – Current time point.

• dt – Increment $$\Delta t$$. Ignored for discrete-time transitions.

• compute_gain – Flag that indicates whether the expected gain of the forward transition shall be computed. This is important if the forward-pass is computed as part of a forward-backward pass, as it is for instance the case in a Kalman update.

• _diffusion – Special diffusion of the driving stochastic process, which is used internally.

• _linearise_at – Specific point of linearisation for approximate forward passes (think: extended Kalman filtering). Used internally for iterated filtering and smoothing.

Returns:

• RandomVariable – New state, after applying the forward-pass.

• Dict – Information about the forward pass. Can for instance contain a gain key, if compute_gain was set to True (and if the transition supports this functionality).

abstract forward_rv(rv, t, dt=None, compute_gain=False, _diffusion=1.0, _linearise_at=None)[source]

Forward-pass of a state, according to the transition. In other words, return a description of

$p(\mathcal{G}_t[x(t)] \,|\, x(t)),$

or, if we take a message passing perspective,

$p(\mathcal{G}_t[x(t)] \,|\, x(t), z_{\leq t}),$

for past observations $$z_{\leq t}$$. (This perspective will be more interesting in light of backward_rv()).

Parameters:
• rv – Random variable that describes the current state.

• t – Current time point.

• dt – Increment $$\Delta t$$. Ignored for discrete-time transitions.

• compute_gain – Flag that indicates whether the expected gain of the forward transition shall be computed. This is important if the forward-pass is computed as part of a forward-backward pass, as it is for instance the case in a Kalman update.

• _diffusion – Special diffusion of the driving stochastic process, which is used internally.

• _linearise_at – Specific point of linearisation for approximate forward passes (think: extended Kalman filtering). Used internally for iterated filtering and smoothing.

Returns:

• RandomVariable – New state, after applying the forward-pass.

• Dict – Information about the forward pass. Can for instance contain a gain key, if compute_gain was set to True (and if the transition supports this functionality).

jointly_transform_base_measure_realization_list_backward(base_measure_realizations, t, rv_list, _diffusion_list, _previous_posterior=None)[source]

Transform samples from a base measure into joint backward samples from a list of random variables.

Parameters:
• base_measure_realizations (ndarray) – Base measure realizations (usually samples from a standard Normal distribution). These are transformed into joint realizations of the random variable list.

• rv_list (_RandomVariableList) – List of random variables to be jointly sampled from.

• t (float | Real | floating) – Locations of the random variables in the list. Assumed to be sorted.

• _diffusion_list (ndarray) – List of diffusions that correspond to the intervals in the locations. If locations=(t0, …, tN), then _diffusion_list=(d1, …, dN), i.e. it contains one element less.

• _previous_posterior – Previous posterior. Used for iterative posterior linearisation.

Returns:

Jointly transformed realizations.

Return type:

np.ndarray

jointly_transform_base_measure_realization_list_forward(base_measure_realizations, t, initrv, _diffusion_list, _previous_posterior=None)[source]

Transform samples from a base measure into joint backward samples from a list of random variables.

Parameters:
• base_measure_realizations (ndarray) – Base measure realizations (usually samples from a standard Normal distribution). These are transformed into joint realizations of the random variable list.

• initrv (RandomVariable) – Initial random variable.

• t (float | Real | floating) – Locations of the random variables in the list. Assumed to be sorted.

• _diffusion_list (ndarray) – List of diffusions that correspond to the intervals in the locations. If locations=(t0, …, tN), then _diffusion_list=(d1, …, dN), i.e. it contains one element less.

• _previous_posterior – Previous posterior. Used for iterative posterior linearisation.

Returns:

Jointly transformed realizations.

Return type:

np.ndarray

smooth_list(rv_list, locations, _diffusion_list, _previous_posterior=None)[source]

Apply smoothing to a list of random variables, according to the present transition.

Parameters:
• rv_list (randvars._RandomVariableList) – List of random variables to be smoothed.

• locations – Locations $$t$$ of the random variables in the time-domain. Used for continuous-time transitions.

• _diffusion_list – List of diffusions that correspond to the intervals in the locations. If locations=(t0, …, tN), then _diffusion_list=(d1, …, dN), i.e. it contains one element less.

• _previous_posterior – Specify a previous posterior to improve linearisation in approximate backward passes. Used in iterated smoothing based on posterior linearisation.

Returns:

List of smoothed random variables.

Return type:

randvars._RandomVariableList