en.wikipedia.org

Stationary process - Wikipedia

  • ️Wed May 18 2016

From Wikipedia, the free encyclopedia

In mathematics and statistics, a stationary process (also called a strict/strictly stationary process or strong/strongly stationary process) is a stochastic process whose statistical properties, such as mean and variance, do not change over time. More formally, the joint probability distribution of the process remains the same when shifted in time. This implies that the process is statistically consistent across different time periods. Because many statistical procedures in time series analysis assume stationarity, non-stationary data are frequently transformed to achieve stationarity before analysis.

A common cause of non-stationarity is a trend in the mean, which can be due to either a unit root or a deterministic trend. In the case of a unit root, stochastic shocks have permanent effects, and the process is not mean-reverting. With a deterministic trend, the process is called trend-stationary, and shocks have only transitory effects, with the variable tending towards a deterministically evolving mean. A trend-stationary process is not strictly stationary but can be made stationary by removing the trend. Similarly, processes with unit roots can be made stationary through differencing.

Another type of non-stationary process, distinct from those with trends, is a cyclostationary process, which exhibits cyclical variations over time.

Strict stationarity, as defined above, can be too restrictive for many applications. Therefore, other forms of stationarity, such as wide-sense stationarity or N-th-order stationarity, are often used. The definitions for different kinds of stationarity are not consistent among different authors (see Other terminology).

Strict-sense stationarity

[edit]

Formally, let {\displaystyle \left\{X_{t}\right\}} be a stochastic process and let {\displaystyle F_{X}(x_{t_{1}+\tau },\ldots ,x_{t_{n}+\tau })} represent the cumulative distribution function of the unconditional (i.e., with no reference to any particular starting value) joint distribution of {\displaystyle \left\{X_{t}\right\}} at times {\displaystyle t_{1}+\tau ,\ldots ,t_{n}+\tau }. Then, {\displaystyle \left\{X_{t}\right\}} is said to be strictly stationary, strongly stationary or strict-sense stationary if[1]: p. 155 

{\displaystyle F_{X}(x_{t_{1}+\tau },\ldots ,x_{t_{n}+\tau })=F_{X}(x_{t_{1}},\ldots ,x_{t_{n}})\quad {\text{for all }}\tau ,t_{1},\ldots ,t_{n}\in \mathbb {R} {\text{ and for all }}n\in \mathbb {N} _{>0}} Eq.1

Since {\displaystyle \tau } does not affect {\displaystyle F_{X}(\cdot )}, {\displaystyle F_{X}} is independent of time.

Two simulated time series processes, one stationary and the other non-stationary, are shown above. The augmented Dickey–Fuller (ADF) test statistic is reported for each process; non-stationarity cannot be rejected for the second process at a 5% significance level.

White noise is the simplest example of a stationary process.

An example of a discrete-time stationary process where the sample space is also discrete (so that the random variable may take one of N possible values) is a Bernoulli scheme. Other examples of a discrete-time stationary process with continuous sample space include some autoregressive and moving average processes which are both subsets of the autoregressive moving average model. Models with a non-trivial autoregressive component may be either stationary or non-stationary, depending on the parameter values, and important non-stationary special cases are where unit roots exist in the model.

Let {\displaystyle Y} be any scalar random variable, and define a time-series {\displaystyle \left\{X_{t}\right\}}, by

{\displaystyle X_{t}=Y\qquad {\text{ for all }}t.}

Then {\displaystyle \left\{X_{t}\right\}} is a stationary time series, for which realisations consist of a series of constant values, with a different constant value for each realisation. A law of large numbers does not apply on this case, as the limiting value of an average from a single realisation takes the random value determined by {\displaystyle Y}, rather than taking the expected value of {\displaystyle Y}.

The time average of {\displaystyle X_{t}} does not converge since the process is not ergodic.

As a further example of a stationary process for which any single realisation has an apparently noise-free structure, let {\displaystyle Y} have a uniform distribution on {\displaystyle [0,2\pi ]} and define the time series {\displaystyle \left\{X_{t}\right\}} by

{\displaystyle X_{t}=\cos(t+Y)\quad {\text{ for }}t\in \mathbb {R} .}

Then {\displaystyle \left\{X_{t}\right\}} is strictly stationary since ({\displaystyle (t+Y)} modulo {\displaystyle 2\pi }) follows the same uniform distribution as {\displaystyle Y} for any {\displaystyle t}.

Keep in mind that a weakly white noise is not necessarily strictly stationary. Let {\displaystyle \omega } be a random variable uniformly distributed in the interval {\displaystyle (0,2\pi )} and define the time series {\displaystyle \left\{z_{t}\right\}}

{\displaystyle z_{t}=\cos(t\omega )\quad (t=1,2,...)}

Then

{\displaystyle {\begin{aligned}\mathbb {E} (z_{t})&={\frac {1}{2\pi }}\int _{0}^{2\pi }\cos(t\omega )\,d\omega =0,\\\operatorname {Var} (z_{t})&={\frac {1}{2\pi }}\int _{0}^{2\pi }\cos ^{2}(t\omega )\,d\omega =1/2,\\\operatorname {Cov} (z_{t},z_{j})&={\frac {1}{2\pi }}\int _{0}^{2\pi }\cos(t\omega )\cos(j\omega )\,d\omega =0\quad \forall t\neq j.\end{aligned}}}

So {\displaystyle \{z_{t}\}} is a white noise in the weak sense (the mean and cross-covariances are zero, and the variances are all the same), however it is not strictly stationary.

Nth-order stationarity

[edit]

In Eq.1, the distribution of {\displaystyle n} samples of the stochastic process must be equal to the distribution of the samples shifted in time for all {\displaystyle n}. N-th-order stationarity is a weaker form of stationarity where this is only requested for all {\displaystyle n} up to a certain order {\displaystyle N}. A random process {\displaystyle \left\{X_{t}\right\}} is said to be N-th-order stationary if:[1]: p. 152 

{\displaystyle F_{X}(x_{t_{1}+\tau },\ldots ,x_{t_{n}+\tau })=F_{X}(x_{t_{1}},\ldots ,x_{t_{n}})\quad {\text{for all }}\tau ,t_{1},\ldots ,t_{n}\in \mathbb {R} {\text{ and for all }}n\in \{1,\ldots ,N\}} Eq.2

Weak or wide-sense stationarity

[edit]

A weaker form of stationarity commonly employed in signal processing is known as weak-sense stationarity, wide-sense stationarity (WSS), or covariance stationarity. WSS random processes only require that 1st moment (i.e. the mean) and autocovariance do not vary with respect to time and that the 2nd moment is finite for all times. Any strictly stationary process which has a finite mean and covariance is also WSS.[2]: p. 299 

So, a continuous time random process {\displaystyle \left\{X_{t}\right\}} which is WSS has the following restrictions on its mean function {\displaystyle m_{X}(t)\triangleq \operatorname {E} [X_{t}]} and autocovariance function {\displaystyle K_{XX}(t_{1},t_{2})\triangleq \operatorname {E} [(X_{t_{1}}-m_{X}(t_{1}))(X_{t_{2}}-m_{X}(t_{2}))]}:

{\displaystyle {\begin{aligned}&m_{X}(t)=m_{X}(t+\tau )&&{\text{for all }}\tau ,t\in \mathbb {R} \\&K_{XX}(t_{1},t_{2})=K_{XX}(t_{1}-t_{2},0)&&{\text{for all }}t_{1},t_{2}\in \mathbb {R} \\&\operatorname {E} [|X_{t}|^{2}]<\infty &&{\text{for all }}t\in \mathbb {R} \end{aligned}}} Eq.3

The first property implies that the mean function {\displaystyle m_{X}(t)} must be constant. The second property implies that the autocovariance function depends only on the difference between {\displaystyle t_{1}} and {\displaystyle t_{2}} and only needs to be indexed by one variable rather than two variables.[1]: p. 159  Thus, instead of writing,

{\displaystyle \,\!K_{XX}(t_{1}-t_{2},0)\,}

the notation is often abbreviated by the substitution {\displaystyle \tau =t_{1}-t_{2}}:

{\displaystyle K_{XX}(\tau )\triangleq K_{XX}(t_{1}-t_{2},0)}

This also implies that the autocorrelation depends only on {\displaystyle \tau =t_{1}-t_{2}}, that is

{\displaystyle \,\!R_{X}(t_{1},t_{2})=R_{X}(t_{1}-t_{2},0)\triangleq R_{X}(\tau ).}

The third property says that the second moments must be finite for any time {\displaystyle t}.

The main advantage of wide-sense stationarity is that it places the time-series in the context of Hilbert spaces. Let H be the Hilbert space generated by {x(t)} (that is, the closure of the set of all linear combinations of these random variables in the Hilbert space of all square-integrable random variables on the given probability space). By the positive definiteness of the autocovariance function, it follows from Bochner's theorem that there exists a positive measure {\displaystyle \mu } on the real line such that H is isomorphic to the Hilbert subspace of L2(μ) generated by {e−2πiξ⋅t}. This then gives the following Fourier-type decomposition for a continuous time stationary stochastic process: there exists a stochastic process {\displaystyle \omega _{\xi }} with orthogonal increments such that, for all {\displaystyle t}

{\displaystyle X_{t}=\int e^{-2\pi i\lambda \cdot t}\,d\omega _{\lambda },}

where the integral on the right-hand side is interpreted in a suitable (Riemann) sense. The same result holds for a discrete-time stationary process, with the spectral measure now defined on the unit circle.

When processing WSS random signals with linear, time-invariant (LTI) filters, it is helpful to think of the correlation function as a linear operator. Since it is a circulant operator (depends only on the difference between the two arguments), its eigenfunctions are the Fourier complex exponentials. Additionally, since the eigenfunctions of LTI operators are also complex exponentials, LTI processing of WSS random signals is highly tractable—all computations can be performed in the frequency domain. Thus, the WSS assumption is widely employed in signal processing algorithms.

Definition for complex stochastic process

[edit]

In the case where {\displaystyle \left\{X_{t}\right\}} is a complex stochastic process the autocovariance function is defined as {\displaystyle K_{XX}(t_{1},t_{2})=\operatorname {E} [(X_{t_{1}}-m_{X}(t_{1})){\overline {(X_{t_{2}}-m_{X}(t_{2}))}}]} and, in addition to the requirements in Eq.3, it is required that the pseudo-autocovariance function {\displaystyle J_{XX}(t_{1},t_{2})=\operatorname {E} [(X_{t_{1}}-m_{X}(t_{1}))(X_{t_{2}}-m_{X}(t_{2}))]} depends only on the time lag. In formulas, {\displaystyle \left\{X_{t}\right\}} is WSS, if

{\displaystyle {\begin{aligned}&m_{X}(t)=m_{X}(t+\tau )&&{\text{for all }}\tau ,t\in \mathbb {R} \\&K_{XX}(t_{1},t_{2})=K_{XX}(t_{1}-t_{2},0)&&{\text{for all }}t_{1},t_{2}\in \mathbb {R} \\&J_{XX}(t_{1},t_{2})=J_{XX}(t_{1}-t_{2},0)&&{\text{for all }}t_{1},t_{2}\in \mathbb {R} \\&\operatorname {E} [|X(t)|^{2}]<\infty &&{\text{for all }}t\in \mathbb {R} \end{aligned}}} Eq.4

The concept of stationarity may be extended to two stochastic processes.

Joint strict-sense stationarity

[edit]

Two stochastic processes {\displaystyle \left\{X_{t}\right\}} and {\displaystyle \left\{Y_{t}\right\}} are called jointly strict-sense stationary if their joint cumulative distribution {\displaystyle F_{XY}(x_{t_{1}},\ldots ,x_{t_{m}},y_{t_{1}^{'}},\ldots ,y_{t_{n}^{'}})} remains unchanged under time shifts, i.e. if

{\displaystyle F_{XY}(x_{t_{1}},\ldots ,x_{t_{m}},y_{t_{1}^{'}},\ldots ,y_{t_{n}^{'}})=F_{XY}(x_{t_{1}+\tau },\ldots ,x_{t_{m}+\tau },y_{t_{1}^{'}+\tau },\ldots ,y_{t_{n}^{'}+\tau })\quad {\text{for all }}\tau ,t_{1},\ldots ,t_{m},t_{1}^{'},\ldots ,t_{n}^{'}\in \mathbb {R} {\text{ and for all }}m,n\in \mathbb {N} } Eq.5

Joint (M + N)th-order stationarity

[edit]

Two random processes {\displaystyle \left\{X_{t}\right\}} and {\displaystyle \left\{Y_{t}\right\}} is said to be jointly (M + N)-th-order stationary if:[1]: p. 159 

{\displaystyle F_{XY}(x_{t_{1}},\ldots ,x_{t_{m}},y_{t_{1}^{'}},\ldots ,y_{t_{n}^{'}})=F_{XY}(x_{t_{1}+\tau },\ldots ,x_{t_{m}+\tau },y_{t_{1}^{'}+\tau },\ldots ,y_{t_{n}^{'}+\tau })\quad {\text{for all }}\tau ,t_{1},\ldots ,t_{m},t_{1}^{'},\ldots ,t_{n}^{'}\in \mathbb {R} {\text{ and for all }}m\in \{1,\ldots ,M\},n\in \{1,\ldots ,N\}} Eq.6

Joint weak or wide-sense stationarity

[edit]

Two stochastic processes {\displaystyle \left\{X_{t}\right\}} and {\displaystyle \left\{Y_{t}\right\}} are called jointly wide-sense stationary if they are both wide-sense stationary and their cross-covariance function {\displaystyle K_{XY}(t_{1},t_{2})=\operatorname {E} [(X_{t_{1}}-m_{X}(t_{1}))(Y_{t_{2}}-m_{Y}(t_{2}))]} depends only on the time difference {\displaystyle \tau =t_{1}-t_{2}}. This may be summarized as follows:

{\displaystyle {\begin{aligned}&m_{X}(t)=m_{X}(t+\tau )&&{\text{for all }}\tau ,t\in \mathbb {R} \\&m_{Y}(t)=m_{Y}(t+\tau )&&{\text{for all }}\tau ,t\in \mathbb {R} \\&K_{XX}(t_{1},t_{2})=K_{XX}(t_{1}-t_{2},0)&&{\text{for all }}t_{1},t_{2}\in \mathbb {R} \\&K_{YY}(t_{1},t_{2})=K_{YY}(t_{1}-t_{2},0)&&{\text{for all }}t_{1},t_{2}\in \mathbb {R} \\&K_{XY}(t_{1},t_{2})=K_{XY}(t_{1}-t_{2},0)&&{\text{for all }}t_{1},t_{2}\in \mathbb {R} \end{aligned}}} Eq.7

Relation between types of stationarity

[edit]

The terminology used for types of stationarity other than strict stationarity can be rather mixed. Some examples follow.

  • Priestley uses stationary up to order m if conditions similar to those given here for wide sense stationarity apply relating to moments up to order m.[3][4] Thus wide sense stationarity would be equivalent to "stationary to order 2", which is different from the definition of second-order stationarity given here.
  • Honarkhah and Caers also use the assumption of stationarity in the context of multiple-point geostatistics, where higher n-point statistics are assumed to be stationary in the spatial domain.[5]

One way to make some time series stationary is to compute the differences between consecutive observations. This is known as differencing. Differencing can help stabilize the mean of a time series by removing changes in the level of a time series, and so eliminating trends. This can also remove seasonality, if differences are taken appropriately (e.g. differencing observations 1 year apart to remove a yearly trend).

Transformations such as logarithms can help to stabilize the variance of a time series.

One of the ways for identifying non-stationary times series is the ACF plot. Sometimes, patterns will be more visible in the ACF plot than in the original time series; however, this is not always the case.[6]

Another approach to identifying non-stationarity is to look at the Laplace transform of a series, which will identify both exponential trends and sinusoidal seasonality (complex exponential trends). Related techniques from signal analysis such as the wavelet transform and Fourier transform may also be helpful.

  1. ^ a b c d e f g Park,Kun Il (2018). Fundamentals of Probability and Stochastic Processes with Applications to Communications. Springer. ISBN 978-3-319-68074-3.
  2. ^ a b Ionut Florescu (7 November 2014). Probability and Stochastic Processes. John Wiley & Sons. ISBN 978-1-118-59320-2.
  3. ^ Priestley, M. B. (1981). Spectral Analysis and Time Series. Academic Press. ISBN 0-12-564922-3.
  4. ^ Priestley, M. B. (1988). Non-linear and Non-stationary Time Series Analysis. Academic Press. ISBN 0-12-564911-8.
  5. ^ Honarkhah, M.; Caers, J. (2010). "Stochastic Simulation of Patterns Using Distance-Based Pattern Modeling". Mathematical Geosciences. 42 (5): 487–517. Bibcode:2010MatGe..42..487H. doi:10.1007/s11004-010-9276-7.
  6. ^ 8.1 Stationarity and differencing | OTexts. Retrieved 2016-05-18.