Independence for Continuous Joint Density Function

Probability Distributions I

B.R. Martin , in Statistics for Physical Science, 2012

3.3.1 Joint Probability Distributions

The multivariate joint density function $f(x_1,x_2,\dots ,x_n)$ of the n continuous random variables $x_1,x_2,\dots ,x_n$ is a single-valued non-negative real number for all real values of $x_1,x_2,\dots ,x_n$ , normalized so that

(3.17) $\underset{-\mathrm{\infty }}{\overset{+\mathrm{\infty }}{\int }}\cdots \underset{-\mathrm{\infty }}{\overset{+\mathrm{\infty }}{\int }}f(x_1,x_2,\dots ,x_n)\prod _{i=1}^n\mathrm{d}{x}_i=1,$

and the probability that $x_1$ falls between any two numbers $a_1$ and $b_1$ , $x_2$ falls between any two numbers $a_2$ and $b_2$ , $\dots$ , and $x_n$ falls between any two numbers $a_n$ and $b_n$ , simultaneously, is defined by

(3.18) $P[a_1\le x_1\le b_1;\dots ;a_n\le x_n\le b_n]\equiv \underset{a_n}{\overset{b_n}{\int }}\dots \underset{a_1}{\overset{b_1}{\int }}f(x_1,x_2,\dots ,x_n)\prod _{i=1}^n\mathrm{d}{x}_i.$

Similarly, the multivariate joint distribution function $F(x_1,x_2,\dots ,x_n)$ of the n random variables $x_1,x_2,\dots ,x_m$ is

(3.19) $F(x_1,x_2,\dots ,x_n)\equiv \underset{-\mathrm{\infty }}{\overset{x_n}{\int }}\dots \underset{-\mathrm{\infty }}{\overset{x_1}{\int }}f(t_1,t_2,\dots ,t_n)\prod _{i=1}^n\mathrm{d}{t}_i.$

For simplicity, consider the case of just two random variables x and y. These could correspond to the energy and angle of emission of a particle emitted in a nuclear scattering reaction. If an event A corresponds to the variable x being observed in the range $(x,x+\text{d}x)$ and the event B corresponds to the variable y being observed in the range $(y,y+\text{d}y)$ , then

$\begin{array}{c}P[A\cap B]=\text{probability of}x\text{being in}(x,x+\mathrm{d}x)\text{and}y\text{being in}(y,y+\mathrm{d}y)\\ =f(x,y)\mathrm{d}x\mathrm{d}y.\end{array}$

As noted in Chapter 1, the joint density function corresponds to the density of points on a scatter plot of x and y in the limit of an infinite number of points. This is illustrated in Fig. 3.3, using the data shown on the scatter plot of Fig. 1.3(b).

FIGURE 3.3. A scatter plot of 1000 events that are functions of two random variables x and y showing two infinitesimal bands dx and dy. The area of intersection of the bands is $\mathrm{d}x\mathrm{d}y$ and $f(x,y)dxdy$ is the probability of finding x in the interval $(x,x+\text{d}x)$ and y in the interval $(y,y+\text{d}y)$ .

Read full chapter

URL:

https://www.sciencedirect.com/science/article/pii/B9780123877604000032

Random Variables

Sheldon M. Ross , in Introduction to Probability Models (Twelfth Edition), 2019

2.5.3 Covariance and Variance of Sums of Random Variables

The covariance of any two random variables X and Y, denoted by $\mathrm{Cov}(X,Y)$ , is defined by

$\begin{array}{cc}\hfill \mathrm{Cov}(X,Y)& =E[(X-E[X])(Y-E[Y])]\hfill \\ \hfill & =E[XY-YE[X]-XE[Y]+E[X]E[Y]]\hfill \\ \hfill & =E[XY]-E[Y]E[X]-E[X]E[Y]+E[X]E[Y]\hfill \\ \hfill & =E[XY]-E[X]E[Y]\hfill \end{array}$

Note that if X and Y are independent, then by Proposition 2.3 it follows that $\mathrm{Cov}(X,Y)=0$ .

Let us consider now the special case where X and Y are indicator variables for whether or not the events A and B occur. That is, for events A and B, define

$X=\{\begin{array}{cc}1,\hfill & \mathrm{if}A\mathrm{occurs}\hfill \\ 0,\hfill & \mathrm{otherwise},\hfill \end{array}Y=\{\begin{array}{cc}1,\hfill & \mathrm{if}B\mathrm{occurs}\hfill \\ 0,\hfill & \mathrm{otherwise}\hfill \end{array}$

Then,

$\mathrm{Cov}(X,Y)=E[XY]-E[X]E[Y]$

and, because XY will equal 1 or 0 depending on whether or not both X and Y equal 1, we see that

$\mathrm{Cov}(X,Y)=P\{X=1,Y=1\}-P\{X=1\}P\{Y=1\}$

From this we see that

$\begin{array}{cc}\hfill \mathrm{Cov}(X,Y)>0& \iff P\{X=1,Y=1\}>P\{X=1\}P\{Y=1\}\hfill \\ \hfill & \iff \frac{P\{X=1,Y=1\}}{P\{X=1\}}>P\{Y=1\}\hfill \\ \hfill & \iff P\{Y=1|X=1\}>P\{Y=1\}\hfill \end{array}$

That is, the covariance of X and Y is positive if the outcome $X=1$ makes it more likely that $Y=1$ (which, as is easily seen by symmetry, also implies the reverse).

In general it can be shown that a positive value of $\mathrm{Cov}(X,Y)$ is an indication that Y tends to increase as X does, whereas a negative value indicates that Y tends to decrease as X increases.

Example 2.33

The joint density function of $X,Y$ is

$f(x,y)=\frac{1}{y}{e}^{-(y+x/y)},0<x,y<\infty$

(a)

Verify that the preceding is a joint density function.

(b)

Find Cov $(X,Y)$ .

Solution:  To show that $f(x,y)$ is a joint density function we need to show it is nonnegative, which is immediate, and that ${\int }_{-\infty }^{\infty }{\int }_{-\infty }^{\infty }f(x,y)dydx=1$ . We prove the latter as follows:

$\begin{array}{cc}\hfill {\int }_{-\infty }^{\infty }{\int }_{-\infty }^{\infty }f(x,y)dydx& ={\int }_0^{\infty }{\int }_0^{\infty }\frac{1}{y}{e}^{-(y+x/y)}dydx\hfill \\ \hfill & ={\int }_0^{\infty }{e}^{-y}{\int }_0^{\infty }\frac{1}{y}{e}^{-x/y}dxdy\hfill \\ \hfill & ={\int }_0^{\infty }{e}^{-y}dy\hfill \\ \hfill & =1\hfill \end{array}$

To obtain Cov $(X,Y)$ , note that the density function of Y is

$f_Y(y)=e^{-y}{\int }_0^{\infty }\frac{1}{y}{e}^{-x/y}dx=e^{-y}$

Thus, Y is an exponential random variable with parameter 1, showing (see Example 2.21) that

$E[Y]=1$

We compute $E[X]$ and $E[XY]$ as follows:

$\begin{array}{cc}\hfill E[X]& ={\int }_{-\infty }^{\infty }{\int }_{-\infty }^{\infty }xf(x,y)dydx\hfill \\ \hfill & ={\int }_0^{\infty }{e}^{-y}{\int }_0^{\infty }\frac{x}{y}{e}^{-x/y}dxdy\hfill \end{array}$

Now, ${\int }_0^{\infty }\frac{x}{y}{e}^{-x/y}dx$ is the expected value of an exponential random variable with parameter $1/y$ , and thus is equal to y. Consequently,

$E[X]={\int }_0^{\infty }y{e}^{-y}dy=1$

Also

$\begin{array}{cc}\hfill E[XY]& ={\int }_{-\infty }^{\infty }{\int }_{-\infty }^{\infty }xyf(x,y)dydx\hfill \\ \hfill & ={\int }_0^{\infty }y{e}^{-y}{\int }_0^{\infty }\frac{x}{y}{e}^{-x/y}dxdy\hfill \\ \hfill & ={\int }_0^{\infty }{y}^2{e}^{-y}dy\hfill \end{array}$

Integration by parts $(dv=e^{-y}dy,u=y^2)$ gives

$E[XY]={\int }_0^{\infty }{y}^2{e}^{-y}dy=-y^2{e}^{-y}{|}_0^{\infty }+{\int }_0^{\infty }2y{e}^{-y}dy=2E[Y]=2$

Consequently,

$\mathrm{Cov}(X,Y)=E[XY]-E[X]E[Y]=1■$

The following are important properties of covariance.

Read full chapter

URL:

https://www.sciencedirect.com/science/article/pii/B978012814346900007X

Elements of Probability

Sheldon Ross , in Simulation (Fifth Edition), 2013

2.10 Conditional Expectation and Conditional Variance

If $X$ and $Y$ are jointly discrete random variables, we define $E\left[X|Y=y\right]$ , the conditional expectation of $X$ given that $Y=y$ , by

$\begin{array}{cc}\hfill E\left[X|Y=y\right]=& {\displaystyle \sum _x}\mathit{xP}\{X=x|Y=y\}\hfill \\ \hfill =& \frac{{\displaystyle \sum _x}\mathit{xP}\{X=x\text{,}Y=y\}}{P\{Y=y\}}\hfill \end{array}$

In other words, the conditional expectation of $X$ , given that $Y=y$ , is defined like $E\left[X\right]$ as a weighted average of all the possible values of $X$ , but now with the weight given to the value $x$ being equal to the conditional probability that $X$ equals $x$ given that $Y$ equals $y$ .

Similarly, if $X$ and $Y$ are jointly continuous with joint density function $f(x\text{,}y)$ , we define the conditional expectation of $X$ , given that $Y=y$ , by

$E\left[X|Y=y\right]=\frac{\int \mathit{xf}(x\text{,}y)\mathit{dx}}{\int f(x\text{,}y)\mathit{dx}}$

Let $E\left[X|Y\right]$ denote that function of the random variable $Y$ whose value at $Y=y$ is $E\left[X|Y=y\right]$ ; and note that $E\left[X|Y\right]$ is itself a random variable. The following proposition is quite useful.

Proposition 7

(2.11) $E\left[E\left[X|Y\right]\right]=E\left[X\right]$

If $Y$ is a discrete random variable, then Equation (2.11) states that

$E\left[X\right]={\displaystyle \sum _y}E\left[X|Y=y\right]P\{Y=y\}$

whereas if $Y$ is continuous with density $g$ , then (2.11) states

$E\left[X\right]=\int E\left[X|Y=y\right]g(y)\mathit{dy}$

We now give a proof of the preceding proposition when $X$ and $Y$ are discrete:

$\begin{array}{cc}\hfill {\sum }_{y}E\left[X|Y=y\right]P\{Y=y\}=& {\displaystyle \sum _y}{\displaystyle \sum _x}\mathit{xP}\{X=x|Y=y\}P\{Y=y\}\hfill \\ \hfill =& {\displaystyle \sum _y}{\displaystyle \sum _x}\mathit{xP}\{X=x\text{,}Y=y\}\hfill \\ \hfill =& {\displaystyle \sum _x}x{\displaystyle \sum _y}P\{X=x\text{,}Y=y\}\hfill \\ \hfill =& {\displaystyle \sum _x}\mathit{xP}\{X=x\}\hfill \\ \hfill =& E\left[X\right]\hfill \end{array}$

We can also define the conditional variance of $X$ , given the value of $Y$ , as follows:

$\mathrm{Var}(X|Y)=E\left[(X-E\left[X|Y\right]{)}^2|Y\right]$

That is, $\mathrm{Var}(X|Y)$ is a function of $Y$ , which at $Y=y$ is equal to the variance of $X$ given that $Y=y$ . By the same reasoning that yields the identity $\mathrm{Var}(X)=E\left[X^2\right]-(E\left[X\right]{)}^2$ we have that

$\mathrm{Var}(X|Y)=E\left[X^2|Y\right]-(E\left[X|Y\right]{)}^2$

Taking expectations of both sides of the above equation gives

(2.12) $\begin{array}{cc}\hfill E\left[\mathrm{Var}(X|Y)\right]=& E\left[E\left[X^2|Y\right]\right]-E\left[(E\left[X|Y\right]{)}^2\right]\hfill \\ \hfill =& E\left[X^2\right]-E\left[(E\left[X|Y\right]{)}^2\right]\hfill \end{array}$

Also, because $E\left[E\left[X|Y\right]\right]=E\left[X\right]$ , we have that

(2.13) $\mathrm{Var}(E\left[X|Y\right])=E\left[(E\left[X|Y\right]{)}^2\right]-(E\left[X\right]{)}^2$

Upon adding Equations (2.12) and (2.13) we obtain the following identity, known as the conditional variance formula.

The Conditional Variance Formula

$\mathrm{Var}(X)=E\left[\mathrm{Var}(X|Y)\right]+\mathrm{Var}(E\left[X|Y\right])$

Read full chapter

URL:

https://www.sciencedirect.com/science/article/pii/B9780124158252000024

The local Gaussian partial correlation

Dag Tjøstheim , ... Bård Støve , in Statistical Modeling Using Local Gaussian Approximation, 2022

11.2 The local Gaussian partial correlation

Let $\mathit{X}={(X_1,\dots ,X_p)}^T$ be a random vector. Denote by $({\mathit{X}}_1,{\mathit{X}}_2,{\mathit{X}}_3)$ a partition of X into vectors of dimensions $p_1$ , $p_2$ , and $p_3$ , respectively, such that ${\mathit{X}}^{(1)}=({\mathit{X}}_1,{\mathit{X}}_2)={(X_1,\dots ,X_{p_1+p_2})}^T$ consists of the first $p_1+p_2$ components in X , and ${\mathit{X}}^{(2)}={\mathit{X}}_3={(X_{p_1+p_2+1},\dots ,X_{p_1+p_2+p_3})}^T$ contains the remaining $p_3$ variables, where $p=p_1+p_2+p_3$ . We assume that the mean vector μ and covariance matrix Σ of X exist and partition them correspondingly writing

(11.1) $\mathit{\mu }=\left(\begin{array}{c}\hfill {\mathit{\mu }}_1\hfill \\ \hfill {\mathit{\mu }}_2\hfill \\ \hfill {\mathit{\mu }}_3\hfill \end{array}\right)\mathrm{\text{and}}\mathbf{\Sigma }=\left(\begin{array}{ccc}\hfill {\mathbf{\Sigma }}_{11}\hfill & \hfill {\mathbf{\Sigma }}_{12}\hfill & \hfill {\mathbf{\Sigma }}_{13}\hfill \\ \hfill {\mathbf{\Sigma }}_{21}\hfill & \hfill {\mathbf{\Sigma }}_{22}\hfill & \hfill {\mathbf{\Sigma }}_{23}\hfill \\ \hfill {\mathbf{\Sigma }}_{31}\hfill & \hfill {\mathbf{\Sigma }}_{32}\hfill & \hfill {\mathbf{\Sigma }}_{33}\hfill \end{array}\right),$

where ${\mathbf{\Sigma }}_{ij}$ is the covariance matrices of $({\mathit{X}}_i,{\mathit{X}}_j)$ , $i,j=1,2,3$ . There are two main concepts of correlation when ${\mathit{X}}^{(2)}={\mathit{X}}_3$ is given, the partial and conditional correlations, which coincide in several joint distributions, among them, the Gaussian. See, for example, Baba et al. (2004) for details. We will use the partial correlation as a starting point when defining the LGPC. The partial variance–covariance matrix of ${\mathit{X}}^{(1)}=({\mathit{X}}_1,{\mathit{X}}_2)$ given ${\mathit{X}}^{(2)}={\mathit{X}}_3$ is

(11.2) ${\mathbf{\Sigma }}_{12|3}={\mathbf{\Sigma }}^{11}-{\mathbf{\Sigma }}^{12}{\left({\mathbf{\Sigma }}^{22}\right)}^{-1}{\mathbf{\Sigma }}^{21},$

where

${\mathbf{\Sigma }}^{11}=\left(\begin{array}{cc}\hfill {\mathbf{\Sigma }}_{11}\hfill & \hfill {\mathbf{\Sigma }}_{12}\hfill \\ \hfill {\mathbf{\Sigma }}_{21}\hfill & \hfill {\mathbf{\Sigma }}_{22}\hfill \end{array}\right),{\mathbf{\Sigma }}^{12}=\left(\begin{array}{c}\hfill {\mathbf{\Sigma }}_{13}\hfill \\ \hfill {\mathbf{\Sigma }}_{23}\hfill \end{array}\right),{\mathbf{\Sigma }}^{21}=\left(\begin{array}{cc}\hfill {\mathbf{\Sigma }}_{31}\hfill & \hfill {\mathbf{\Sigma }}_{32}\hfill \end{array}\right),\mathrm{\text{and}}{\mathbf{\Sigma }}^{22}={\mathbf{\Sigma }}_{33},$

and ${\mathbf{\Sigma }}_{12|3}$ is the covariance matrix in the conditional (Gaussian) distribution of ${\mathit{X}}^{(1)}$ given ${\mathit{X}}_3$ if X is jointly normal. The partial correlation matrix between ${\mathit{X}}_1$ and ${\mathit{X}}_2$ given ${\mathit{X}}_3$ is naturally defined as

(11.3) ${\mathit{R}}_{12|3}={\mathit{D}}^{-1/2}{\mathbf{\Sigma }}_{12|3}{\mathit{D}}^{-1/2},$

where $\mathit{D}=\mathrm{\text{diag}}({\mathbf{\Sigma }}_{12|3})$ . We identify in the same way the partial correlation matrix (11.3) with the correlation matrix in the conditional (Gaussian) distribution of ${\mathit{X}}^{(1)}$ given ${\mathit{X}}^{(2)}$ if X is jointly normal. Eqs. (11.2) and (11.3) will serve as the starting point for our definition of the local partial correlation.

11.2.1 Definition

We further assume that the components of X are continuous with joint density function $f_{\mathit{X}}$ , and we again set up a local likelihood framework for obtaining local estimates of the parameters in the multivariate normal distribution. Given a point x , we approximate $f_{\mathit{X}}$ in a neighborhood of x by a multivariate Gaussian density

(11.4) $\psi (\mathit{x},\mathit{v})=\frac{1}{{(2\pi )}^{p/2}|\mathbf{\Sigma }(\mathit{x}){|}^{1/2}}\exp \{-\frac{1}{2}{(\mathit{v}-\mathit{\mu }(\mathit{x}))}^T{\mathbf{\Sigma }}^{-1}(\mathit{x})(\mathit{v}-\mathit{\mu }(\mathit{x}))\},$

where $\mathit{x}={(x_1\dots ,x_p)}^T$ , $\mathit{\mu }(\mathit{x})=\{{\mu }_j(\mathit{x})\}$ , and $\mathbf{\Sigma }(\mathit{x})=\{{\sigma }_{jk}(\mathit{x})\}$ for $j,k=1,\dots ,p$ . Moving to another point y , there is another (generally, different) Gaussian approximation $\psi (\mathit{y},\mathit{v})$ . In this way, we approximate $f_{\mathit{X}}$ by a family of multivariate Gaussian densities defined by a set of smooth parameter functions $\{\mathit{\mu }(\mathit{x}),\mathbf{\Sigma }(\mathit{x})\}$ , and if $f_{\mathit{X}}$ is itself a Gaussian density, then the parameter functions collapse to constants corresponding to the true parameter values, and $\psi (\mathit{x})\equiv f_{\mathit{X}}(\mathit{x})$ . Hjort and Jones (1996) provide a general framework for estimating such parameter functions non-parametrically from a given data set using a local likelihood procedure, and the basic idea in the following treatment is replacing the components in the partial covariance matrix (11.2) by their locally estimated counterparts to obtain a local measure of conditional dependence.

In this chapter, we use the same transformation technique as that introduced already in Chapter 4.7 and subsequently used in the following chapters. This improves and simplifies the estimation of the LGPC. It is shown that the estimation of the local parameter functions $\{\mathit{\mu }(\mathit{x}),\mathbf{\Sigma }(\mathit{x})\}$ becomes easier by transforming each $X_j$ to a standard normal variable $Z_j={\mathrm{\Phi }}^{-1}(U_j)$ , where $U_j$ is a uniform variable, $U_j=F_j(X_j)$ with $F_j$ being the cumulative distribution function of $X_j$ . Define the random vector Z by this transformation of $\mathit{X}=({\mathit{X}}^{(1)},{\mathit{X}}^{(2)})=({\mathit{X}}_1,{\mathit{X}}_2,{\mathit{X}}_3)={(X_1,X_2,\dots ,X_p)}^T$ to marginal standard normality:

(11.5) $\mathit{Z}={({\mathrm{\Phi }}^{-1}(F_1(X_1)),{\mathrm{\Phi }}^{-1}(F_2(X_2)),\dots ,{\mathrm{\Phi }}^{-1}(F_p(X_p)))}^T.$

The transformation enables us to simplify the local Gaussian approximation (11.4) by writing the density $f_{\mathit{Z}}$ of Z at the point $\mathit{v}=\mathit{z}$ as

(11.6) $f_{\mathit{Z}}(\mathit{z})=\psi (\mathit{z},R(\mathit{z}))=\frac{1}{|2\pi \mathit{R}(\mathit{z}){|}^{1/2}}\exp \{-\frac{1}{2}{\mathit{z}}^T{\mathit{R}}^{-1}(\mathit{z})\mathit{z}\},$

where, as in Otneim and Tjøstheim (2017, 2018) and in Eqs. (9.6) and (9.7) in Chapter 9, in a further simplified approximation, we have fixed local means and standard deviations ${\mu }_j(\mathit{z})\equiv 0$ and ${\sigma }_j^2(\mathit{z})\equiv 1$ , $j=1,\dots ,p$ , and where $\mathit{R}(\mathit{z})=\{{\rho }_{jk}(\mathit{z})\}$ is the local correlation matrix.

In practice, we do not know $F_j$ , but we can instead use the empirical distribution function

${\hat{F}}_j(x)=\frac{1}{n}\sum _{i=1}^{n}I(X_{ji}⩽x),$

where I is the indicator function, n is the number of observations, and $X_{ji}$ is the ith observation of $X_j$ , and where $1/n$ can be replaced by $1/(n+1)$ for small or moderate sample sizes. This results in pseudo-standard normal variables ${\hat{Z}}_j={\mathrm{\Phi }}^{-1}({\hat{F}}_j(X_j))$ .

In the following, we will not always distinguish between $Z_j$ and ${\hat{Z}}_j$ . In fact, by using the technique of proof of Theorem 4.5 under the regularity conditions of that theorem, the error made by estimating $\mathit{R}(\mathit{z})$ using the empirically transformed variables ${\hat{Z}}_j$ instead of $Z_j$ is smaller in the limit than the estimation error made when estimating the local correlations themselves.

In this chapter, we refer to X and its probability density function $f_{\mathit{X}}$ as being on the x -scale and to Z and its probability density function $f_{\mathit{Z}}$ as being on the z -scale. For further discussion of the simplified z -approximation, we refer to Chapters 9.2.2 and 9.7.

Denote by $({\mathit{Z}}^{(1)},{\mathit{Z}}^{(2)})=({\mathit{Z}}_1,{\mathit{Z}}_2,{\mathit{Z}}_3)$ the partitioning of Z corresponding to the partitioning $({\mathit{X}}^{(1)},{\mathit{X}}^{(2)})=({\mathit{X}}_1,{\mathit{X}}_2,{\mathit{X}}_3)$ of X . A natural definition of the local partial covariance matrix of ${\mathit{Z}}^{(1)}|{\mathit{Z}}^{(2)}$ is the local version of Eq. (11.2):

(11.7) ${\mathbf{\Sigma }}_{12|3}(\mathit{z})={\mathit{R}}^{11}({\mathit{z}}^{(1)})-{\mathit{R}}^{12}(\mathit{z}){({\mathit{R}}^{22}({\mathit{z}}^{(2)}))}^{-1}{\mathit{R}}^{21}(\mathit{z}).$

If $p_1=p_2=1$ , then ${\mathbf{\Sigma }}_{12|3}(\mathit{z})$ is a $2\times 2$ matrix, and we define the local Gaussian partial correlation $\alpha (\mathit{z})$ between the two variables in ${\mathit{Z}}^{(1)}=(Z_1,Z_2)$ given ${\mathit{Z}}^{(2)}={\mathit{Z}}_3$ in accordance with the ordinary (global) partial correlation provided by Eq. (11.3):

(11.8) $\alpha (\mathit{z})=R_{12|3}(\mathit{z})=\frac{{\{{\mathbf{\Sigma }}_{12|3}(\mathit{z})\}}_{12}}{{\{{\mathbf{\Sigma }}_{12|3}(\mathit{z})\}}_{11}^{1/2}{\{{\mathbf{\Sigma }}_{12|3}(\mathit{z})\}}_{22}^{1/2}},$

which, when ${\mathit{Z}}^{(2)}=Z_3$ is scalar, reduces to

(11.9) $\alpha (\mathit{z})={\rho }_{12|3}(z_1,z_2|z_3)=\frac{{\rho }_{12}(z_1,z_2)-{\rho }_{13}(z_1,z_3){\rho }_{23}(z_2,z_3)}{\sqrt{1-{\rho }_{13}^2(z_1,z_3)}\sqrt{1-{\rho }_{23}^2(z_2,z_3)}}.$

This is easily recognizable as a local version of the standard global partial correlation coefficient. It is of course possible to introduce an LGPC $\alpha (\mathit{x})$ directly on the x -scale, but that representation is in many ways harder to handle both computationally and asymptotically. For a multivariate Gaussian distribution, we have that ${\alpha }_{\mathit{X}}(\mathit{x})={\alpha }_{\mathit{Z}}(\mathit{z})=\alpha$ . In the remainder of this chapter, we write mainly in terms of the z -representation using the LGPC $\alpha (\mathit{z})={\alpha }_{\mathit{Z}}(\mathit{z})$ , but when we write the local partial correlation between $X_1$ and $X_2$ given ${\mathit{X}}_3={\mathit{x}}_3$ at the point $(x_1,x_2,{\mathit{x}}_3)$ , this is simply $\alpha (\mathit{z})$ with inserted $z_j={\mathrm{\Phi }}^{-1}(F_j(x_j))$ , $j=1,\dots ,p$ .

In the more general case where $p_1>1$ and/or $p_2>1$ , ${\mathbf{\Sigma }}_{12|3}(\mathit{z})$ is a $(p_1+p_2)\times (p_1+p_2)$ matrix that describes the non-linear conditional dependence among the variables in ${\mathit{Z}}^{(1)}=({\mathit{Z}}_1,{\mathit{Z}}_2)$ given ${\mathit{Z}}^{(2)}={\mathit{Z}}_3$ (or, alternatively, ${\mathit{X}}^{(1)}=({\mathit{X}}_1,{\mathit{X}}_2)$ given ${\mathit{X}}^{(2)}={\mathit{X}}_3$ ), which in particular can be used to analyze the conditional dependence between two sets of random variables of dimensions $p_1$ and $p_2$ , respectively, given a third set of variables of dimension $p_3$ . This case poses no conceptual challenges to our approach, but it requires a rather large investment in new notation and does not lead to simple formulas like Eq. (11.9). We will therefore, for the most part, focus on the local Gaussian partial correlation between two stochastic scalar variables in the remainder of this chapter. A complete description of the general multivariate case can be found in the online supplement of Otneim and Tjøstheim (2021) and, somewhat briefly, in Section 11.8.

Read full chapter

URL:

https://www.sciencedirect.com/science/article/pii/B9780128158616000183

RANDOM VARIABLE THEORY

Daniel T. Gillespie , in Markov Processes, 1992

Random Variable Transformation (RVT) Theorem

If the n random variables X ₁, …, X_n have the joint density function P(x ₁,…,x_n ), and if the m random variables Y ₁, …, Y_m are defined by Y_i =f_i (X ₁,…,X_n ) [i=1 to m], then the joint density function Q(y ₁,…,y_m ) of Y ₁, …, Y_m is given by

(1.6-4) $Q\left(y_1,\dots ,y_m\right)={\displaystyle {\int }_{–\infty }^{\infty }\text{d}{x}_1\dots {\displaystyle {\int }_{–\infty }^{\infty }\text{d}{x}_{n}P\left(x_1,\dots ,x_n\right){\displaystyle \prod _{i=1}^m\delta \left(y_i–f_i\left(x_1,\dots ,x_n\right)\right)}}}.$

A useful special case of the RVT theorem is the following result:

Read full chapter

URL:

https://www.sciencedirect.com/science/article/pii/B9780080918372500065

SPECIAL RANDOM VARIABLES

Sheldon M. Ross , in Introduction to Probability and Statistics for Engineers and Scientists (Fourth Edition), 2009

EXAMPLE 5.4e

The random vector X, Y is said to have a uniform distribution over the two-dimensional region R if its joint density function is constant for points in R, and is 0 for points outside of R. That is, if

$f(x,y)=\{\begin{array}{cc}c& \text{if}(x,y)\in \text{R}\\ 0& \text{if}\text{otherwise}\end{array}$

Because

$\begin{array}{l}1={\int }_{R}f(x,y)dxdy\\ ={\int }_{R}cdxdy\\ =c\times \text{Area}\text{of}R\end{array}$

it follows that

$c=\frac{1}{\text{Area}\text{of}R}$

For any region A ⊂ R,

$\begin{array}{l}P\{(X,Y)\in A\}=\int {\int }_{(x,y)\in A}f(x,y)dxdy\\ =\int {\int }_{(x,y)\in A}cdxdy\\ =\frac{\text{Area}\text{of}A}{\text{Area}\text{of}R}\end{array}$

Suppose now that X, Y is uniformly distributed over the following rectangular region R:

Its joint density function is

$f(x,y)=\{\begin{array}{cc}c& \text{if}\text{0}\le x\le a,0\le y\le b\\ 0& \text{otherwise}\end{array}$

where $c=\frac{1}{\text{Area}\text{of}\text{rectangle}}=\frac{1}{ab}$ . In this case, X and Y are independent uniform random variables. To show this, note that for 0 ≤ x ≤ a, 0 ≤ y ≤ b

(5.4.5) $P\{X\le x,Y\le y\}=c{\int }_0^x{\int }_0^{y}dydx=\frac{xy}{ab}$

First letting y = b, and then letting x = a, in the preceding shows that

(5.4.6) $P\{X\le x\}=\frac{x}{a},P\{Y\le y\}=\frac{y}{b}$

Thus, from Equations 5.4.5 and 5.4.6 we can conclude that X and Y are independent, with X being uniform on (0, a) and Y being uniform on (0, b).

Read full chapter

URL:

https://www.sciencedirect.com/science/article/pii/B9780123704832000102

Multivariate stochastic orders

Félix Belzunce , ... Julio Mulero , in An Introduction to Stochastic Orders, 2016

3.5 The multivariate likelihood ratio order

In a similar way to the univariate case, it is possible to check the multivariate stochastic and hazard rate orders in terms of a property of the joint density functions. This property leads to the definition of the multivariate likelihood ratio order, which is a generalization of the univariate likelihood ratio order. In this section, the results are given in the continuous case but they can be restated for the general case. The main intention of the multivariate likelihood ratio order is to provide a sufficient condition for the multivariate hazard rate order.

Definition 3.5.1

Given two continuous random vectors X = (X ₁,…,X _n ) and Y = (Y ₁,…,Y _n ) with joint density functions f and g, respectively, we say that X is smaller than Y in the multivariate likelihood ratio order, denoted by X ≤_lr Y, if

$f(\mathbf{x})g(\mathbf{y})\le f(\mathbf{x}\wedge \mathbf{y})g(\mathbf{x}\vee \mathbf{y}),\text{for all}\mathbf{x},\mathbf{y}\in {\mathbb{R}}^n.$

Clearly, this is a generalization of the likelihood ratio order in the univariate case. However, the multivariate likelihood ratio order is not necessarily reflexive. When a random vector X satisfies X ≤_lr X, we have the MTP₂ property introduced in Section 1.3.

The following result provides a set of sufficient conditions for the multivariate likelihood ratio order.

Theorem 3.5.2

Let X = (X ₁,…,X _n ) and Y = (Y ₁,…,Y _n ) be two continuous random vectors with joint density functions f and g, respectively. If X or Y or both are MTP₂, and

(3.13) $\begin{array}{l}\hfill f(\mathbf{y})g(\mathbf{x})\le f(\mathbf{x})g(\mathbf{y}),\text{for all}\mathbf{x},\mathbf{y}\in {\mathbb{R}}^n,\text{such that}\mathbf{x}\le \mathbf{y},\end{array}$

then

$\mathbf{X}{\le }_{\text{lr}}\mathbf{Y}.$

Proof

Let us assume that X is MTP₂ (in the other case the proof is similar), then we have the following inequalities:

$f(\mathbf{x})g(\mathbf{y})\le \frac{f(\mathbf{x}\wedge \mathbf{y})f(\mathbf{x}\vee \mathbf{y})}{f(\mathbf{y})}g(\mathbf{y})\le f(\mathbf{x}\wedge \mathbf{y})g(\mathbf{x}\vee \mathbf{y}),$

for all $\mathbf{x},\mathbf{y}\in {\mathbb{R}}^n$ , such that x ≤y, where the first inequality follows from the MTP₂ property and the second one from (3.13). Therefore, X ≤_lr Y.

The multivariate likelihood ratio order is preserved under conditioning on sublattices, as we see next. Recall that a subset $A\subseteq {\mathbb{R}}^n$ is called a sublattice if x,y ∈ A implies x ∧y ∈ A and x ∨y ∈ A. This result will be used to show the relationship among the multivariate likelihood ratio order and the multivariate dynamic hazard rate order. The proof is obvious from the definition.

Theorem 3.5.3

Let X = (X ₁,…,X _n ) and Y = (Y ₁,…,Y _n ) be two continuous random vectors. If X ≤_lr Y, then

$[\mathbf{X}|\mathbf{X}\subseteq A]{\le }_{\text{lr}}[\mathbf{Y}|\mathbf{Y}\subseteq A],\text{for all sublattice}A\subseteq {\mathbb{R}}^n.$

In particular, from the previous theorem, the multivariate likelihood ratio order is preserved under marginalization. This result is useful because, in some cases, it is easier to provide a result in the multivariate case rather that in the univariate case.

Theorem 3.5.4

Let X = (X ₁,…,X _n ) and Y = (Y ₁,…,Y _n ) be two continuous random vectors. If X ≤_lr Y, then

${\mathbf{X}}_I{\le }_{\text{lr}}{\mathbf{Y}}_I,\text{for all}I\subseteq \{1,\dots ,n\}.$

Next, it is showed that the multivariate likelihood ratio order is stronger than the multivariate hazard rate order.

Theorem 3.5.5

Let X = (X ₁,…,X _n ) and Y = (Y ₁,…,Y _n ) be two continuous random vectors. If X ≤_lr Y, then

$\mathbf{X}{\le }_{\text{hr}}\mathbf{Y}.$

Proof

Let us check the conditions for the definition of the multivariate dynamic hazard rate order. In particular, denoting by η and λ the multivariate dynamic hazard rates of X and Y, respectively, let us see if

${\eta }_k(t|h_t)\ge {\lambda }_k(t|h_t^{\prime }),\text{for all}t\ge 0,$

where

$h_t=\{{\mathbf{X}}_{I\cup J}={\mathbf{x}}_{I\cup J},{\mathbf{X}}_{\overline{I\cup J}}>t\mathbf{e}\},$

and

$h_t^{\prime }=\{{\mathbf{Y}}_I={\mathbf{y}}_I,{\mathbf{Y}}_{\overline{I}}>t\mathbf{e}\},$

whenever $I\cap J=\varnothing$ , 0 ≤x _I ≤y _I ≤ t e, 0 ≤x _J ≤ t e, and for all $k\in \overline{I\cup J}$ .

The result will follow by proving that, as we shall see later,

(3.14) $\begin{array}{l}\hfill \left[\left.{\mathbf{X}}_{\overline{I\cup J}}\right|h_t\right]{\le }_{\text{lr}}\left[\left.{\mathbf{Y}}_{\overline{I\cup J}}\right|h_t^{\prime }\right].\end{array}$

Condition (3.14) will follow if

(3.15) $\begin{array}{l}\hfill \left[\left.{\mathbf{X}}_{\overline{I\cup J}}\right|{\mathbf{X}}_{I\cup J}={\mathbf{x}}_{I\cup J}\right]{\le }_{\text{lr}}\left[\left.{\mathbf{Y}}_{\overline{I\cup J}}\right|{\mathbf{Y}}_I={\mathbf{y}}_I,{\mathbf{Y}}_J>t\mathbf{e}\right],\end{array}$

holds.

Let us see that (3.15) follows if X ≤_lr Y holds. Denoting by f and g the joint density functions of $({\mathbf{X}}_I,{\mathbf{X}}_J,{\mathbf{X}}_{I\cup J})$ and $({\mathbf{Y}}_I,{\mathbf{Y}}_J,{\mathbf{Y}}_{I\cup J})$ , respectively, and by f _I,J and g _I,J the joint densities of X _I,J and Y _I,J , respectively, we see that the joint density function of $\left[\left.{\mathbf{X}}_{\overline{I\cup J}}\right|{\mathbf{X}}_{I\cup J}={\mathbf{x}}_{I\cup J}\right]$ is given by

$f_{\overline{I\cup J}}({\mathbf{x}}_{\overline{I\cup J}})=\frac{f({\mathbf{x}}_I,{\mathbf{x}}_J,{\mathbf{x}}_{\overline{I\cup J}})}{f_{I,J}({\mathbf{x}}_I,{\mathbf{x}}_J)},$

and the joint density function of $\left[\left.{\mathbf{Y}}_{\overline{I\cup J}}\right|{\mathbf{Y}}_I={\mathbf{y}}_I,{\mathbf{Y}}_J>t\mathbf{e}\right]$ is given by

$g_{\overline{I\cup J}}({\mathbf{x}}_{\overline{I\cup J}})=\frac{{\int }_{{\mathbf{y}}_J>t\mathbf{e}}g({\mathbf{y}}_I,{\mathbf{y}}_J,{\mathbf{x}}_{\overline{I\cup J}})d{\mathbf{y}}_J}{{\int }_{{\mathbf{y}}_J>t\mathbf{e}}{g}_{I,J}({\mathbf{y}}_I,{\mathbf{y}}_J)d{\mathbf{y}}_J}.$

Given y _J > t e, we see that x _J < t e <y _J and, analogously, x _I ≤y _I . Since X ≤_lr Y, given ${\mathbf{y}}_{\overline{I\cup J}}$ , we see that

$\begin{array}{ll}\hfill & f({\mathbf{x}}_I,{\mathbf{x}}_J,{\mathbf{x}}_{\overline{I\cup J}})g({\mathbf{y}}_I,{\mathbf{y}}_J,{\mathbf{y}}_{\overline{I\cup J}})\hfill \\ \hfill & \le f({\mathbf{x}}_I,{\mathbf{x}}_J,{\mathbf{x}}_{\overline{I\cup J}}\wedge {\mathbf{y}}_{\overline{I\cup J}})g({\mathbf{y}}_I,{\mathbf{y}}_J,{\mathbf{x}}_{\overline{I\cup J}}\vee {\mathbf{y}}_{\overline{I\cup J}}),\hfill \end{array}$

which, upon integration, yields

$f_{\overline{I\cup J}}({\mathbf{x}}_{\overline{I\cup J}})g_{\overline{I\cup J}}({\mathbf{y}}_{\overline{I\cup J}})\le f_{\overline{I\cup J}}({\mathbf{x}}_{\overline{I\cup J}}\wedge {\mathbf{y}}_{\overline{I\cup J}})g_{\overline{I\cup J}}({\mathbf{x}}_{\overline{I\cup J}}\vee {\mathbf{y}}_{\overline{I\cup J}}).$

Therefore, from previous inequality, we see that (3.15) holds. Now, from Theorem 3.5.3, we see that (3.14) also holds. In particular, $\left[\left.X_k\right|h_t\right]{\le }_{\text{lr}}\left[\left.Y_k\right|h_t^{\prime }\right]$ holds, for all $k\in \overline{I\cup J}$ . Now, since (2.25), we see that the hazard rates of $\left[\left.X_k\right|h_t\right]$ and $\left[\left.Y_k\right|h_t^{\prime }\right]$ are ordered and, consequently, we see that

${\eta }_k(t|h_t)\ge {\lambda }_k(t|h_t^{\prime }).$

Finally, a result for the multivariate likelihood ratio order among random vectors with conditionally independent components is provided. In particular, we consider the same background as that in Theorem 3.3.8.

Theorem 3.5.6

Assume that

(i)

X _i (θ) =_st Y _i (θ), for all θ and for all i = 1,…,n,

(ii)

X _i (θ) ≤_lr Y _j (θ′), for all θ ≤ θ′ and for all 1 ≤ i ≤ j ≤ n, and

(iii)

θ ₁ ≤_lr θ ₂.

Then,

$(X_1,\dots ,X_n){\le }_{\text{lr}}(Y_1,\dots ,Y_n).$

Proof

Let f _i (x|θ) be the density function of X _i (θ). From condition (ii), we see that

$\prod _{i=1}^n{f}_i(x_i|{\theta }_i){\text{is TP}}_2\text{in}(x_1,\dots ,x_n,{\theta }_1,\dots ,{\theta }_n).$

Furthermore, condition (iii) is equivalent to the fact that h _i (θ ₁,…,θ _n ) is TP₂ in (θ ₁,…,θ _n ,i) ∈ S ×{1,2}. Due to these facts and from Theorem 1.2.3, we see that $\int f(x_1,\dots ,x_n|{\theta }_1,\dots ,{\theta }_n)d{H}_i(\mathit{\theta })$ is TP₂ in (x ₁,…,x _n ,i), and we get the result.

Read full chapter

URL:

https://www.sciencedirect.com/science/article/pii/B978012803768300003X

Convex Functions, Partial Orderings, and Statistical Applications

In Mathematics in Science and Engineering, 1992

13.6 Some Properties of Log-Concave Density Functions

Log-concave density functions which satisfy (13.19) play an important role in statistics and probability. In the following we observe some known facts concerning this class of densities.

13.24 Fact

Let X ₁ …, X _n be i.i.d. univariate random variables with a common density function h(x). If h(x) is a log-concave function of x for x ∈ ℝ, then the joint density function of (X ₁, …, X _n ) is a log-concave function of x for x ∈ ℝ ⁿ .

13.25 Fact

If f(x) = g(T(x)) where g: ℝ → [0, ∞) is decreasing and T(x) is a convex function of x for x ∈ ℝ ⁿ , then f is a log-concave function of x for x ∈ ℝ ⁿ .

The following theorem, due to Prékopa (1971) and Brascamp and Lieb (1975), shows that the integral of a log-concave function is log-concave:

13.26 Theorem

Let f(x, y): ℝ ^n+m → [0, ∞) be a log-concave function of (x, y) for x ∈ ℝ ⁿ and y ∈ ℝ ^m . Then the function g: ℝ ⁿ → [0, ∞) given by

(13.27) $g(\text{x})={\displaystyle \underset{{ℝ}^m}{\int }f(\text{x},y)dy}$

is log-concave.

Proof

We adopt the proof in Brascamp and Lieb (1975). First note that it suffices to prove the theorem for m = n = 1 because the general case follows by Fubini's theorem and induction. Let x ₁, x ₂ be two points in ℝ such that g(x ₁)g(x ₂) ≠ 0. For convenience we may assume that

$\underset{y}{\sup ╡}f(\text{x},y)=\underset{y}{\sup ╡}f({\text{x}}^{\prime },y);$

because otherwise we can replace f(x, y) by e ^bx f(x, y) for suitably chosen b and the problem remains unchanged. For each fixed λ > 0, denote

$\begin{array}{c}{C}_1(\lambda )=\{(\text{x},y):f(\text{x},y)\ge \lambda \}\subset {ℝ}^2,\\ C_2(\text{x},\lambda )=\{y:f(\text{x},y)\ge \lambda \}\subset ℝ.\end{array}$

Then, by log-concavity of f, C ₁(λ) is convex and C ₂(x, λ) is an interval. (For the convexity of C ₁(λ), see Fact 13.28). Letting v(x, y) = ∫ _{C 2(x,λ)} dy be the Lebesgue measure of the set C ₂(x, λ), we have, by Theorem 13.18,

$\begin{array}{ccc}v(\alpha {\text{x}}_1+(1-\alpha ){\text{x}}_2,\lambda )\ge \alpha v({\text{x}}_1,\lambda )+(1-\alpha )v({\text{x}}_2,\lambda )& \text{for}\text{all}& \alpha \in [0,1]\end{array}.$

Since g(x) can be expressed as g(x) = ∫₀ ^∞ v(x, λ) dλ, we have

$g(\alpha {\text{x}}_1)+(1-\alpha ){\text{x}}_2)\ge \alpha g({\text{x}}_1)+(1-\alpha )g({\text{x}}_2)\ge {(g({\text{x}}_1))}^{\alpha }{(g({\text{x}}_2))}^{1-\alpha }$

for all α ∈ [0, 1], where the second inequality follows from the arithmetic mean-geometric mean inequality.

A simple application of Theorem 13.26 is (Brascamp and Lieb, 1975; see also Barlow and Proschan, 1981, p. 104):

13.27 Corollary

The convolution of two log-concave density functions in ℝ ⁿ is log-concave.

Proof

Let f ₁, f ₂ be log-concave density functions. Then f ₁(x − y)f ₂(y) is jointly log-concave in (x, y) ∈ ℝ²ⁿ . Thus by Theorem 13.26

$g(\text{x})={\displaystyle \underset{{ℝ}^n}{\int }{f}_1(\text{x}-y)f_2(y)dy}$

is log-concave.

A density function f is said to be unimodal if the set

(13.28) $D_{\lambda }=\{\text{x}:\text{x}\in {ℝ}^n,f(\text{x})\ge \lambda \}$

is a convex set in ℝ ⁿ for all λ > 0. The following facts show how log-concavity and unimodality are related.

13.28 Fact

If f: ℝ ⁿ → [0, ∞) is a probability density function, then log-concavity off implies unimodality of f.

Proof

Let x ₁, x ₂ ∈ ℝ ⁿ be in D _λ. Then for every α ∈ [0, 1], we have, by (13.20),

(13.29) $\begin{array}{c}\log ╡f(\alpha {\text{x}}_1+(1-\alpha ){\text{x}}_2)\ge \alpha \log ╡f({\text{x}}_1)+(1-\alpha )\log ╡f({\text{x}}_2)\\ \ge \alpha \log ╡\lambda +(1-\alpha )\log ╡\lambda \ge \log ╡\lambda .\end{array}$

Thus αx ₁ + (1 − α)x ₂ is also in D _λ.

A function f is said to be Schur-concave if y ≻ x implies f(x) ≥ f(y) for all x, y ∈ ℝ ⁿ (see Definition 12.23). It is known that all Schur-concave functions are permutation invariant (see Theorem 12.24). Furthermore, it is known that

13.29 Fact

If f: ℝ ⁿ → [0, ∞) is a permutation-invariant and log-concave function of x ∈ ℝ ⁿ , then it is a Schur-concave function of x ∈ ℝ ⁿ .

Proof

Assume y ≻ x and, without loss of generality, it may be assumed that x, y are of the form

$\begin{array}{cc}\text{x}=({\text{x}}_1,{\text{x}}_2,{\text{x}}_3,\dots ,{\text{x}}_n),& y=(y_1,y_2,{\text{x}}_3,\dots ,{\text{x}}_n),\end{array}$

where y ₂ < x ₂ ≤ x ₁ < y ₁ and x ₁ + x ₂ = y ₁ + y ₂. Let y ^* = (y ₂, y ₁, x ₃, …, x _n ). Then there exists an α ∈ (0, 1) such that x = αy + (1 − α)y ^*. Thus by the permutation-invariance and log-concavity properties of f, we have

$\begin{array}{c}\log ╡f(\text{x})=\log ╡f(\alpha y+(1-\alpha )y^{*})\ge \alpha \log ╡f(y)+(1-\alpha )\log ╡f(y^{\ast })\\ =\log ╡f(y).\hfill \end{array}$

Read full chapter

URL:

https://www.sciencedirect.com/science/article/pii/S0076539208628258

Preliminaries

Félix Belzunce , ... Julio Mulero , in An Introduction to Stochastic Orders, 2016

1.3.5 Parametric families of multivariate distributions

In this section, some known multivariate distributions are recalled. These models will be compared in Chapter 3 in some multivariate stochastic orders.

It is worth mentioning that the mean of a random vector X = (X ₁,…,X _n ) is given by

$E[\mathbf{X}]=\left(\begin{array}{l}\hfill E[X_1]\hfill \\ \hfill ⋮\hfill \\ \hfill E[X_n]\hfill \end{array}\right),$

and the covariance matrix of X is

$\mathrm{Cov}(\mathbf{X})=E[(\mathbf{X}-E[\mathbf{X}]){(\mathbf{X}-E[\mathbf{X}])}^{\mathrm{T}}].$

Next, we consider some multivariate distributions. First, the definition of the multivariate normal distribution is recalled.

Definition 1.3.2

Given a random vector X = (X ₁,…,X _n ), it is said that X follows a multivariate normal distribution with mean vector $\mathbf{\mu }\in {\mathbb{R}}^n$ and covariance matrix $\mathbf{\Sigma }\in {\mathbb{R}}^n\times {\mathbb{R}}^n$ , denoted by X ∼ N _n (μ,Σ ), if its joint density function is given by

$f(\mathbf{x})=\frac{1}{{(2\pi )}^{\frac{n}{2}}|\mathbf{\Sigma }{|}^{\frac{1}{2}}}exp\left\{-\frac{1}{2}{(\mathbf{x}-\mathbf{\mu })}^{\mathrm{T}}{\mathbf{\Sigma }}^{-1}(\mathbf{x}-\mathbf{\mu })\right\},\text{for all}\mathbf{x}\in {\mathbb{R}}^n.$

The marginal distribution functions follow univariate normal models. Furthermore, the copula of a N _n (μ,Σ) is the same to that of N _n (0,P) where P is the correlation matrix obtained through the covariance matrix Σ. In this sense, all multivariate normal distributions with the same dimension and correlation matrix have the same (Gaussian) copula.

Another well-known multivariate family is the elliptically contoured model. Let us consider the formal definition.

Definition 1.3.3

Given a random vector X = (X ₁,…,X _n ), it is said that X follows an elliptically contoured distribution, denoted by E _n (μ,Σ,g), if its joint density function is given by

(1.17) $\begin{array}{l}\hfill f(\mathbf{x})=\frac{1}{\sqrt{|\mathbf{\Sigma }|}}g({(\mathbf{x}-\mathbf{\mu })}^t{\mathbf{\Sigma }}^{-1}(\mathbf{x}-\mathbf{\mu })),\text{for all}\mathbf{x}\in {\mathbb{R}}^n,\end{array}$

where μ is the median vector (which is also the mean vector if the latter exists), Σ is a symmetric positive definite matrix which is proportional to the covariance matrix, if the latter exists, and $g:{\mathbb{R}}_{+}\mapsto {\mathbb{R}}_{+}$ such that $\int g(x)\mathrm{d}x<+\infty$ .

A particular case of elliptically contoured distributions is the case of a multivariate normal taking

$g(x)=\frac{1}{{(2\pi )}^n}exp\left\{-\frac{1}{2}x\right\}.$

Some other particular cases are described in Ref. [52].

Read full chapter

URL:

https://www.sciencedirect.com/science/article/pii/B9780128037683000016

GENERAL FEATURES OF A MARKOV PROCESS

Daniel T. Gillespie , in Markov Processes, 1992

2.1 The Markov State Density Function

We consider a time-evolving or 'dynamical' system whose possible states can be represented by points on the real axis, and we let

(2.1-1) $X\mathrm{(}t\mathrm{)}\equiv \text{the state point, or state, of the system at time t.}$

We shall assume that the value of X at some initial time t ₀ is fixed,

(2.1-2) $X\mathrm{(}{t}_0\mathrm{)}\mathrm{=}{x}_0,$

but that X(t) for any t> t ₀ can be predicted only probabilistically; more specifically, we assume that X(t) for any given t> t ₀ is a random variable, as defined in Section 1.2 Since it makes sense to inquire about the state of the system at successive instants t ₁, t ₂, …, t_n , where t ₀ < t ₁ < t ₂ < … < t_n , then we can ascribe to the corresponding n random variables X(t ₁), X(t ₂), …, X(t_n ) a joint density function P _n ⁽¹⁾, which is defined as follows:

(2.1-3) $\begin{array}{l}{P}_n^{\mathrm{(}1\mathrm{)}}\mathrm{(}{x}_n,t_n;x_{n-1},t_{n-1};\text{…};x_1,t_1|x_0,t_0\mathrm{)}\text{d}{x}_n\text{d}{x}_{n-1}\text{…}\text{d}{x}_1\\ \equiv \text{P}\text{r}\text{o}\text{b}\{X\mathrm{(}{t}_i\mathrm{)}\in [x_i,x_i\mathrm{+}\text{d}{x}_i\mathrm{)}\text{for}i\mathrm{=}1,2,\text{…},n\\ \text{given that}X\mathrm{(}{t}_0\mathrm{)}\mathrm{=}{x}_0,\text{with}{t}_0\le t_1\le \text{…}\le t_n\}.\end{array}$

If all these assumptions are satisfied, then we say that X(t) is a stochastic process.

It is evident that a stochastic process X(t) has infinitely many joint density functions P_n ⁽¹⁾, corresponding to n = 1, 2, … . And associated with each of these joint density functions is a plethora of subordinate density functions; for example,

$P_{n-j}^{\mathrm{(}j\mathrm{+}1\mathrm{)}}\mathrm{(}{x}_n,t_n;\text{…};x_{j\mathrm{+}1},t_{j\mathrm{+}1}|x_j,t_J;\text{…};x_1,t_1;x_0,t_0\mathrm{)},$

is defined to be the joint density function of the n – j random variables $X\left(t_j+1\right),\dots ,X\left(t_n\right)$ given the j + 1 conditions $X\left(t_0\right)=x_0,X\left(t_1\right)=x_1,\dots ,X\left(t_j\right)=x_j.$ Notice that the subscript on the density function P_k ^(j) refers to the number of (x,t) pairs to the left of the 'given' bar, while the superscript refers to the number of (x,t) pairs to the right of the 'given' bar; thus, P_k ^(j) is a k-variate joint density function with j conditionings.

It is always possible to calculate the function P _n-1 ⁽¹⁾ from the function P_n ⁽¹⁾ by simply integrating the latter over any one of the variables x ₁, …, x_n . However, it is not in general possible to deduce the function P _n+1 ⁽¹⁾ from the function P_n ⁽¹⁾. This 'open-ended' nature of the density functions for a general stochastic process usually makes any substantive analysis extremely difficult. But we shall be concerned here with only a very restricted subclass of stochastic processes, namely those that have the 'past-forgetting' property that, for all j ≥ 2 and t _i-1 ≤ t_i ,

(2.1-4) $\begin{array}{l}{P}_1^{\mathrm{(}j\mathrm{)}}\mathrm{(}{x}_j,t_j|x_{j-1},t_{j-1};\text{…};x_1,t_1;x_0,t_0\mathrm{)}\\ \mathrm{=}{P}_1^{\mathrm{(}1\mathrm{)}}\mathrm{(}{x}_j,t_j|x_{j-1},t_{j-1}\mathrm{)}\equiv P\mathrm{(}{x}_j,t_j|x_{j-1},t_{J^{-1}}\mathrm{)}.\end{array}$

This is called the Markov property, and it says that only the most recent conditioning matters: Given that X(t') = x', then our ability to predict X(t) for any t > t' will not be enhanced by a knowledge of any values of the process earlier than t'. Any stochastic process X(t) that has this past-forgetting property is called a Markovian stochastic process, or more simply, a Markov process. In what follows it may always be assumed, unless explicitly stated otherwise, that the stochastic process X(t) under consideration is a Markov process.

The Markov property (2.1-4) breaks the open-endedness of the hierarchy of joint state density functions in a dramatic way. For the joint density function P ₂ ⁽¹⁾ we have

$\begin{array}{l}{P}_2^{\mathrm{(}1\mathrm{)}}\mathrm{(}{x}_2,t_2;x_1,t_1|x_0,t_0\mathrm{)}\\ \mathrm{=}{P}_1^{\mathrm{(}1\mathrm{)}}\mathrm{(}{x}_1,t_1|x_0,t_0\mathrm{)}{P}_1^{\mathrm{(}2\mathrm{)}}\mathrm{(}{x}_2,t_2|x_1,t_1;x_0,t_0\mathrm{)}\left[\text{by}(1.5–9\text{d})\right]\end{array}$

$\mathrm{=}{P}_1^{\mathrm{(}1\mathrm{)}}\mathrm{(}{x}_1,t_1|x_0,t_0\mathrm{)}{P}_1^{\mathrm{(}1\mathrm{)}}\mathrm{(}{x}_2,t_2|x_1,t_1\mathrm{)}.\left[\text{by}(2.1–4)\right]$

Hence, writing P ₁ ⁽¹⁾ ≡ P in accordance with the notation suggested in Eq. (2.1-4), we have

(2.1-5) $P_2^{\mathrm{(}1\mathrm{)}}\mathrm{(}{x}_2,t_2;x_1,t_1|x_0,t_0\mathrm{)}\mathrm{=}P\mathrm{(}{x}_2,t_2|x_1,t_1\mathrm{)}P\mathrm{(}{x}_1,t_1|x_0,t_0\mathrm{)}.$

The same kind of reasoning shows that

$P_3^{\mathrm{(}1\mathrm{)}}\mathrm{(}{x}_3,t_3;x_2,t_2;x_1,t_1|x_0,t_0\mathrm{)}\mathrm{=}P\mathrm{(}{x}_{\mathrm{3}},t_3|x_2,t_2\mathrm{)}P\mathrm{(}{x}_2,t_2|x_1,t_1\mathrm{)}P\mathrm{(}{x}_1,t_1|x_0,t_0\mathrm{)},$

and more generally, for any set of times $t_n\ge t_{n-1}\ge \text{…}\ge t_0,$

(2.1-6) $P_n^{\mathrm{(}1\mathrm{)}}\mathrm{(}{x}_n,t_n;\text{…};x_1,t_1|x_0,t_0\mathrm{)}={\displaystyle \prod _{i=1}^{n}P\mathrm{(}{x}_i,t_i|x_{i-1},t_{i-1}\mathrm{)}}$

So for a Markov process, every conditioned state density function $P_n^{\mathrm{(}1\mathrm{)}}$ can be written solely in terms of the particular conditioned state density function $P_1^{\mathrm{(}1\mathrm{)}}\equiv P$ The function $P_1^{\mathrm{(}1\mathrm{)}}\equiv P$ thus becomes the principle focus of our study, and we shall henceforth refer to it as the Markov state density function. For future reference, the formal definition of the Markov state density function is [cf.Eq. (2.1-3)]

(2.1-7) $\begin{array}{l}P\mathrm{(}{x}_2,t_2|x_1,t_1\mathrm{)}\text{d}{x}_2\\ \equiv \text{Prob}\{X\mathrm{(}{t}_2\mathrm{)}\in \mathrm{[}{x}_2,x_2\mathrm{+}\text{d}{x}_2\mathrm{)},\text{given}X\mathrm{(}{t}_1\mathrm{)}\mathrm{=}{x}_1,\text{with}{t}_2\ge t_1\}.\end{array}$

Read full chapter

URL:

https://www.sciencedirect.com/science/article/pii/B9780080918372500077

ungcyruch.blogspot.com

Source: https://www.sciencedirect.com/topics/mathematics/joint-density-function

Share this post