1.10 Important Inequalities

<< 1.9 Some Special Expectations.md | 2.1 Distributions of Two Random Variables.md >>

This page will show overview of some famous inequalities involving expectations.

Theorem 1.10.1: Existence of lower order moments

Let:

• $X$ random variable
• $m\in \mathbb{P}$
• $k\in \mathbb{P},k\le m$

If $E(X^m)$ exists, then $E(X^k)$ exists

If a higher-order moment exists, then all lower-order moments must also exist

Link to original

Theorem 1.10.2: Markov’s inequality

Let:

• $X$ random variable
• $u(X)\to \mathbb{P}$
• $E[u(X)]$ exists $\forall c\in \mathbb{P}$

Then

$$P[u(X)\ge c]\le \frac{E[u(X)]}{c}$$

Link to original

Theorem 1.10.3: Chebyshev’s inequality

Let:

• $X$ Random Variable
• ${\sigma }^2\in \mathbb{R}$ Variance of $X$
• $\mu =E(X)$ (by Existence of Lower Order Moments, ${\sigma }^2\in \mathbb{R}$ implies that $E(X)$ exists)

Then, for every $k>0$

$$P(|X-\mu |\ge k\sigma )\le \frac{1}{k^2}$$

Or equivalently,

$$P(|X-\mu |<k\sigma )\ge 1-\frac{1}{k^2}$$ Link to original

Definition 1.10.1: Convex function

Let $\varphi$ : Function defined on interval $(a,b)$, where $-\infty \le a<b\le \infty$

If $\varphi [\gamma x+(1-\gamma )y]\le \gamma \varphi (x)+(1-\gamma )\varphi (y),\forall x,y\in (a,b),0<\gamma <1$

Then we say $\varphi$ is a convex function

Note

If the inequality is strict (i.e., $<$ instead of $\le$), then we say $\varphi$ is strictly convex.

Link to original

Theorem 1.10.4

If $\varphi$ is differentiable on $(a,b)$

Then

• $\varphi$ is convex $⟺{\varphi }^{\prime }(x)\le {\varphi }^{\prime }(y),\forall a<x<y<b$
• $\varphi$ is strictly convex $⟺{\varphi }^{\prime }(x)<{\varphi }^{\prime }(y),\forall a<x<y<b$

If $\varphi$ is twice differentiable on $(a,b)$

Then

• $\varphi$ is convex $⟺{\varphi }^{\prime \prime }(x)\ge 0,\forall a<x<b$
• $\varphi$ is strictly convex $⟺{\varphi }^{\prime \prime }(x)>0,\forall a<x<b$

Theorem 1.10.5: Jensen’s inequality

Let

• $\varphi$ is convex on an open interval $I$
• $X$ : Random variable
• ${\mathcal{S}}_X$ : Support of $X$

If

• ${\mathcal{S}}_X\subseteq I$
• $E(X)$ is finite

Then $\varphi [E(X)]\le E[\varphi (X)]$

Link to original