Expectation | Mean | Average (2024)

Video Available

• Solution
  • For the Bernoulli distribution, the range of $X$ is $R_X=\{0,1\}$, and $P_X(1)=p$and $P_X(0)=1-p$. Thus,

    $EX$	$=0 \cdot P_X(0)+1 \cdot P_X(1)$
    	$=0 \cdot (1-p)+ 1 \cdot p$
    	$=p$.

    
    

• Solution
  • For the geometric distribution, the range is $R_X=\{1,2,3,... \}$ and the PMF is given by$$P_X(k) = q^{k-1}p, \hspace{20pt} \text{ for } k=1,2,...$$where, $0 < p < 1$ and $q=1-p$. Thus, we can write

    $EX$	$=\sum_{x_k \in R_X} x_k P_X(x_k)$
    	$=\sum_{k=1}^{\infty} k q^{k-1}p$
    	$=p\sum_{k=1}^{\infty} k q^{k-1}$.

    
    Now, we already know the geometric sum formula$$\sum_{k=0}^{\infty} x^k= \frac{1}{1-x}, \hspace{20pt} \textrm{ for } |x| < 1.$$But we need to find a sum $\sum_{k=1}^{\infty} k q^{k-1}$. Luckily, we can convert the geometricsum to the form we want by taking derivative with respect to $x$, i.e.,$$\frac{d}{dx} \sum_{k=0}^{\infty} x^k= \frac{d}{dx} \frac{1}{1-x}, \hspace{20pt} \textrm{ for } |x| < 1.$$Thus, we have$$\sum_{k=0}^{\infty} k x^{k-1}= \frac{1}{(1-x)^2}, \hspace{20pt} \textrm{ for } |x| < 1.$$To finish finding the expectation, we can write

    $EX$	$=p\sum_{k=1}^{\infty} k q^{k-1}$
    	$=p \frac{1}{(1-q)^2}$
    	$=p \frac{1}{p^2}$
    	$=\frac{1}{p}$.

    
    So, for $X \sim Geometric(p)$, $EX=\frac{1}{p}$. Note that this makes sense intuitively.The random experiment behind the geometric distribution was that we tossed a coin untilwe observed the first heads, where $P(H)=p$. Here, we found out that on average you needto toss the coin $\frac{1}{p}$ times in this experiment. In particular, if $p$ is small(heads are unlikely), then $\frac{1}{p}$ is large, so you need to toss the coin a largenumber of times before you observe a heads. Conversely, for large $p$ a few coin tossesusually suffices.

• Solution
  • Before doing the math, we suggest that you try to guess what the expected value would be.It might be a good idea to think about the examples where the Poisson distribution is used. Forthe Poisson distribution, the range is $R_X=\{0,1,2,\cdots \}$ and the PMF is given by$$P_X(k) = \frac{e^{-\lambda} \lambda^k}{k!}, \hspace{20pt} \text{ for } k=0,1,2,...$$Thus, we can write

    $EX$	$=\sum_{x_k \in R_X} x_k P_X(x_k)$
    	$= \sum_{k=0}^{\infty} k \frac{e^{-\lambda} \lambda^k}{k!}$
    	$=e^{-\lambda} \sum_{k=1}^{\infty} \frac{ \lambda^k}{(k-1)!}$
    	$=e^{-\lambda} \sum_{j=0}^{\infty} \frac{\lambda^{(j+1)}}{j!}$	$(\textrm{ by letting }j=k-1)$
    	$=\lambda e^{-\lambda} \sum_{j=0}^{\infty} \frac{ \lambda^j}{j!}$
    	$=\lambda e^{-\lambda} e^{\lambda}$	$(\textrm{ Taylor series for } e^{\lambda})$
    	$=\lambda$.

    
    So the expected value is $\lambda$. Remember, when we first talked about the Poisson distribution,we introduced its parameter $\lambda$ as the average number of events. So it is not surprisingthat the expected value is $EX=\lambda$.

Expectation is linear:
Theorem
We have

• $E[aX+b]=aEX+b$, for all $a,b \in \mathbb{R}$;
• $E[X_1+X_2+\cdots+X_n]=EX_1+EX_2+\cdots+EX_n$, for any set of random variables $X_1, X_2,\cdots,X_n$.

• Solution
  • We provide two ways to solve this problem. One way is as before: we do the math andcalculate $EX=\sum_{x_k \in R_X} x_k P_X(x_k)$ which will be a little tedious. A muchfaster way would be to use linearity of expectation. In particular, remember that if$X_1, X_2, ...,X_n$ are independent $Bernoulli(p)$ random variables, then the randomvariable $X$ defined by $X=X_1+X_2+...+X_n$ has a $Binomial(n,p)$ distribution. Thus,we can write

    $EX$	$=E[X_1+X_2+\cdots+X_n]$
    	$=EX_1+EX_2+\cdots+EX_n$	$\hspace{20pt} \textrm{by linearity of expectation}$
    	$=p+p+\cdots+p$
    	$=np$.

    
    We will provide the direct calculation of $EX=\sum_{x_k \in R_X} x_k P_X(x_k)$ in the SolvedProblems section and as you will see it needs a lot more algebra than above. The bottom lineis that linearity of expectation can sometimes make our calculations much easier. Let's lookat another example.

• Solution
  • We claim that if the $X_i$'s are independent and $X_i \sim Geometric(p)$, for $i=1$, $2$, $\cdots$,$m$, then the random variable $X$ defined by $X=X_1+X_2+\cdots+X_m$ has $Pascal(m,p)$. To seethis, you can look atProblem 5 in Section 3.1.6and the discussion there. Now, since we already know $EX_i=\frac{1}{p}$, we conclude

    $EX$	$=E[X_1+X_2+\cdots+X_m]$
    	$=EX_1+EX_2+\cdots+EX_m$	$\hspace{20pt} \textrm{by linearity of expectation}$
    	$=\frac{1}{p}+\frac{1}{p}+\cdots+\frac{1}{p}$
    	$=\frac{m}{p}$.

    
    Again, you can try to find $EX$ directly and as you will see, you need much more algebracompared to using the linearity of expectation.