A Note on Conditional Expectations

This post is about a very small problem on conditional expectations but which got me thinking for a couple of days. Eventually I realized how I never really fully understood them.

The problems is this: Let $X, Y$ be two random variables uniformly distributed between $0$ and $1$. Find $\mathbb{E}(X\vert X+Y \leq 1)$.

I was discussing with a friend and he even tried to write the Bayes formula for exceptions. Only when I started thinking about this I realized, how may people, including me, don't see the absurdity of trying to find Expectation of events and Probability of variables.

For example, $\mathbb{E}(X+Y \leq 1)$ doesn't make any sense at all. Neither does $\mathbb{P}(X)$. Both of them completely nonsensical. For the first one to make sense, it would be more appropriate to talk about $\mathbb{E}(I_{X+Y \leq 1})$ where $I$ is an indicator random variable. Similarly, $\mathbb{P}(0.25\leq X \leq 0.5)$ is valid expression.

Let $X$ be a variable and $Z$ be an event, something like $X+Y\leq1$ or $X \geq Y$. Let $I_Z$ be the indicator random variable of the event $Z$. Now, conditioning on $Z$,

$\begin{align}\mathbb{E}(X\cdot I_z)&=\mathbb{P}(Z)\mathbb{E}(X\cdot I_z\vert Z)+\mathbb{P}(Z^c)\mathbb{E}(X\cdot I_z \vert Z^c)\\
&=\mathbb{P}(Z)\mathbb{E}(X\cdot 1\vert Z)+\mathbb{P}(Z^c)\mathbb{E}(X\cdot 0\vert Z^c)\\
&=\mathbb{P}(Z)\mathbb{E}(X\vert Z)\\
\end{align}$

On the other hand, conditioning on $X$,

$\begin{align}\mathbb{E}(X\cdot I_z)&=\mathbb{E}(\mathbb{E}(X\cdot I_z \vert X))\\
&=\mathbb{E}(X\cdot \mathbb{E}(I_z\vert X))\\
&=\mathbb{E}(X\cdot \mathbb{P}(Z\vert X))
\end{align}$

Equating the two, we finally get,

$\mathbb{E}(X\vert Z)=\displaystyle\frac{\mathbb{E}(X\cdot \mathbb{P}(Z\vert X))}{\mathbb{P}(Z)}$

Now this makes the problem tractable, as all the terms makes perfect sense provided we understand that $X$ is a variable and $Z$ is an event (which could even be something like $X=3$ in the discrete case).

An alternate way of deriving the same result will be like the following. Assume $X$ is a continuous random variable. The reasoning will be very similar if $X$ is discrete.

$\begin{align}
\mathbb{E}(X\vert Z)&=\int x f_{X\vert Z}(x)\,dx\\
&=\int x \text{ }\frac{\mathbb{P}(Z\vert X)f_{X}(x)}{\mathbb{P}(Z)}\,dx=\displaystyle\frac{1}{\mathbb{P}(Z)}\int x\text{ } \mathbb{P}(Z\vert X)f_{X}(x)\,dx\\
&=\displaystyle\frac{\mathbb{E}(X\cdot \mathbb{P}(Z\vert X))}{\mathbb{P}(Z)}
\end{align}$

However, the first way includes a lot of tiny beautiful details like the conditioning on a random variable, law of total expectation, and most importantly it gives a chance to understand that,

$\mathbb{E}(X\cdot I_z\vert I_z=1)\ne\mathbb{E}(X)$ in general. Rather when the dependence between $X$ and $Z$ is not clear or not given, it is always better to state the relation as,

$\mathbb{E}(X\cdot I_z\vert I_z=1)=\mathbb{E}(X\vert Z)$

That's what I wanted to convey in this post because I had a very difficult time on how to solve that conditional expectation. Hope you enjoyed this.


Until then
Yours Aye,
Me