\Question{Three Tails}
You flip a fair coin until you see three tails in a row. What is the average number of heads that you'll see until getting $TTT$?
\Question{Markov Property Practice}
Let $X_0, X_1, ...$ be a Markov chain with state space $S$.
One of the properties that it satisfies is the Markov property:
\[
\Pr(X_n = i_n | X_{n - 1} = i_{n-1}, \ldots, X_0 = i_0) =
\Pr(X_n = i_n | X_{n - 1} = i_{n-1}), ~
\text{for all} ~ i_0, i_1, \ldots, i_n \in S, ~ n \in \Z_{>0}.
\]
Use the Markov property and the total probability theorem to prove the following.
\begin{Parts}
\Part $\Pr(X_3 = i_3 | X_2 = i_2, X_1 = i_1) = \Pr(X_3 = i_3 | X_2 = i_2), ~
\text{for all} ~ i_1, i_2, i_3 \in S$.
\textit{Note:} This is not exactly the Markov property because it does not condition on $X_0$.
\Part $\Pr(X_3 = i_3 | X_1 = i_1, X_0 = i_0) = \Pr(X_3 = i_3 | X_1 = i_1), ~
\text{for all} ~ i_0, i_1, i_3 \in S.$
\Part $\Pr(X_1 = i_1 | X_2 = i_2, X_3 = i_3) = \Pr(X_1 = i_1 | X_2 = i_2), ~
\text{for all} ~ i_1, i_2, i_3 \in S.$
\end{Parts}
\Question{Playing Blackjack}
You are playing a game of Blackjack where you start with \$100. You are a particularly risk-loving player who does not believe in leaving the table until you either make \$400, or lose all your money. At each turn you either win \$100 with probability $p$, or you lose \$100 with probability $1 - p$.
\begin{Parts}
\Part Formulate this problem as a Markov chain i.e. define your state space, transition probabilities, and determine your starting state.
\nosolspace{2cm}
\Part Classify your states as being recurrent or transient. If a given state is recurrent, also determine whether it is an absorbing state.
\nosolspace{2cm}
\Part Find the probability that you end the game with \$400.
\nosolspace{2cm}
\end{Parts}
\Question{Boba in a Straw}
Imagine that Wan Fung is drinking milk tea and he has a very short straw: it has enough room to fit two boba (see Figure \ref{fig:straw}).
\begin{figure}[H]
\centering
\begin{tikzpicture}[scale=1.5]
\draw (-0.5, 1) -- (-0.5, -1);
\draw (0.5, 1) -- (0.5, -1);
\draw[dashed] (-0.5, 0) -- (0.5, 0);
\draw (-0.5, 1) -- (0.5, 1);
\draw (-0.5, -1) -- (0.5, -1);
\draw node[fill, circle, scale=3] at (0, 0.5) {};
\end{tikzpicture}
\caption{A straw with one boba currently inside. The straw only has enough room to fit two boba.}
\label{fig:straw}
\end{figure}
Here is a formal description of the drinking process: We model the straw as having two ``components'' (the top component and the bottom component). At any given time, a component can contain nothing, or one boba. As Wan Fung drinks from the straw, the following happens every second:
\begin{enumerate}
\item The contents of the top component enter Wan Fung's mouth.
\item The contents of the bottom component move to the top component.
\item With probability $p$, a new boba enters the bottom component; otherwise the bottom component is now empty.
\end{enumerate}
Help Wan Fung evaluate the consequences of his incessant drinking!
\begin{Parts}
\Part At the very start, the straw starts off completely empty. What is the expected number of seconds that elapse before the straw is completely filled with boba for the first time? [Write down the equations; you do not have to solve them.]
\Part Consider a slight variant of the previous part: now the straw is narrower at the bottom than at the top. This affects the drinking speed: if either (i) a new boba is about to enter the bottom component or (ii) a boba from the bottom component is about to move to the top component, then the action takes two seconds. If both (i) and (ii) are about to happen, then the action takes three seconds. Otherwise, the action takes one second. Under these conditions, answer the previous part again. [Write down the equations; you do not have to solve them.]
\Part Wan Fung was annoyed by the straw so he bought a fresh new straw (the straw is no longer narrow at the bottom). What is the long-run average rate of Wan Fung's calorie consumption? (Each boba is roughly $10$ calories.)
\Part What is the long-run average number of boba which can be found inside the straw? [Maybe you should first think about the long-run distribution of the number of boba.]
\end{Parts}