This post continues the discussion of the previous post on random walks. The previous post looks at random walks graphically. This post introduces the notion of random walks more formally.
Consider , which is a sequence of random variables indexed by the non-negative integers. By definition this is a stochastic process, to be precise a discrete stochastic process since the random variables are indexed by integers. The integers in the subscripts can be interpreted as time. Thus the random variables describe the states of some process over time. The stochastic processes discussed in the previous post, the future state of the process is obtained by adding one (with probability ) or subtracting one (with probability ) to the current state . So a given state of the process is one higher or lower than the state in the previous period. The random walks in the previous post go up or down with equal chance, i.e. . The following graph is from the previous post, which shows 5 simulations of such random walks.
Figure 1 – Examples of Random Walks in One Dimension
Random Walks – A General Description
We now give a more general description of the random walk. Instead of a random one-unit up or down move, the moves in the random walk are determined by a predetermined discrete distribution. Let be integer-valued random variables that are independent and identically distributed. Let be the common probability function. Let be an integer-valued random variable that is independent of the random variables . The random variable will be the initial position of the random walk. The random variables are the increments (they are the amounts added to the stochastic process as time increases).
For , define . Then is called a random walk. Wherever the process is at any given time, the next step in the process is always a walk in a distance and direction that is dictated by the independent random variables . For convenience, we will call the random variables the increments in the random walk since they tell the process how far (and what direction) to walk in the next step.
The state space is the set of all the values that can be taken on by the random variables . In this case, the state space is either the set of all integers or the set of all non-negative integers or some appropriate subset of or .
Given the current state of the process , the next state is obtained by adding the random value to the previous state . If the random variables are independent Bernoulli variables with and . Then each state is the result of an up or down move by one unit from the previous state. The examples in the previous post are based on this type of Bernoulli one-unit up and down move. In the general definition, the future state , instead of being adjusted by a uniform up or down adjustment, is adjusted by the random variable .
One important task is to establish the probabilities of the transition from one state to another. For example, if the process is currently in state , what is the probability that it will be in state in the next period? More specifically, given that the random walk is in state , i.e. , what is the probability that the process in in state in the next period, i.e. ? Such probabilities are called one-step transition probabilities.
The probabilities that we wish to obtain are for . Recall that is the common probability function for the random variables . Given states , define .
When , in order for to happen, the increment must be . Note that . Observe that this probability is independent of the time period since the have a common distribution . Thus it is appropriate to use to express this probability, i.e. there is no need to include the time in the subscript of . For all ,
The number is the probability of the process transitioning from state to state in one step, which is evaluated based on the common probability function of the increments .
Now that we know the probability of the process going one state to another state in one step, other probability questions are: what is the probability of a path and what is the probability of going from state to state in steps? The probabilities in the second question are called -step transition probabilities. The probability of a path is discussed here. The -step transition probabilities are discussed here.
The first question has straightforward answers. For example, if the process starts in state at time 0, moves to state at time 1 and so on until reaching state at time , then the following is the probability of this path:
Conditional on the initial state , the probability of the process going through the path is simply the product of the one-step transition probabilities.
If the path is observed to start at a time period other than time 0, conditional on the first state in the path, the probability of the process going through the path would be:
Note that the transition probabilities and the probabilities of paths are independent of the current period . Random walks with such property are called time-homogeneous or stationary. Thus the probability of transitioning into state from state or the probability of transitioning through a path is identical regardless of where the process is in the time scale (at the beginning in the process or later in the process).
For special cases of the random walks discussed above, let the increments be defined by a Bernoulli random variable with and . Then the resulting random walk is a series of up and down moves as shown in Figure 1 above. The one-step transition probabilities are:
If the increments are defined by the distribution where , and , then the following gives the transition probabilities:
This random walk has three moves at any given state, an up move by one unit, a down move by one unit and a possibility that the process stays at the same state. Of course, the possibility for modeling is limitless. We can use other distributions for the common distribution of the increments , e.g. binomial distribution, Poisson, preferably having these distributions shifted in order to account for up moves and down moves.
Finite-State Random Walks
The random walks discussed above potentially have an infinite state space, i.e. the process can potentially reach far from the initial state without bounds. Depending on the common distribution of the increments , the process can go up indefinitely to or go down to . The random walks in Figure 1 have increments 1 and -1 and can reach states that are arbitrarily high or low, though it would take a great many steps to reach such high levels.
Any random walk with unbounded state space can be modified to have a finite state space. Pick two states that serve as boundaries, one lower and one upper. The states in between the boundary states are called interior states. Whenever the process reaches the upper boundary or the lower boundary, the process either stays there and not move any further or transition back into an interior state. The transition probabilities in the interior states remain the same as discussed above. If a boundary state is such that the process always stays there after reaching it, it is called an absorbing state.
A handy example of a finite-state random walk is the gambler’s ruin. Suppose that a gambler starts out with units in capital (in dollars or other monetary units) and makes a series of one-unit bets against the house. For the gambler, the probabilities of winning and losing each bet are and , respectively. Whenever the capital reaches zero, the gambler is in ruin and his capital remains zero thereafter. On the other hand, if the capital of the gambler increases to where , then the gambler quits playing. Let be the capital of the gambler after the th bet. Then is a random walk. The starting state is . The states and are absorbing states. The following gives the transition probabilities.
Whenever the gambler reaches the absorbing state of 0, the gambler is in ruin. If the process reaches the absorbing state of , the gambler strikes gold. If the initial capital of the gambler is small relative to the casino, there is a virtually 100% chance that the gambler will be in ruin even if each bet has even odds (see here for the calculation). One interesting questions: on average how long does it take for the gambler to be in ruin? Such questions will be answered after necessary tools are built in subsequent posts.
A random walk is a stochastic process such that at each time point, the process walks a certain distance that is dictated by an independent and identically distributed sequence of random variables. In such a stochastic process, the future state depends only on the preceding state and not on the past states before the preceding state. For example, in the gambler’s ruin, the gambler’s capital at any given point depends only on the his capital in the previous play.
In a stochastic process , the property that the future state depends only on the preceding state and not on the past states is called the Markov property. Any stochastic process with the Markov property is called a Markov chain. A random walk is a Markov chain. There are Markov chains that are not random walk. The notion of Markov chains is introduced in this post.