Abstract
In this chapter, we define the Discrete-Time Markov Chain Process operator, all the initial components seen in the previous chapter are applied, and the
vector of final conditions as known as steady-state vector is defined and exemplified, this vector shows the final state of the process and depends on the initial state
vector and the matrix of transition probabilities. The solution mechanism is shown
both by the iteration of the vector-matrix product and by determining the eigenvalues and eigenvectors of the matrix of transition probabilities. In an effort to categorize the possible matrix of transition probabilities, they are illustrated as reducible
form, trasient form and recurrent form. In an effort to categorize the possible matrix
of transition probabilities, they are illustrated as reducible form, trasient form and
recurrent form. As a direct application of the Discrete-Time Markov process, the
Metropolis Algorithm is presented, as well as a regularity that can be observed in
the matrix of transition of probabilities and that is described in the section Law of
Large Numbers. Some full basic examples are provided to illustrate the definition
and operation of this ramdon walk.