This analysis covers implementation of two different MDP in a non-grid and a grid-based search problem with various solvers (i.e., value iteration, policy iteration, Q-learning) to assess the advantages and disadvantages of each approach