CONTENTS

The MDP toolbox proposes functions related to the resolution of discrete-time Markov Decision Processes: backwards induction, value iteration, policy iteration, linear programming algorithms with some variants.
The functions (m-functions) were developped with MATLAB (one of the functions requires the Mathworks Optimization Toolbox) by the Biometry and Artificial Intelligence Unit of INRA Toulouse (France).
The version 2.0 (February 2005) handles sparse matrices and adds an example.
The version 3.0 (September 2009) adds several functions related to Reinforcement Learning and improve the handling of sparse matrices. For more detail see the README file.

FUNCTIONS DESCRIPTION

• Functions by category
• Alphabetical list of functions

NOTATION

• states: set {1, 2, ..., S}
• actions: set {1, 2, ..., A}
• transitions: P(s,s',a) is the probability to reach state s' when the system is in state s and action a is performed by the decision maker
• rewards: R(s,s',a) is the reward obtained when the system is in state s on decision epoch t and is in state s' at decision epoch t+1, with action a performed
  R(s,a): reward when the system is in state s at decision epoch t and action a is performed by the decision maker

USE OF DOCUMENTATION

The documentation pages can be displayed with the MATLAB navigator (used for the MATLAB help). In the File menu, choose the Open item. Open the sub-directory documentation. Select the item 'All Files (*.*)' for the attribut Files of type. Then select the file DOCUMENTATION.html and open it.
The sub-directory documentation contains all the pages describing the m-functions in HTML.