Package: markovDP 0.99.0
markovDP: Infrastructure for Discrete-Time Markov Decision Processes (MDP)
The package provides the infrastructure to work with MDPs in R. The focus is on convenience in formulating MDPs, the support of sparse representations (using sparse matrices, lists and data.frames) and visualization of results. Some key components are implemented in C++ to speed up computation. It also implements several popular solvers.
Authors:
markovDP_0.99.0.tar.gz
markovDP_0.99.0.zip(r-4.5)markovDP_0.99.0.zip(r-4.4)markovDP_0.99.0.zip(r-4.3)
markovDP_0.99.0.tgz(r-4.4-arm64)markovDP_0.99.0.tgz(r-4.4-x86_64)markovDP_0.99.0.tgz(r-4.3-arm64)markovDP_0.99.0.tgz(r-4.3-x86_64)
markovDP_0.99.0.tar.gz(r-4.5-noble)markovDP_0.99.0.tar.gz(r-4.4-noble)
markovDP_0.99.0.tgz(r-4.4-emscripten)markovDP_0.99.0.tgz(r-4.3-emscripten)
markovDP.pdf |markovDP.html✨
markovDP/json (API)
NEWS
| # Install 'markovDP' in R: |
| install.packages('markovDP', repos = c('https://mhahsler.r-universe.dev', 'https://cloud.r-project.org')) |
Bug tracker:https://github.com/mhahsler/markovdp/issues
- Cliff_walking - Cliff Walking Gridworld MDP
- DynaMaze - The Dyna Maze
- Maze - Steward Russell's 4x3 Maze Gridworld MDP
- Windy_gridworld - Windy Gridworld MDP Windy Gridworld MDP
control-theorymarkov-decision-processoptimization
Last updated 2 months agofrom:519b951586
Exports:absorbing_statesactionactionsadd_policybellman_operatorcolors_continuouscolors_discretecurve_multiple_directedepoch_to_episodegreedy_actiongreedy_policygridworld_animategridworld_initgridworld_matrixgridworld_maze_MDPgridworld_plotgridworld_plot_transition_graphgridworld_rc2sgridworld_read_mazegridworld_s2rcis_solved_MDPmanual_policyMDPnormalize_MDPplot_transition_graphplot_value_functionpolicypolicy_evaluationq_valuesR_random_policyreachable_statesregretremove_unreachable_statesrewardreward_matrixround_stochasticsimulate_MDPsolve_MDPsolve_MDP_DPsolve_MDP_LPsolve_MDP_TDstart_vectorT_transition_graphtransition_matrixvalue_function
Dependencies:clicodetoolscpp11foreachglueigraphiteratorslatticelifecyclelpSolvemagrittrMatrixpkgconfigRcpprlangvctrs
Readme and manuals
Help Manual
| Help page | Topics |
|---|---|
| Access to Parts of the Model Description | accessors normalize_MDP reward_matrix start_vector transition_matrix |
| Action Given a Policy | action action.MDP |
| Available Actions in a State | actions |
| Add a Policy to a MDP Problem Description | add_policy |
| Cliff Walking Gridworld MDP | Cliff_walking cliff_walking |
| Default Colors for Visualization | colors colors_continuous colors_discrete |
| The Dyna Maze | DynaMaze dynamaze |
| Helper Functions for Gridworld MDPs | gridworld gridworld_animate gridworld_init gridworld_matrix gridworld_maze_MDP gridworld_plot gridworld_plot_transition_graph gridworld_rc2s gridworld_read_maze gridworld_s2rc |
| Steward Russell's 4x3 Maze Gridworld MDP | Maze maze |
| Define an MDP Problem | epoch_to_episode is_solved_MDP MDP R_ T_ |
| Extract or Create a Policy | manual_policy policy random_policy |
| Policy Evaluation | bellman_operator policy_evaluation |
| Q-Value Functions | greedy_action greedy_policy q_values |
| Reachable and Absorbing States | absorbing_states reachable_and_absorbing reachable_states remove_unreachable_states |
| Calculate the Regret of a Policy | regret |
| Calculate the Expected Reward of a Policy | reward reward.MDP |
| Round a stochastic vector or a row-stochastic matrix | round_stochastic |
| Simulate Trajectories in a MDP | simulate_MDP |
| Solve an MDP Problem | solve_MDP solve_MDP_DP solve_MDP_LP solve_MDP_TD |
| Transition Graph | curve_multiple_directed plot_transition_graph transition_graph |
| Value Function | plot_value_function value_function |
| Windy Gridworld MDP Windy Gridworld MDP | Windy_gridworld windy_gridworld |
