Package: pomdp 1.2.3-1
pomdp: Infrastructure for Partially Observable Markov Decision Processes (POMDP)
Provides the infrastructure to define and analyze the solutions of Partially Observable Markov Decision Process (POMDP) models. Interfaces for various exact and approximate solution algorithms are available including value iteration, point-based value iteration and SARSOP. Smallwood and Sondik (1973) <doi:10.1287/opre.21.5.1071>.
Authors:
pomdp_1.2.3-1.tar.gz
pomdp_1.2.3-1.zip(r-4.5)pomdp_1.2.3-1.zip(r-4.4)pomdp_1.2.3-1.zip(r-4.3)
pomdp_1.2.3-1.tgz(r-4.4-x86_64)pomdp_1.2.3-1.tgz(r-4.4-arm64)pomdp_1.2.3-1.tgz(r-4.3-x86_64)pomdp_1.2.3-1.tgz(r-4.3-arm64)
pomdp_1.2.3-1.tar.gz(r-4.5-noble)pomdp_1.2.3-1.tar.gz(r-4.4-noble)
pomdp_1.2.3-1.tgz(r-4.4-emscripten)pomdp_1.2.3-1.tgz(r-4.3-emscripten)
pomdp.pdf |pomdp.html✨
pomdp/json (API)
NEWS
| # Install 'pomdp' in R: |
| install.packages('pomdp', repos = c('https://mhahsler.r-universe.dev', 'https://cloud.r-project.org')) |
Bug tracker:https://github.com/mhahsler/pomdp/issues
- Cliff_walking - Cliff Walking Gridworld MDP
- DynaMaze - The Dyna Maze
- Maze - Steward Russell's 4x3 Maze Gridworld MDP
- RussianTiger - Russian Tiger Problem POMDP Specification
- Three_doors - Tiger Problem POMDP Specification
- Tiger - Tiger Problem POMDP Specification
- Windy_gridworld - Windy Gridworld MDP
control-theorymarkov-decision-processesoptimization
Last updated 3 months agofrom:2e6c1ae286. Checks:OK: 9. Indexed: yes.
| Target | Result | Date |
|---|---|---|
| Doc / Vignettes | OK | Oct 27 2024 |
| R-4.5-win-x86_64 | OK | Oct 27 2024 |
| R-4.5-linux-x86_64 | OK | Oct 27 2024 |
| R-4.4-win-x86_64 | OK | Oct 27 2024 |
| R-4.4-mac-x86_64 | OK | Oct 27 2024 |
| R-4.4-mac-aarch64 | OK | Oct 27 2024 |
| R-4.3-win-x86_64 | OK | Oct 27 2024 |
| R-4.3-mac-x86_64 | OK | Oct 27 2024 |
| R-4.3-mac-aarch64 | OK | Oct 27 2024 |
Exports:absorbing_statesactionsadd_policycolors_continuouscolors_discretecurve_multiple_directedepoch_to_episodeestimate_belief_for_nodesgreedy_MDP_actiongreedy_MDP_policygridworld_animategridworld_initgridworld_matrixgridworld_maze_MDPgridworld_plot_policygridworld_plot_transition_graphgridworld_rc2sgridworld_s2rcis_converged_POMDPis_solved_MDPis_solved_POMDPis_timedependent_POMDPmake_fully_observablemake_partially_observablemanual_MDP_policyMDPMDP_policy_evaluationnormalize_MDPnormalize_POMDPO_observation_matrixobservation_valoptimal_actionplot_belief_spaceplot_policy_graphplot_transition_graphplot_value_functionpolicypolicy_graphPOMDPprojectionq_values_MDPR_random_MDP_policyreachable_statesread_POMDPregretremove_unreachable_statesrewardreward_matrixreward_node_actionreward_valround_stochasticsample_belief_spacesimulate_MDPsimulate_POMDPsolve_MDPsolve_MDP_DPsolve_MDP_TDsolve_POMDPsolve_POMDP_parametersolve_SARSOPstart_vectorT_transition_graphtransition_matrixtransition_valupdate_beliefvalue_functionwrite_POMDP
Dependencies:clicodetoolscpp11foreachglueigraphiteratorslatticelifecyclemagrittrMatrixpkgconfigpomdpSolveprocessxpsR6Rcpprlangvctrs
Gridworlds in Package pomdp
Rendered fromgridworlds.Rmdusingknitr::rmarkdownon Oct 27 2024.Last update: 2024-02-22
Started: 2024-02-15
pomdp: Introduction to Partially Observable Markov Decision Processes
Rendered frompomdp.Rmdusingknitr::rmarkdownon Oct 27 2024.Last update: 2024-04-22
Started: 2024-02-15
Readme and manuals
Help Manual
| Help page | Topics |
|---|---|
| Access to Parts of the Model Description | accessors normalize_MDP normalize_POMDP observation_matrix observation_val reward_matrix reward_val start_vector transition_matrix transition_val |
| Available Actions | actions |
| Add a Policy to a POMDP Problem Description | add_policy |
| Cliff Walking Gridworld MDP | Cliff_walking cliff_walking |
| Default Colors for Visualization in Package pomdp | colors colors_continuous colors_discrete |
| The Dyna Maze | DynaMaze dynamaze |
| Estimate the Belief for Policy Graph Nodes | estimate_belief_for_nodes |
| Helper Functions for Gridworld MDPs | gridworld gridworld_animate gridworld_init gridworld_matrix gridworld_maze_MDP gridworld_plot_policy gridworld_plot_transition_graph gridworld_rc2s gridworld_s2rc |
| Steward Russell's 4x3 Maze Gridworld MDP | Maze maze |
| Define an MDP Problem | is_solved_MDP MDP |
| Functions for MDP Policies | greedy_MDP_action greedy_MDP_policy manual_MDP_policy MDP_policy_evaluation MDP_policy_functions q_values_MDP random_MDP_policy |
| Convert between MDPs and POMDPs | make_fully_observable make_partially_observable MDP2POMDP |
| Optimal action for a belief | optimal_action |
| Plot a 2D or 3D Projection of the Belief Space | plot_belief_space |
| POMDP Plot Policy Graphs | curve_multiple_directed plot_policy_graph |
| Extract the Policy from a POMDP/MDP | policy |
| POMDP Policy Graphs | policy_graph |
| Define a POMDP Problem | epoch_to_episode is_converged_POMDP is_solved_POMDP is_timedependent_POMDP O_ POMDP R_ T_ |
| POMDP Example Files | POMDP_example_files |
| Defining a Belief Space Projection | projection |
| Reachable and Absorbing States | absorbing_states reachable_and_absorbing reachable_states remove_unreachable_states |
| Calculate the Regret of a Policy | regret |
| Calculate the Reward for a POMDP Solution | reward reward_node_action |
| Round a stochastic vector or a row-stochastic matrix | round_stochastic |
| Russian Tiger Problem POMDP Specification | RussianTiger |
| Sample from the Belief Space | sample_belief_space |
| Simulate Trajectories in a MDP | simulate_MDP |
| Simulate Trajectories Through a POMDP | simulate_POMDP |
| Solve an MDP Problem | solve_MDP solve_MDP_DP solve_MDP_TD |
| Solve a POMDP Problem using pomdp-solver | solve_POMDP solve_POMDP_parameter |
| Solve a POMDP Problem using SARSOP | solve_SARSOP |
| Tiger Problem POMDP Specification | Three_doors Tiger |
| Transition Graph | plot_transition_graph transition_graph |
| Belief Update | update_belief |
| Value Function | plot_value_function value_function |
| Windy Gridworld MDP | Windy_gridworld windy_gridworld |
| Read and write a POMDP Model to a File in POMDP Format | read_POMDP write_POMDP |
