The Generative Model

In the Generative model, given any state s and action a, you can get a new state s' from P(s'|s,a).

Example: Robbie has a simulator for his world.

Deterministic Generative Model

The same as a generative model, except that you control the random numbers.

Example: Robbie has direct control over the random numbers in his simulator.