Write the state and the clock
Define which information the state retains, whether time is discrete or continuous, and whether independence or memorylessness is justified. A Markov shortcut fails if the state discards relevant history.
Worked example: two transitions
Let P(A→A)=0.8 and P(B→A)=0.3. Starting in A, the chance of being in A after two transitions is 0.8² + 0.2×0.3 = 0.70. The paths are A→A→A and A→B→A. Multiplying 0.8 twice misses the return path.
For many transitions use a matrix. A stationary distribution solves πP=π and sums to one; it is not automatically the distribution after two steps.
Arrivals and decisions
Independent thinning of Poisson arrivals at rate λ with retention probability p gives rate λp. Independent exponential clocks race with first-arrival probability λA/(λA+λB). Both statements depend on the specified models.
Stopping without a free lunch
A bounded stopping time preserves a martingale’s expectation. Unbounded doubling requires additional integrability analysis; “eventual success” is not an expected-profit argument. Study pricing and advanced reasoning, then use an external notebook to simulate and inspect limiting behavior.
Try a fresh question
This optional exercise uses the actual parameterized bank.
Worked method
Keep the practice connected
Train Stochastic Processes · Follow the Quant Path · Evidence standard