n-step and eligibility traces

Not written yet

This chapter is planned but not yet drafted. The outline below is the intended scope; chapters are written in reading order.

Learning objective

One knob spans Monte Carlo at one end and TD(0) at the other. Eligibility traces make that knob cheap.

Planned scope

• n-step returns and n-step TD
• The $\lambda$-return
• TD($\lambda$) with accumulating traces
• Forward view vs. backward view
• Choosing $n$ and $\lambda$ in practice

Prerequisites

Everything above this chapter in the sidebar.