This material describe the Q-function approximator for DQN/DDQN and the policy-value network used in PPO. Key components include experience replay and target network updates for DQN-based methods, and clipped objective with generalized advantage estimation for PPO.