In Q-learning

$Y_t = R_{t+1} + \gamma \max_{a} \hat{q}(S_{t+1}, a; \mathbf{w})$

depends on the same parameters $\mathbf{w}$ as the prediction $\hat{q}(S_t, A_t; \mathbf{w})$ , so the target moves as $\mathbf{w}$ changes. A semi-gradient method constructs the target and then treats it as a fixed constant: we differentiate only $\hat{q}(S_t, A_t; \mathbf{w})$ :

$\mathbf{w} \leftarrow \mathbf{w} + \alpha \left[ Y_t - \hat{q}(S_t, A_t; \mathbf{w}) \right] \nabla_{\mathbf{w}} \hat{q}(S_t, A_t; \mathbf{w}).$

It is "semi" because the true gradient of $\tfrac{1}{2}\left[Y_t(\mathbf{w}) - \hat{q}(S_t, A_t; \mathbf{w})\right]^2$ would also contain $\nabla_{\mathbf{w}} Y_t(\mathbf{w})$ , and the method ignores that part. The part treated as fixed is the target $Y_t$ .