On the Time Derivative of the KL Divergence for a Generalized Langevin Annealing Scheme

Consider the Langevin diffusion process $\mathrm{d} X_t = \nabla \log p_t(X_t) + \sqrt{2}\mathrm{d} W_t$ guided by the time-dependent probability density $p_t(x)$. Let $q_t$ be the density of $X_t$. Recently, in order to analyze convergence in the Kullback-Leibler divergence, the time derivative of $t\mapsto \mathrm{KL}(q_t|p_t)$ has been used in several works without investigating in detail when such a derivative exists. In this short manuscript we provide a rigorous derivation of the quantity $\frac{\mathrm{d}}{\mathrm{d} t}\mathrm{KL}(q_t|p_t)$.