In this post I want to prove the following Theorem leveraging Stein’s Identity:

Let $p$ and $q$ be two continuous densities defined on $\mathcal{X} \subset \mathbb{R}^d$. Then $p = q$ if and only if $\mathbb{E}_{x\sim q}\left[\mathcal{A}_{p} f(x)\right] = 0$ for every function $f$. That is, two distributions are equal if and only if Stein’s identity holds for every sufficiently regular function.

To begin with let us note that if $p = q$ then it is obvious that Stein’s identity must hold. Therefore, the more interesting direction is to demonstrate that if Stein’s identity holds for every $f$ then $p$ and $q$ must be equal. We proceed as follows, \begin{align} \mathbb{E}_{x\sim q}\left[\mathcal{A}_{p} f(x)\right] &= \mathbb{E}_{x\sim q}\left[\mathcal{A}_{p} f(x)\right] - \mathbb{E}_{x\sim q}\left[\mathcal{A}_{q} f(x)\right] \\
&= \mathbb{E}_{x\sim q}\left[f(x)\left(\nabla_{x}\log p(x) - \nabla_{x}\log q(x)\right)\right] \\
&= 0 \end{align} Now, if for all functions this statement holds true, it requires that $\nabla_{x}\log p(x) = \nabla_{x}\log q(x)$ for all $x\in \mathcal{X}$. Finally, if the derivatives of the log-probabilities are equal, then the distributions themselves must certainly be equal. This proves the theorem.