We verify the formula for the eigenfunctions of the Brownian motion covariance kernel, one of the few examples of a Gaussian process whose eigenfunctions can be identified in closed-form. Read more. 26 Nov 2023 - 2 minute read
We compute the expected number of iterations for an inductively-defined stochastic process to reach zero. Read more. 26 Jun 2022 - 2 minute read
In this post, I compute the expectation and maximization steps for a Bernoulli distribution with two unknown success probabilities. Read more. 19 May 2022 - 1 minute read
This article continues our analysis of the Fourier transform by defining the Fourier transform of measures instead of functions. Read more. 08 May 2022 - 3 minute read
This article investigates the Fourier transform and derives some basic properties such as the fact that the Fourier transform of a Gaussian function is itself Gaussian and the equation for the Fourier transform of the rect function. Read more. 07 May 2022 - 3 minute read
I read a very useful description of the EM algorithm as coordinate ascent and wanted to rederive some of the ideas. Read more. 19 Apr 2022 - 2 minute read
Bayesian optimization is an exciting idea in machine learning that has recently achieved state-of-the-art performance by modeling a performance metric (e.g. accuracy or mean squared error) as a function of model hyperparameters. In this post, I speculate about how this technique may be employed in the design of frequentist Hypothesis tests. Read more. 24 Sep 2017 - 2 minute read
Yesterday I was reading two very interesting papers, which basically express the same idea. In particular, that we can iteratively refine a variational posterior approximation by taking a Gaussian mixture distribution and appending components to it, at each stage reducing the KL-divergence between the true posterior and the mixture approximation. The two papers are Variational Boosting Iteratively Refining Posterior Approximations and Boosting Variation Inference (you can see they almost have exactly the same name!). In this post, I want to try to connect at least one of the insights here to my research on Stein variational gradient descent. Read more. 18 Sep 2017 - 7 minute read
Read more. 13 Sep 2017 - less than 1 minute read
The good folks over at QuantStart wrote an excellent article detailing a strategy that periodically rebalances a portfolio according to some weighting scheme that is specified *a priori*. In this article, I want to demonstrate how a similar strategy can be implemented in Odin. Read more. 06 Jul 2017 - 5 minute read
In this article, I want to introduce the Odin algorithmic trading and backtesting platform. I will exhibit how Odin can be leveraged to backtest a very simple buy-and-hold strategy for the Spyder ETF (SPY). Read more. 06 Jul 2017 - 6 minute read
In this post we'll be seeking to understand some of the principals underlying SteinGAN, a method for reducing sampling complexity by training a model (often called the generator) that leverages Stein variational gradient descent. Specifically, let us denote by $\xi$ a random variable drawn from a prior noise distribution; then the objective of SteinGAN is to produce a model $G\left(\xi; \nu\right)$ such that the output of $G$ is distributed according to a target distribution $p\left(x;\theta\right)$. In this setup, both $\nu$, the parameters of the generative model, and $\theta$, the parameters of the target distribution, are both unknown and need to be estimated. Read more. 13 Jun 2017 - 3 minute read
I wanted to do a quick post to verify an identity regarding exponential families that I encountered while reading about SteinGANs. Read more. 12 Jun 2017 - less than 1 minute read
In our last discussion, we focused on the kernelized Stein discrepancy and how variational Stein gradient descent can be leveraged to sample from posterior distributions of Bayesian models. Our application area in that case was specifically Bayesian linear regression, where a prior (with a fixed precision) was placed over the linear coefficients of the linear model. Unfortunately, Bayesian linear regression is somewhat uninteresting because it is possible to compute the closed-form posterior for the coefficients. In this post, we'll demonstrate two statistical models where this is not the case. Read more. 10 Jun 2017 - 13 minute read
A common metric in the field of quantitative finance is the Sharpe ratio. The Sharpe ratio is a measure of the extent of the returns (over the risk-free rate) that a strategy will produce relative to the amount of risk it assumes. Here we examine the Sharpe ratio of a simple game. Read more. 06 Jun 2017 - 2 minute read
In this post, we'll be taking a closer look at the *kernelized* Stein discrepancy that provides the real utility underlying Stein variational gradient descent. Read more. 06 Jun 2017 - 8 minute read
Stein variational gradient descent is a technique developed by the Dartmouth machine learning group. The essential idea to perturb samples from a simple distribution until they approximate draws from a target distribution. Read more. 05 Jun 2017 - 5 minute read