User:Jadrian Miles/Thesis manifesto: probabilistic worldview

My dissertation involves solving three problems:

1. Automatically clustering tractography curves together so that the resulting clusters are neither too small (low reconstruction error, high model complexity) nor too big (high reconstruction error, low model complexity)
2. Automatically adjusting macrostructure elements to match input DWIs so that the elements' surfaces are neither too bumpy (low reconstruction error, high model complexity) nor too smooth (high reconstruction error, low model complexity)
3. Automatically adjusting microstructure properties within a given region in space to match input DWIs so that the spatial frequency of the microstructure parameters is neither too high (low reconstruction error, high model complexity) nor too low (high reconstruction error, low model complexity)

In each case, our goal is to balance the tradeoff between reconstruction error and model complexity. In order to do so in a principled fashion, we must define, for each case, what is meant by reconstruction error and model complexity.

${\chi }^2$ Fitting

One apparent option is ${\chi }^2$ fitting. Given:

• $N$ observations $y_i$ at independent variables $x_i$
• The knowledge that the measurement error on each observation is Gaussian with standard deviation ${\sigma }_i$
• A model instance with $M$ parameters $a_j$, which gives a reconstructed "observation" for a given independent variable as $y(x_i|a_0\dots a_{M-1})$

we define the ${\chi }^2$ statistic as follows:

${\chi }^2=\sum _{i=0}^{N-1}{\left(\frac{y_i-y(x_i|a_0\dots a_{M-1})}{{\sigma }_i}\right)}^2$

This is essentially a normalized error score. The interesting thing that we can do here, though, is compute the probability of seeing an error as bad or worse than the observed error score under a different set of observations drawn from the same underlying distribution, given the degrees of freedom $\nu \equiv N-M$. This probability is called $Q$, and is defined as:

$Q=1-\underset{{\chi }^2}{\overset{\mathrm{\infty }}{\int }}F(\psi ,\nu )\mathrm{d}\psi$

where $F(\psi ,\nu )$ is the CDF of the ${\chi }^2$ distribution. Here's how to interpret $Q$:

• Really low ($Q<0.001$) means that you can't do a much worse job describing the data than you did with your model instance. This means either that your fit is bad (${\chi }^2$ is really high) or that you have too many degrees of freedom.
• Really high ($Q\approx 1$) means that you can't do much better, even given the number of degrees of freedom. This is a suspicious result and may indicate data-fudging, so it shouldn't come up in an automated system.

This means that maximizing $Q$ over all values of $M$ (from very simple to very complex models) should find the "happy medium".