Vtyjkyuk

I am working through the Deep Learning book chapter on regularization (https://www.deeplearningbook.org/contents/regularization.html#pf5).

On page 253 there is a derivation of the expected squared error of the ensemble predictor where the predictor is composed of $k$ regression models.

We are given that each model makes an error $epsilon_i$ on each example with errors drawn from a zero-mean multivariate normal distribution. We are also given that the variance is $E(epsilon_i^2)=v$ and the covariance is $E(epsilon_i epsilon_j)=c$.

Hence the average prediction of all of the ensemble models is $frac1ksum_i epsilon_i$.

My question regards the following claim that the expected squared prediction error of the ensemble is thus:

$beginalign
E((frac1ksum_i epsilon_i)^2)&=frac1k^2E(sum_i(epsilon_i^2+sum_jneq i epsilon_i epsilon_j))\
&=frac1kv + frack-1kc.
endalign$

I am fine with the first line but how do we from $frac1k^2E(sum_i(epsilon_i^2+sum_jneq i epsilon_i epsilon_j))$ to $frac1kv + frack-1kc$?

It seems like (assuming we have $N$ examples) the correct equality should be

$$frac1k^2E(sum_i(epsilon_i^2+sum_jneq i epsilon_i epsilon_j))=fracNk^2v +fracN(N-1)k^2c.$$

I know that to make the equality match the equality given in the book my $N$s should be $k$s... what am I missing here? We are summing over examples not models.

I think the problem stems from the very confusing formulation in the text that “each model makes an error $epsilon_i$ on each example”, which leaves it unclear whether the index $i$ refers to models or examples. As I interpret the text, this index does indeed refer to models, and the expectation is taken with respect to examples, which are not indexed but considered to be statistically distributed. So the sums over constants to lead to factors of $k$ and $k(k-1)$.