1. \[ \mathbf u^T\mathbf v = 3(4) + 0(3) + 4(0) = 12 \]
2. \[ \|\mathbf u\| = \sqrt{9 + 0 + 16} = 5 \]
3. \[ \|\mathbf v\| = \sqrt{16 + 9 + 0} = 5 \]

Marks: 1 for the dot product, 1 for both norms.

1. \[ \cos\theta = \frac{\mathbf u^T\mathbf v}{\|\mathbf u\|\,\|\mathbf v\|} = \frac{12}{25} = 0.48 \]
2. \[ \theta = \cos^{-1}(0.48) \approx 61.3^\circ \]

The vectors have the same length, so the cosine is small-ish and the angle is noticeably larger than 45°. Positive cosine means they lean the same way; 0 would mean orthogonal.

1. \[ \mathrm{proj}_{\mathbf v}(\mathbf u) = \frac{\mathbf u^T\mathbf v}{\mathbf v^T\mathbf v}\,\mathbf v = \frac{12}{25}\begin{bmatrix} 4 \\ 3 \\ 0 \end{bmatrix} \]
2. \[ \mathrm{proj}_{\mathbf v}(\mathbf u) = \begin{bmatrix} 1.92 \\ 1.44 \\ 0 \end{bmatrix} \]
3. \[ \mathbf u - \mathrm{proj}_{\mathbf v}(\mathbf u) = \begin{bmatrix} 1.08 \\ -1.44 \\ 4 \end{bmatrix} \]
4. \[ (1.08)(4) + (-1.44)(3) + (4)(0) = 4.32 - 4.32 = 0 \quad\checkmark \]

The post-quiz note in your notebook says you memorised this formula wrong once. The denominator is \(\|\mathbf v\|^2 = \mathbf v^T\mathbf v\), the squared length of the vector you project onto, and the result is a multiple of \(\mathbf v\), never of \(\mathbf u\).

1. \[ \mathbf u - \mathbf v = \begin{bmatrix} -1 \\ -3 \\ 4 \end{bmatrix} \]
2. \[ \|\mathbf u - \mathbf v\|_2 = \sqrt{1 + 9 + 16} = \sqrt{26} \approx 5.10 \]
3. \[ \|\mathbf u - \mathbf v\|_1 = 1 + 3 + 4 = 8 \]

The Euclidean distance can also come from the cosine rule: \(\|\mathbf u-\mathbf v\|^2 = 25 + 25 - 2(12) = 26\). Same answer, and a good way to catch an arithmetic slip.

1. Every value is between 0 and 1, and \[ \tfrac{1}{10} + \tfrac{2}{10} + \tfrac{3}{10} + \tfrac{4}{10} = 1 \]
2. \[ F(1) = 0.1,\quad F(2) = 0.3,\quad F(3) = 0.6,\quad F(4) = 1 \]

\(F(x) = 0\) for \(x \lt 1\), it is a staircase that jumps at 1, 2, 3, 4, and it stays at 1 for \(x \ge 4\).

1. Only \(x = 2\) and \(x = 3\) fall in the interval: \[ P(1.5 \le X \le 3.5) = F(3) - F(1) = 0.6 - 0.1 = 0.5 \]

\(F(1.5) = F(1)\) because nothing happens between 1 and 2. Writing \(F(3) - F(1.5)\) is also right; writing \(F(3.5) - F(1.5)\) with made-up values is the common slip.

1. \[ E[X] = \frac{1 + 4 + 9 + 16}{10} = 3 \]
2. \[ E[X^2] = \frac{1 + 8 + 27 + 64}{10} = 10 \]
3. \[ \mathrm{Var}(X) = E[X^2] - (E[X])^2 = 10 - 9 = 1 \]

1. \[ \int_0^2 c\,y\,dy = c\left[\frac{y^2}{2}\right]_0^2 = 2c = 1 \]
2. \[ c = \tfrac12 \]
3. \[ F(y) = \int_0^y \frac{t}{2}\,dt = \frac{y^2}{4}, \qquad 0 \le y \le 2 \]

\(F(y) = 0\) below 0 and \(F(y) = 1\) above 2. Say so: the piecewise form is part of the answer.

1. \[ P(1 \le Y \le 2) = F(2) - F(1) = 1 - \tfrac14 = 0.75 \]
2. Median: solve \(F(\nu) = \tfrac12\): \[ \frac{\nu^2}{4} = \frac12 \;\Rightarrow\; \nu = \sqrt 2 \approx 1.414 \]

The mean is \(\int_0^2 y \cdot \tfrac{y}{2}\,dy = \tfrac43 \approx 1.333\). Median and mean differ because the density is not symmetric.

1. \[ \bar X = \tfrac{15}{5} = 3, \qquad \bar Y = \tfrac{15}{5} = 3 \]
2. \[ \hat{\boldsymbol\mu} = \begin{bmatrix} 3 \\ 3 \end{bmatrix} \]
3. Sort \(Y\): \((1, 2, 3, 4, 5)\). With \(n = 5\) odd, the median is the 3rd value: \[ \operatorname{median}(X) = 3, \qquad \operatorname{median}(Y) = 3 \]

1. Centre: \[ X - \bar X = (-2, -1, 0, 1, 2), \qquad Y - \bar Y = (-2, 0, -1, 2, 1) \]
2. \[ \hat\sigma_{XX} = \tfrac15(4 + 1 + 0 + 1 + 4) = 2 \]
3. \[ \hat\sigma_{YY} = \tfrac15(4 + 0 + 1 + 4 + 1) = 2 \]
4. \[ \hat\sigma_{XY} = \tfrac15\big(4 + 0 + 0 + 2 + 2\big) = 1.6 \]
5. \[ \hat\Sigma = \begin{bmatrix} 2 & 1.6 \\ 1.6 & 2 \end{bmatrix} \]

Marks: 1 for the centred columns, 1 for the two variances, 1 for the covariance placed symmetrically.

1. \[ r = \frac{\hat\sigma_{XY}}{\sqrt{\hat\sigma_{XX}\,\hat\sigma_{YY}}} = \frac{1.6}{\sqrt{2 \cdot 2}} = 0.8 \]
2. A strong positive linear relationship: when \(X\) is above its mean, \(Y\) tends to be above its mean too.

1. For \(\begin{bmatrix} a & b \\ b & a \end{bmatrix}\) the eigenvalues are \(a \pm b\): \[ \lambda_1 = 2 + 1.6 = 3.6, \qquad \lambda_2 = 2 - 1.6 = 0.4 \]
2. Check: \[ \lambda_1 + \lambda_2 = 4 = \operatorname{tr}\hat\Sigma \]
3. \((\hat\Sigma - 3.6 I)\mathbf v = 0\) gives \(-1.6 v_1 + 1.6 v_2 = 0\), so \(v_1 = v_2\): \[ \mathbf v_1 = \tfrac{1}{\sqrt2}\begin{bmatrix} 1 \\ 1 \end{bmatrix} \]
4. \[ \frac{\lambda_1}{\lambda_1 + \lambda_2} = \frac{3.6}{4} = 0.9 = 90\% \]

Then \(a \pm b\) does not apply. Use the characteristic equation \(\lambda^2 - \operatorname{tr}(\hat\Sigma)\lambda + \det(\hat\Sigma) = 0\) and the quadratic formula, as in Problem 4.

1. \[ \sigma_X = \sqrt 2 \approx 1.414 \]
2. \[ z = \frac{5 - 3}{1.414} = 1.41 \]
3. \[ P(X \ge 5) = 1 - \Phi(1.41) = 1 - 0.9207 = 0.0793 \]
4. \[ z = \frac{2 - 3}{1.414} = -0.71 \]
5. \[ P(X \le 2) = \Phi(-0.71) = 1 - \Phi(0.71) = 1 - 0.7611 = 0.2389 \]

The 2025 HW1 key answered "0" to the matching question because no sampled age reached 80. The question asks for the normal-model probability: compute the z-score, use the table. That is what earns the marks.

1. \[ \boldsymbol\mu = \tfrac14\big((1,1) + (3,3) + (3,5) + (5,3)\big) = (3, 3) \]
2. \[ \mathbf Z = \begin{bmatrix} -2 & -2 \\ 0 & 0 \\ 0 & 2 \\ 2 & 0 \end{bmatrix} \]

Each column of \(\mathbf Z\) sums to zero. If yours does not, the mean is wrong.

1. \[ \mathbf Z^T\mathbf Z = \begin{bmatrix} 4 + 0 + 0 + 4 & 4 + 0 + 0 + 0 \\ 4 & 4 + 0 + 4 + 0 \end{bmatrix} = \begin{bmatrix} 8 & 4 \\ 4 & 8 \end{bmatrix} \]
2. \[ \Sigma = \begin{bmatrix} 2 & 1 \\ 1 & 2 \end{bmatrix} \]

1. \[ \det(\Sigma - \lambda I) = (2 - \lambda)^2 - 1 = 0 \]
2. \[ \lambda_1 = 3, \qquad \lambda_2 = 1 \]
3. For \(\lambda_1 = 3\): \(-v_1 + v_2 = 0\): \[ \mathbf v_1 = \tfrac{1}{\sqrt 2}\begin{bmatrix} 1 \\ 1 \end{bmatrix} \]
4. For \(\lambda_2 = 1\): \(v_1 + v_2 = 0\): \[ \mathbf v_2 = \tfrac{1}{\sqrt 2}\begin{bmatrix} 1 \\ -1 \end{bmatrix} \]
5. \(\mathbf v_1\) is the first principal component: it has the largest eigenvalue, so the projected data has the largest variance along it. Fraction captured: \[ \frac{3}{3 + 1} = 75\% \]

Either sign of an eigenvector is accepted. Both quiz keys say so explicitly.

1. \[ y_i = \frac{z_{i1} + z_{i2}}{\sqrt 2} \]
2. \[ \mathbf y = \Big(\tfrac{-4}{\sqrt2},\ 0,\ \tfrac{2}{\sqrt2},\ \tfrac{2}{\sqrt2}\Big) = (-2.828,\ 0,\ 1.414,\ 1.414) \]

Project the centred rows. Projecting the raw points adds \(\mathbf v_1^T\boldsymbol\mu = 4.243\) to every score. The variance comes out the same, but the coordinates and the reconstruction in (f) do not.

1. The scores have mean 0, so \[ \mathrm{Var}(y) = \tfrac14\big(8 + 0 + 2 + 2\big) = 3 = \lambda_1 \quad\checkmark \]

1. \[ \tilde{\mathbf x}_i = \boldsymbol\mu + y_i\,\mathbf v_1 \]
2. \[ y_i \mathbf v_1 = \tfrac{z_{i1} + z_{i2}}{2}\,(1, 1): \quad (-2,-2),\ (0,0),\ (1,1),\ (1,1) \]
3. \[ \tilde{\mathbf x} : \quad (1,1),\ (3,3),\ (4,4),\ (4,4) \]
4. Errors \(\mathbf x_i - \tilde{\mathbf x}_i\): \[ (0,0),\ (0,0),\ (-1,1),\ (1,-1) \]
5. \[ \mathrm{MSE} = \tfrac14\big(0 + 0 + 2 + 2\big) = 1 = \lambda_2 \]

Total variance is \(\lambda_1 + \lambda_2\). The first component keeps \(\lambda_1\), so what is lost per point is \(\lambda_2\). The formula sheet states this as \(\mathrm{SSE} = n\lambda_2\); here \(4 \times 1 = 4\), and \(\mathrm{MSE} = \mathrm{SSE}/n = 1\).

1. \[ \mathbf x_1^T\mathbf x_1 = 4, \qquad \mathbf x_1^T\mathbf x_2 = 2, \qquad \mathbf x_2^T\mathbf x_2 = 2 \]
2. \[ K_{11} = (1 + 4)^2 = 25, \qquad K_{12} = (1 + 2)^2 = 9, \qquad K_{22} = (1 + 2)^2 = 9 \]
3. \[ \mathbf K = \begin{bmatrix} 25 & 9 \\ 9 & 9 \end{bmatrix} \]

1. Lecture 5 slide 10, quadratic kernel with \(m = 2\): \[ \phi(\mathbf x) = \big(1,\ \sqrt2 x_1,\ \sqrt2 x_2,\ x_1^2,\ x_2^2,\ \sqrt2 x_1 x_2\big)^T \]
2. \[ \phi(\mathbf x_1) = (1,\ 2\sqrt2,\ 0,\ 4,\ 0,\ 0)^T \]
3. \[ \phi(\mathbf x_2) = (1,\ \sqrt2,\ \sqrt2,\ 1,\ 1,\ \sqrt2)^T \]
4. \[ \phi(\mathbf x_1)^T\phi(\mathbf x_2) = 1 + 4 + 0 + 4 + 0 + 0 = 9 = K_{12} \quad\checkmark \]

1. \[ \|\phi(\mathbf x_1)\| = \sqrt{K_{11}} = 5, \qquad \|\phi(\mathbf x_2)\| = \sqrt{K_{22}} = 3 \]
2. \[ \|\phi(\mathbf x_1) - \phi(\mathbf x_2)\|^2 = K_{11} + K_{22} - 2K_{12} = 25 + 9 - 18 = 16 \]
3. \[ \|\phi(\mathbf x_1) - \phi(\mathbf x_2)\| = 4 \]
4. \[ \cos\theta_\phi = \frac{K_{12}}{\sqrt{K_{11}K_{22}}} = \frac{9}{15} = 0.6 \]

1. \[ \|\boldsymbol\mu_\phi\|^2 = \frac{1}{n^2}\sum_{i,j} K_{ij} = \frac{25 + 9 + 9 + 9}{4} = 13 \]
2. \[ \|\boldsymbol\mu_\phi\| = \sqrt{13} \approx 3.606 \]
3. Row means \(\bar r_1 = 17\), \(\bar r_2 = 9\); grand mean \(\bar g = 13\). Use \(\hat K_{ij} = K_{ij} - \bar r_i - \bar r_j + \bar g\): \[ \hat K_{11} = 25 - 17 - 17 + 13 = 4 \]
4. \[ \hat K_{12} = 9 - 17 - 9 + 13 = -4 \]
5. \[ \hat K_{22} = 9 - 9 - 9 + 13 = 4 \]
6. \[ \hat{\mathbf K} = \begin{bmatrix} 4 & -4 \\ -4 & 4 \end{bmatrix} \]

Every row of a centred kernel matrix sums to zero. With two points the centred vectors are \(\pm\) each other, so \(\hat K_{12} = -\hat K_{11}\) is forced.

1. \[ \det(\hat{\mathbf K} - \eta I) = (4 - \eta)^2 - 16 = \eta(\eta - 8) = 0 \]
2. \[ \eta_1 = 8,\ \mathbf c_1 \propto \tfrac{1}{\sqrt2}(1, -1)^T; \qquad \eta_2 = 0,\ \mathbf c_2 = \tfrac{1}{\sqrt2}(1, 1)^T \]
3. Eigenvalues of \(\Sigma_\phi\) are \(\eta/n\): \[ \lambda_1 = \tfrac82 = 4, \qquad \lambda_2 = 0 \]
4. Coordinates \(a_i = \sqrt{\eta_1}\, v_1(i)\): \[ a_1 = \sqrt8 \cdot \tfrac{1}{\sqrt2} = 2, \qquad a_2 = -2 \]
5. Check: \(\tfrac12(4 + 4) = 4 = \lambda_1\), and \(a_1^2 = 4 = \hat K_{11}\). All the variance sits on one direction.

Signs may be flipped. The quiz key accepted \(-\mathbf c_1\) with negated coordinates.

1. \[ \mu_X = 3, \qquad \mu_Y = 6 \]
2. Centred: \(x - \mu_X = (-2,-1,0,1,2)\), \(y - \mu_Y = (-3,-2,0,1,4)\).
3. \[ w = \frac{\sum (x_i - \mu_X)(y_i - \mu_Y)}{\sum (x_i - \mu_X)^2} = \frac{6 + 2 + 0 + 1 + 8}{4 + 1 + 0 + 1 + 4} = \frac{17}{10} = 1.7 \]
4. \[ b = \mu_Y - w\,\mu_X = 6 - 1.7(3) = 0.9 \]
5. \[ \hat y = 0.9 + 1.7x \]

1. \[ \hat{\mathbf y} = (2.6,\ 4.3,\ 6.0,\ 7.7,\ 9.4) \]
2. \[ \boldsymbol\epsilon = \mathbf y - \hat{\mathbf y} = (0.4,\ -0.3,\ 0,\ -0.7,\ 0.6) \]
3. \[ \sum\epsilon_i = 0.4 - 0.3 + 0 - 0.7 + 0.6 = 0 \quad\checkmark \]
4. \[ \sum x_i\epsilon_i = 0.4 - 0.6 + 0 - 2.8 + 3.0 = 0 \quad\checkmark \]

They are the two derivative equations of the least-squares problem. Geometrically, the residual vector is perpendicular to both \(\mathbf 1\) and \(\mathbf x\), so \(\hat{\mathbf y}\) is the orthogonal projection of \(\mathbf y\) onto the plane they span.

1. \[ \mathrm{SSE} = 0.16 + 0.09 + 0 + 0.49 + 0.36 = 1.10 \]
2. \[ \mathrm{RMSE} = \sqrt{\tfrac{1.10}{5}} = \sqrt{0.22} \approx 0.469 \]
3. \[ \hat y(6) = 0.9 + 1.7(6) = 11.1 \]
4. \(x = 6\) is outside the observed range 1 to 5, so this is extrapolation: the line has no data there to support it.

The covariance formula assumes deviations from the mean. With uncentred data, \(\tfrac1n \mathbf X^T\mathbf X\) mixes in the mean (\(\boldsymbol\mu\boldsymbol\mu^T\)) and is not the covariance, so the eigenvectors point in the wrong directions. Projection scores must also be computed from \(\mathbf x_i - \boldsymbol\mu\), otherwise every score is shifted by \(\mathbf v^T\boldsymbol\mu\) and reconstructions are wrong.

Not necessarily. Independence implies zero correlation, but zero correlation only says there is no linear relationship. A nonlinear dependence (for example \(Y = X^2\) with \(X\) symmetric about 0) gives correlation 0 while \(Y\) is fully determined by \(X\). The converse holds only for jointly normal variables.

The mean jumps to \(49/7 = 7\) because every value enters the sum; the median stays at 3 because only the ordering matters and one extreme value does not change which point sits in the middle. The median is the robust measure of central tendency (Lecture 2 slide 15).

\(\Sigma_\phi\) has the size of the feature space, which is huge for polynomial kernels and infinite for the Gaussian kernel, and \(\phi\) is never computed. Every principal direction is a weighted sum of the mapped points, so the problem is rewritten in the \(n\) weights: \(\hat{\mathbf K}\mathbf c = \eta\mathbf c\), with variance \(\eta/n\). Only kernel values appear.