(a)(i) What are principal components ? Show that the principal components are uncorrelated. (10 marks) (a)(ii) Obtain the…

(a)

(i) Principal Components and Uncorrelatedness

Definition: Principal Components (PCs) are a set of new variables, Y₁, Y₂, …, Yₚ, which are linear combinations of the original random variables X₁, X₂, …, Xₚ. They are defined such that the first component explains the maximum variance, the second explains the maximum remaining variance, and so on. Mathematically, Yᵢ = aᵢ₁X₁ + … + aᵢₚXₚ, where the coefficients aᵢⱼ are the elements of the i-th eigenvector of the dispersion matrix Σ.

Proof of Uncorrelatedness: Let Yᵢ and Yⱼ be two distinct principal components (i ≠ j) corresponding to eigenvalues λᵢ and λⱼ. The covariance between them is: Cov(Yᵢ, Yⱼ) = aᵢ' Σ aⱼ Since aⱼ is an eigenvector of Σ, we know Σ aⱼ = λⱼ aⱼ. Substituting this into the equation: Cov(Yᵢ, Yⱼ) = aᵢ' (λⱼ aⱼ) = λⱼ (aᵢ' aⱼ) By the properties of symmetric matrices, eigenvectors corresponding to distinct eigenvalues are orthogonal, meaning aᵢ' aⱼ = 0. Therefore, Cov(Yᵢ, Yⱼ) = 0. This proves that the principal components are uncorrelated.

(ii) Calculation of Principal Components

1. Eigenvalues: We solve the characteristic equation |Σ - λ I| = 0:

| 4-λ & 2 & 1 2 & 3-λ & 1 1 & 1 & 2-λ | = 0 Expanding the determinant yields the cubic equation: λ³ - 9λ² + 20λ - 13 = 0. The roots (eigenvalues) are: λ₁ = 6.049 λ₂ = 1.643 λ₃ = 1.308 (Note: The sum of eigenvalues equals the trace of the matrix: 6.049 + 1.643 + 1.308 = 9.0).

2. Variation Explained: Total Variance = 9.

• PC1: (6.049 / 9) × 100 = 67.2%
• PC2: (1.643 / 9) × 100 = 18.3%
• PC3: (1.308 / 9) × 100 = 14.5%

3. Eigenvectors and components: Solving (Σ − λI)a = 0 and normalising (a'a = 1): Y₁ = 0.737X₁ + 0.591X₂ + 0.328X₃ (λ₁ = 6.049) Y₂ = 0.591X₁ − 0.328X₂ − 0.737X₃ (λ₂ = 1.643) Y₃ = 0.328X₁ − 0.737X₂ + 0.591X₃ (λ₃ = 1.308) The eigenvectors are mutually orthogonal, as the proof in (i) requires.

Comment: The first principal component accounts for 67.2% of the total variation, and the first two together for 85.5%. Y₁ has all positive loadings, so it is a weighted overall "size" index dominated by X₁ and X₂; Y₂ contrasts X₁ with X₃. Since λ₂ and λ₃ are close, Y₂ and Y₃ are not sharply separated, so a one-dimensional summary through Y₁ is reasonable, but retaining two components gives a fuller picture.

(b) Missing Value Estimation and ANOVA

1. Estimation of Missing Value (y): For a Randomized Block Design (RBD) with t treatments and b blocks, the formula for a missing observation y is: y = (t T'_B + b B'_II - G')/((t-1)(b-1)) Where: t = 3 (Treatments A, B, C) b = 3 (Blocks I, II, III) T'_B = 20 + 18 = 38 (Treatment B total excluding y) B'_II = 24 + 25 = 49 (Block II total excluding y) G' = 22+24+22+20+18+21+25+20 = 172 (Grand total excluding y)

y = (3(38) + 3(49) - 172)/((3-1)(3-1)) = (114 + 147 - 172)/4 = 89/4 = 22.25

2. ANOVA Table: The completed Grand Total G = 172 + 22.25 = 194.25. Degrees of Freedom (d.f.) are adjusted: Total d.f. = 9-1-1=7. One d.f. is lost to the estimation of y, so Error d.f. = 7 - 2 - 2 = 3.

• Treatments: d.f. — 2; Sum of Squares (SS) — 10.7917; Mean Square (MS) — 5.3958; F-ratio — 5.25
• Blocks: d.f. — 2; Sum of Squares (SS) — 22.6250; Mean Square (MS) — 11.3125; F-ratio — 11.01
• Error: d.f. — 3; Sum of Squares (SS) — 3.0833; Mean Square (MS) — 1.0278; F-ratio —
• Total: d.f. — 7; Sum of Squares (SS) — 36.5000; Mean Square (MS) — ; F-ratio —

3. Conclusion: (The treatment sum of squares from the filled-in table is slightly biased upward. The bias is [B′ − (t−1)y]²/[t(t−1)] = (49 − 44.5)²/6 = 3.375, so the corrected treatment SS is 7.417 and F = 3.61, which leads to the same conclusion. F(3, 4) is not needed for a 3 × 3 layout.)

• Treatments: F_calc = 5.25. The critical value F(2, 3) = 9.55. Since 5.25 < 9.55, the difference between treatments is not significant.
• Blocks: F_calc = 11.01. The critical value F(2, 3) = 9.55. Since 11.01 > 9.55, the difference between blocks is significant.

(c) Sampling vs. Non-sampling Errors

1. Distinction and Sources:

• Definition: Sampling Error — Chance fluctuation between sample statistic and population parameter.; Non-sampling Error — Bias arising from factors other than sampling.
• Sources: Sampling Error — Random selection, sample size, sampling design.; Non-sampling Error — Specification, coverage, non-response, measurement, processing.
• Control: Sampling Error — Can be reduced by increasing sample size (n) or using stratification.; Non-sampling Error — Controlled by training, call-backs, and data validation.
• Quantification: Sampling Error — Can be estimated (Standard Error).; Non-sampling Error — Generally cannot be quantified.

2. Control Measures: Sampling errors are controlled mathematically by increasing the sample size or using more efficient designs like stratified random sampling. Non-sampling errors are controlled through administrative measures, such as rigorous training of enumerators, conducting follow-up interviews for non-respondents, and implementing data validation checks to minimize measurement and processing errors.