(a) For a simple linear regression model Y = β₀ + β₁Xᵢ + εᵢ, i = 1, ..., n (i) Derive the least square estimators of β₀ and β₁…

(a)(i) Least squares estimators

Assumptions: the model is linear in β₀, β₁; X is fixed (non-random) with not all Xᵢ equal; E(εᵢ) = 0; Var(εᵢ) = σ²; the εᵢ are uncorrelated. (Normality is needed only for inference, not for least squares.)

Minimise S = Σ(Yᵢ − β₀ − β₁Xᵢ)². Setting ∂S/∂β₀ = 0 and ∂S/∂β₁ = 0 gives the normal equations ΣYᵢ = nβ₀ + β₁ΣXᵢ, ΣXᵢYᵢ = β₀ΣXᵢ + β₁ΣXᵢ². Solving, β̂₁ = Σ(Xᵢ − X̄)(Yᵢ − Ȳ)/Σ(Xᵢ − X̄)², β̂₀ = Ȳ − β̂₁X̄. The second derivatives form a positive-definite matrix when Σ(Xᵢ − X̄)² > 0, so this is a minimum.

(a)(ii) Properties of residuals

The normal equations say Σ(Yᵢ − β̂₀ − β̂₁Xᵢ) = 0 and ΣXᵢ(Yᵢ − β̂₀ − β̂₁Xᵢ) = 0, that is Σeᵢ = 0 and ΣXᵢeᵢ = 0.

1. Σeᵢ = 0 (first normal equation).
2. Σeᵢ = ΣYᵢ − ΣŶᵢ = 0, so ΣYᵢ = ΣŶᵢ.
3. ΣXᵢeᵢ = 0 (second normal equation).
4. ΣŶᵢeᵢ = β̂₀Σeᵢ + β̂₁ΣXᵢeᵢ = 0.
5. Dividing the first normal equation by n gives Ȳ = β̂₀ + β̂₁X̄, so the line passes through (X̄, Ȳ).

(b) Inequalities for a BIBD

Relations: vr = bk, λ(v − 1) = r(k − 1), with k < v (incomplete design) and Fisher's inequality b ≥ v. (i) Since k < v, λ(v − 1) = r(k − 1) gives r > λ, so r ≥ λ + 1, i.e. r + 1 ≥ λ + 2. Also bk = vr with v > k gives b > r, so b ≥ r + 1. Hence b ≥ r + 1 ≥ λ + 2. (ii) v ≤ b is Fisher's inequality. For the upper bound, put v = 1 + r(k − 1)/λ in b = vr/k: b = r[λ + r(k − 1)]/(λk) = r²/λ − r(r − λ)/(λk). Then (r² − 1)/λ − b = [r(r − λ) − k]/(λk). From b ≥ v we get r ≥ k, and r − λ ≥ 1, so r(r − λ) ≥ r ≥ k. Hence (r² − 1)/λ − b ≥ 0, i.e. b ≤ (r² − 1)/λ. So v ≤ b ≤ (r² − 1)/λ.

(c) Coefficient of X₁ when X₂ is added

Work with deviations from means. For Y on X₁ alone, β̂₁ = S₁y/S₁₁. For Y on X₁ and X₂, the normal equations are S₁₁β̂₁ + S₁₂β̂₂ = S₁y, S₁₂β̂₁ + S₂₂β̂₂ = S₂y. If X₁ and X₂ are uncorrelated, S₁₂ = Σ(X₁ᵢ − X̄₁)(X₂ᵢ − X̄₂) = 0, so the first equation becomes S₁₁β̂₁ = S₁y, giving β̂₁ = S₁y/S₁₁, the same as before. (Likewise β̂₂ = S₂y/S₂₂.) Adding X₂ leaves β̂₁ unchanged; it only reduces the residual sum of squares by β̂₂²S₂₂.

(d) Sub-sampling estimator

Each of ȳ₁ and ȳ₂ is the mean of a simple random sample without replacement from the population (the n₂ units are the rest of the n-unit sample), so E(ȳ₁) = E(ȳ₂) = Ȳ_N. Hence E(Ŷ_N) = wȲ_N + (1 − w)Ȳ_N = Ȳ_N.

With S² the population mean square, Var(ȳ₁) = (1/n₁ − 1/N)S² and Var(ȳ₂) = (1/n₂ − 1/N)S². The overall mean ȳ = (n₁ȳ₁ + n₂ȳ₂)/n has variance (1/n − 1/N)S²; equating this to the expansion of the variance of the combination gives Cov(ȳ₁, ȳ₂) = −S²/N. Therefore Var(Ŷ_N) = S²[w²(1/n₁ − 1/N) + (1 − w)²(1/n₂ − 1/N) − 2w(1 − w)/N] = S²[w²/n₁ + (1 − w)²/n₂ − 1/N]. This is smallest for w = n₁/n, when it equals (1/n − 1/N)S², the variance of the mean of the full sample.

(e) Efficiency of RBD over CRD

The efficiency of one design relative to another with the same number of experimental units is the ratio of the variances of an estimated treatment contrast, which for the same treatment replication is the ratio of their error variances.

For an RBD with t treatments and r blocks, the CRD error variance is estimated from the total mean square after removing treatments, with block and error sums of squares making up the unit-to-unit variation: σ̂²_CRD = [(r − 1)MSB + r(t − 1)MSE]/(rt − 1), where MSB and MSE are the block and error mean squares of the RBD (the within-block sum of squares is taken as r(t − 1)MSE, as treatment effects are small). Hence E(RBD over CRD) = [(r − 1)MSB + r(t − 1)MSE] / [(rt − 1)MSE]. When the error degrees of freedom differ, Fisher's correction multiplies this by (f₁ + 1)(f₂ + 3)/[(f₁ + 3)(f₂ + 1)], where f₁ = (r − 1)(t − 1) and f₂ = t(r − 1). A value above 1 means blocking has reduced the error and the RBD is more efficient.