Questions 7–9: every weight starts at zero
x ∈ ℝᵈ → m sigmoid hidden layers (k neurons each) → 1 sigmoid output. All weights initialised to 0, no biases.
Q7: 0.5. Every pre-activation is 0, so every neuron outputs σ(0) = 0.5, including the output.
Q8: 0. With a linear output, ŷ = z = Σ 0 · 0.5 = 0.
Q9: option 4 (per the answer key). Option 2 is false: by symmetry, every neuron in a layer gets the same gradient.
Q8: 0. With a linear output, ŷ = z = Σ 0 · 0.5 = 0.
Q9: option 4 (per the answer key). Option 2 is false: by symmetry, every neuron in a layer gets the same gradient.
Caveat on Q9: with standard backprop, δ for each hidden layer is (W above)ᵀ·δ = 0 because those weights are zero, so W⁽¹⁾ gets exactly zero gradient on the first step and stays 0. Only the output weights move, and they all share the sign of (0.5 − y). The key's "all positive or all negative" reasoning fits the output weights, not W⁽¹⁾.