S3U

Reading / Compare and contrast / Graduate / rce01

Synthesize Estimates Without Hiding Heterogeneity

Evaluate a pooled estimate and the assumptions behind its weights.

All worksheets
5 questions0m 0s

Prerequisites

Effect estimates, sampling variances, inverse-variance weighting, and research synthesis.

Learn the skill

For independent estimates of a common effect, inverse-variance weights give theta_hat = sum(w_i theta_i)/sum(w_i), with w_i = 1/v_i. Independence, compatible estimands, and a common-effect assumption need justification before pooling.

Worked example

If equally precise studies estimate different populations or outcomes, equal numerical weights do not make their estimands interchangeable. A pooled number can conceal the question it answers.

Model and Assumptions

Two fictional independent studies estimate effects on the same stated scale: theta1 = 2 with variance v1 = 1; theta2 = 8 with variance v2 = 4. A common-effect calculation uses inverse-variance weights. A proposed random-effects calculation adds a stipulated tau^2 = 3 to each variance; tau^2 is not estimated here.

1. What are the normalized common-effect weights?
2. What is the common-effect pooled estimate?
3. With the stipulated tau^2 = 3, what are the normalized weights?
4. What if the studies reuse some of the same participants?
5. Does adding tau^2 make incompatible outcomes comparable?

Further inquiry

Derive both pooled estimates and their model-based variances with the stipulated weights. Write a synthesis decision memo addressing outcome compatibility, dependence, and why two studies give limited information about heterogeneity.

Review criteria

  • Use raw and normalized weights consistently.
  • State that tau^2 was supplied, not reliably inferred here.
  • Separate numerical pooling from substantive comparability.