SAT Coaching, Bias and Causal Inference
Publisher: Dissertation Abstracts International, DAI-A 64/12, 4433 (UMI No. 3115515)
This study considers the extent to which unbiased causal inferences can be drawn about the effect of coaching on SAT performance. Following a review of the literature, the author presents the linear regression model and the Heckman Model as two statistical approaches that might be used control for bias in an estimated coaching effect. The assumptions necessary before an estimated effect can be given a causal interpretation are described in some detail. The author estimates coaching effects for both sections of the SAT using data from the National Education Longitudinal Study of 1988 (NELS). There is some indication that the linear regression model successfully reduces bias due to omitted variables. It appears that commercial coaching programs have an effect of about 3 to 20 points on the verbal section of the SAT, and an effect of about 10 to 28 points on the math section of the SAT. These effects may be somewhat bigger or smaller if coaching is defined more broadly. There is some evidence that coaching is more effective for certain types of students. The author demonstrates the sensitivity of the Heckman Model to the choice of variables included in the selection function. Small changes in the selection function are shown to have a big impact on estimated coaching effects.