Philip Cohen Misses the Mark

Sociologist Philip Cohen has published a critique of our new report, Stronger Families, Better Schools. Unfortunately, because he did not ask us for our data before publishing his critique and did not analyze the complete data used in the report, he misses the mark in his assessment.

For his critique of our report on Arizona families and schools, Cohen drew on data from Table A2 in the Appendix of the online version of the report on the Institute for Family Studies website. Regrettably, our graphic designer accidentally failed to include 16 of the 99 cases in the online version of Table A2. (All the cases in Table A2 were available in the online PDF of the report and are now also available on the website.) So Cohen’s analyses of the data are based on an incomplete sample. Not surprisingly, he was unable to replicate our results. He then accused us of conducting analyses that were “complete bologna—as in, fraudulent or error-ridden.”

Had Cohen given us the professional courtesy of asking for the complete data on the 99 school districts in Arizona and then analyzing that data, he would have found that our analyses are indeed correct. (We have sent him the data so that he is free to replicate our analyses.) For instance, our Stata analysis of the full data for high school graduation rates in Arizona matches Table 2 from the report (see below).

Stata Output:

Likewise, our Stata analysis of the full data for gender ratios in Arizona high school graduation rates matches Table 4 from the report (see below).

Stata Output:

Clearly, the substance of Philip Cohen’s critique is incorrect, although he was right to pick up on 16 missing cases in our online Appendix. As anyone who follows his work knows, Cohen is quick to accuse others of faulty methods and ill intent on Twitter and on his blog, especially when the researchers differ from him philosophically. As our response here suggests, readers of his blog should be aware that Cohen’s own ideological commitments sometimes lead him to miss his methodological mark (see here and here for another example).