Do American rich kids do worse on international tests than rich kids from other countries?

Do American Rich Kids do Worse on International Tests than Rich Kids from Other Countries?      Stephen Krashen

Hanushek, Peterson and Woessmann (2014) claim that when we examine students from "advantaged" families, American students do poorly in math: Our rich kids do worse than rich kids from other countries.  Hanushek et. al. conclude that this shows that poverty is not the only factor affecting school performance.  Their conclusions are based on their analysis of data from the 2012 PIRLS examination, tests given to 15-year-olds in a large number of countries.

Berliner (2014) argued that Hanushek et. al. used an invalid measure of "advantaged": at least one parent who graduated college. He also argued that a more valid measure is income. Many college graduates, Berliner pointed out, are not in high-income professions.

I present here a secondary analysis of the PISA data presented by Hanushek et. al. to determine the relative influence of parental education and poverty on math and reading achievement, as measured by PISA.  For the most part, the results support Berliner's claim.

Method and Results

Measures of poverty

Two measures of poverty were used in this analysis, one for poverty in the individual states of the US and another for poverty in other countries. Both measures are based on parental income.  Berliner (2014) points out that parental income and school achievement are strongly related: parental income often determines the school the student attends (Berliner, 2014), and the nutrition and health care the student receives (Berliner, 2009). All of these influence school achievement.

Levels of poverty in the states in the US were taken from Kids Count (2012). Poverty was defined as the percentage of children under 18 who live in families with incomes below the US poverty threshold, as defined by the US Census Bureau.

The measure used internationally comes from a report from UNICEF, from the Innocenti Research Centre (2012).  It presents the percentage of children, ages zero to 17, who live in "relative poverty,"  that is, in a household with less than 50% of the national median income, adjusted for family size.

Results: The states

For 49 states in the US,  I examined the relationship between levels of poverty with math scores for students with high parental education. The correlation was negative and substantial: r = -.70 (p < .0001), as was the correlation of income and reading scores (r = -.71; p < .0001). In other words, when we control for the effect of parent education, poverty has a considerable negative impact on test performance in both math and reading: parental education does not tell the whole story.

Results: International comparisons

For the analysis of the international results, I discarded three outliers. Japan and Poland had high levels of poverty (one standard deviation above the mean) and very high scores in reading and math,  two standard deviations above the mean. Iceland was discarded because of it's very low level of poverty and very low reading score, nearly two standard deviations below the mean.  (See appendix table 2).

In International Analysis I, as was done in the states analysis, the impact of parental education was controlled by limiting the analysis to test scores made by those with high parental education (at least one parent a college graduate). Data on child poverty was available for 24 countries.  Correlations of child poverty and achievement were not as high as the results presented above for 49 states of the US, but the correlation was clearly negative for math (r = -.43, p = .02, one-tail). It was also negative but low for reading (r = -.18, ns, p = .20, one-tail).  The correlations for international tests may be lower than those for the states of the US because the definitions of parental education are probably not as consistent from country to country as they are among the states.

For International Analysis II, data was obtained on the percent of college graduates in 18 countries in 2008  (National Center for Educational Statistucs, 2011).  The correlation of percent of colleage graduates in each country and child poverty was in the predicted direction – more poverty was related to less parental education, but the correlation was not high and fell short of statistical significance (r = -.22, ns, p = .20), confirming that the two variables are not strongly related. Similar results were reported by Berliner (2014).

International Analysis III used PISA math and reading scores for all students. Mean PISA scores for each country were used as the dependent variable, with child poverty and percent of college graduates for each country as predictors. The correlation between poverty and total math scores was considerably larger than the correlation between percentage of college graduates and math scores (for poverty, r = - .54, p < .01, one-tail); for percentage of college graduates, r = .19, ns, p = .23, one-tail), and the correlation between poverty and math was fairly close to the correlation that controlled for parental education, presented earlier (r = -.43).

Multiple regression yielded similar results. As presented in table 1, poverty was a much better predictor of math scores than was parental education (compare betas) and was statististically significant.

Table 1: Mutliple regression analysis  for PISA math

	B	beta	P
Poverty	-1.67	0.53	0.02
Grad	0.09	0.069	0.38

r2 = .20

Results for reading PISA scores were somewhat different.  Both poverty and percentage of college graduates correlated with reading scores but the correlation of percentage of college graduates and reading scores (r = .54, p = .01, one-tail) was higher than the correlation of poverty and reaiding scores (r = -.35, p = .08, one-tail).

Again, multiple regression yielded results similar to the correlational analysis: As presented in table 2, the impact (beta) of percentage of college graduates in a state was higher than the impact of poverty, and the college graduate predictor was easily statistically significant, while the poverty predictor fell somewhat short of statistical significance (p = .14). 

Table 2: Multiple regression analysis for PISA reading

	b	beta	P
Poverty	-0.60	0.24	0.14
Grad	0.51	0.51	0.02

r2 = .35

Summary and Conclusions

The states analysis for both reading and math showed that when we control for the effect of parental education, the effect of poverty is powerful.

The first international analysis, controlling for parental education, showed that poverty has a clear effect on math scores, but less on reading scores, and the impact of poverty on math and reading was not as strong as it was in the states analysis.

The second international analysis revealed that parental education and poverty are only modestly correlated.

The third international analysis, using total PISA scores, graduation rates, and poverty, showed that parental education is a much weaker predictor than poverty for math achievement, but the international analysis using total PISA reading scores, graduation rates and poverty showed a clear effect for parental education (college graduation rates), and a weaker effect for poverty.

At least some of the reading results could stem from the failure to include an obvious factor: Access to books, a significant predictor of reading achievement independent of the impact of socio-economic status using several different measures of SES (Krashen, 2011; Krashen, Lee and McQuillan, 2012). Also, before we conclude that poverty is not a good predictor of reading achievement, recall that poverty was a strong predictor of reading test scores among the states in the US when parental education was controlled.

In summary,  the results provide support for Berliner's claim that "One’s level of education and one’s level of income simply do not provide the same information" and that "Parental income and their child’s school achievement are strongly related, perhaps even more so than is parental education level and their children’s school achievement."  Poverty, as measured by income, was a strong predictor of achievement in the states of the US, independent of the effect of parental education,  and was clearly a stronger predictor of math scores, according two different analysis.  Parental education was a stronger predictor of international reading scores, but poverty also a made a clear contribution.

References

Berliner, D. 2014. Criticism via Sleight of Hand http://dianeravitch.net/2014/07/29/david-berliner-responds-to-economists-who-discount-role-of-child-poverty/

Berliner, D. 2009. Poverty and Potential:  Out-of-School Factors and School Success.  Boulder and Tempe: Education and the Public Interest Center & Education Policy Research Unit. http://epicpolicy.org/publication/poverty-and-potential

Hanushek, E. A., Peterson, P. E., and Woessmann, L. 2014. Not just the problems of other people’s children: U.S. Student Performance in Global Perspective. Harvard University, Program on Education Policy and Governance & Education Next, PEPG Report No. 14-01, May 2014.

Kids Count Data Center, 2012. http://datacenter.kidscount.org/data/tables/43-children-in-poverty#detailed/2/2-52/false/868/any/321,322.

Krashen, S. 1997. Bridging inequity with books. Educational Leadership  55(4): 18-22.

Krashen, S., Lee, SY., and McQuillan, J. 2012. Is the library important? Multivariate studies at the national and international level. Journal of Language and Literacy Education, 8(1)? 26-36. (available at www.sdkrashen.com, see "free voluntary reading" section and scroll down)

National Center on Educational Statistics, 2011. Youth Indicators 2011.  First time college gradution rates among 30 OECD countries. http://nces.ed.gov/pubs2012/2012026/tables/table_23.asp

UNICEF Innocenti Research Centre 2012, Measuring Child Poverty: New league tables of child poverty in the world’s rich countries,, Innocenti Report Card 10, UNICEF Innocenti Research Centre, Florence.

Appendix

Table A1: PISA scores from 49 states in the USA

STATES	Math	reading	Poverty
Mass	62.3	59	15
Vermont	59.3	55.4	15
Minnesota	59	48.1	15
Colorado	58.1	52.3	18
New Jersey	57.9	55.6	15
Montana	57.5	48.7	20
Washington	54.3	49.2	19
Texas	54.2	40.4	26
New Hamp	53	47.8	16
Virgina	52.6	46.2	15
Wisconsin	52.6	45.7	18
Kansas	52.2	47.1	19
Maryland	52	51.2	14
S. Dakota	51.6	44.7	17
Connecticut	51.3	56.8	15
Pennsylvania	50.7	48.9	20
N. Dakota	50.2	39	13
Ohio	49.8	48.1	24
Idaho	49.6	44.8	21
Maine	49.4	49	21
Arizona	49.1	41.6	27
Wyoming	48.2	47.2	17
N. Carolina	48.1	39.9	26
Rhode Island	48	46.4	19
Utah	47.8	46.4	15
Indiana	46.9	40.7	22
Oregon	46.3	45.6	23
Illinois	45.6	46.6	21
Iowa	44.9	42.3	16
Nebraska	44.8	45.1	18
Kentucky	44.1	46.2	27
S. Carolina	43.1	34.1	27
California	42.6	36.3	24
Missouri	42.3	47	23
Michigan	42	40.6	25
Oklahoma	41.8	33.6	24
Nevada	41.7	35.6	24
Delaware	40.9	41.1	17
Arkansas	40.2	36.9	29
New York	39.8	46.9	23
Georgia	38.2	36.2	27
Hawaii	37.5	34.4	17
Florida	37.5	37.3	25
New Mexico	37.1	34.5	29
Tennessee	34.1	37	26
West Virgina	31.9	33.9	25
Alabama	27.8	34%	27
Mississippi	25.6	25.9	35
Louisiana	28.1	29.4	28

Table 2: International Data, At least one parent completed college

	pov	math	reading
Finland	5.3	51.8	50.4
netherlands	6.1	54.7	49
Slovenia	6.3	42.8	40.1
Germany	8.5	50	53.1
New Zealand	11.7	43.4	53.3
Ireland	8.4	43.6	53.2
France	8.8	42.4	53
Belgium	10.2	50.3	52
Australia	10.9	45	51.1
Canada	13.3	51	50.6
Portugal	14.7	38.8	47.3
Switzerland	8.1	57.3	47
Czech Rep	7.4	43.7	45.9
Estonia	11.9	51.3	44.9
Luxembourg	12.3	40.3	44.3
Hungary	10.3	33.1	43.9
Norway	6.1	39.3	43.6
UK	12.1	41.3	43.4
USA	23.1	34.7	41.6
Austria	7.3	46.3	40.6
Denmark	6.5	42.8	39.1
Spain	17.1	36.9	38.3
Italy	15.9	37.4	37.7
Sweden	7.3	34.6	35.7
mean	10.4	43.9	45.8
sd	4.3	6.6	5.5

Outliers

	poverty	math	reading
Japan	14.9	59.2	60.4
Poland	14.5	59.3	62.6
Iceland	4.7	45.6	33.9

Table A3: Total scores, graduation rates , and poverty

	poverty	math score	reading score	grad rate
Finland	5.3	519	524	62.6
Netherlands	6.1	523	511	41.4
Germany	8.5	514	508	25.5
New Zealand	11.7	500	512	48.3
Ireland	8.4	501	523	46.1
Portugal	14.7	487	488	45.3
Switzerland	8.1	531	509	32.4
Czech Rep	7.4	499	493	35.8
Luxembourg	12.3	490	488	5.3
Hungary	10.3	477	488	30.1
Norway	6.1	489	504	41.5
UK	17.1	494	499	34.9
USA	23.1	481	498	37.3
Austria	7.3	506	490	25
Denmark	6.5	500	496	46.8
Spain	17.1	484	488	33.1
Italy	15.9	485	490	32.8
Sweden	7.3	478	483	39.9
mean	10.7	497.7	499.6	36.9
sd	5	15.8	12.5	12.05