Sunday, August 03, 2014

Do American rich kids do worse on international tests than rich kids from other countries?


Do American Rich Kids do Worse on International Tests than Rich Kids from Other Countries?      Stephen Krashen

Hanushek, Peterson and Woessmann (2014) claim that when we examine students from "advantaged" families, American students do poorly in math: Our rich kids do worse than rich kids from other countries.  Hanushek et. al. conclude that this shows that poverty is not the only factor affecting school performance.  Their conclusions are based on their analysis of data from the 2012 PIRLS examination, tests given to 15-year-olds in a large number of countries.

Berliner (2014) argued that Hanushek et. al. used an invalid measure of "advantaged": at least one parent who graduated college. He also argued that a more valid measure is income. Many college graduates, Berliner pointed out, are not in high-income professions.

I present here a secondary analysis of the PISA data presented by Hanushek et. al. to determine the relative influence of parental education and poverty on math and reading achievement, as measured by PISA.  For the most part, the results support Berliner's claim.

Method and Results

Measures of poverty

Two measures of poverty were used in this analysis, one for poverty in the individual states of the US and another for poverty in other countries. Both measures are based on parental income.  Berliner (2014) points out that parental income and school achievement are strongly related: parental income often determines the school the student attends (Berliner, 2014), and the nutrition and health care the student receives (Berliner, 2009). All of these influence school achievement.

Levels of poverty in the states in the US were taken from Kids Count (2012). Poverty was defined as the percentage of children under 18 who live in families with incomes below the US poverty threshold, as defined by the US Census Bureau.
The measure used internationally comes from a report from UNICEF, from the Innocenti Research Centre (2012).  It presents the percentage of children, ages zero to 17, who live in "relative poverty,"  that is, in a household with less than 50% of the national median income, adjusted for family size.

Results: The states

For 49 states in the US,  I examined the relationship between levels of poverty with math scores for students with high parental education. The correlation was negative and substantial: r = -.70 (p < .0001), as was the correlation of income and reading scores (r = -.71; p < .0001). In other words, when we control for the effect of parent education, poverty has a considerable negative impact on test performance in both math and reading: parental education does not tell the whole story.

Results: International comparisons

For the analysis of the international results, I discarded three outliers. Japan and Poland had high levels of poverty (one standard deviation above the mean) and very high scores in reading and math,  two standard deviations above the mean. Iceland was discarded because of it's very low level of poverty and very low reading score, nearly two standard deviations below the mean.  (See appendix table 2).

In International Analysis I, as was done in the states analysis, the impact of parental education was controlled by limiting the analysis to test scores made by those with high parental education (at least one parent a college graduate). Data on child poverty was available for 24 countries.  Correlations of child poverty and achievement were not as high as the results presented above for 49 states of the US, but the correlation was clearly negative for math (r = -.43, p = .02, one-tail). It was also negative but low for reading (r = -.18, ns, p = .20, one-tail).  The correlations for international tests may be lower than those for the states of the US because the definitions of parental education are probably not as consistent from country to country as they are among the states.
For International Analysis II, data was obtained on the percent of college graduates in 18 countries in 2008  (National Center for Educational Statistucs, 2011).  The correlation of percent of colleage graduates in each country and child poverty was in the predicted direction – more poverty was related to less parental education, but the correlation was not high and fell short of statistical significance (r = -.22, ns, p = .20), confirming that the two variables are not strongly related. Similar results were reported by Berliner (2014).

International Analysis III used PISA math and reading scores for all students. Mean PISA scores for each country were used as the dependent variable, with child poverty and percent of college graduates for each country as predictors. The correlation between poverty and total math scores was considerably larger than the correlation between percentage of college graduates and math scores (for poverty, r = - .54, p < .01, one-tail); for percentage of college graduates, r = .19, ns, p = .23, one-tail), and the correlation between poverty and math was fairly close to the correlation that controlled for parental education, presented earlier (r = -.43).

Multiple regression yielded similar results. As presented in table 1, poverty was a much better predictor of math scores than was parental education (compare betas) and was statististically significant.

Table 1: Mutliple regression analysis  for PISA math
        B
beta
P
Poverty
-1.67
0.53
0.02
Grad
0.09
0.069
0.38
r2 = .20

Results for reading PISA scores were somewhat different.  Both poverty and percentage of college graduates correlated with reading scores but the correlation of percentage of college graduates and reading scores (r = .54, p = .01, one-tail) was higher than the correlation of poverty and reaiding scores (r = -.35, p = .08, one-tail).

Again, multiple regression yielded results similar to the correlational analysis: As presented in table 2, the impact (beta) of percentage of college graduates in a state was higher than the impact of poverty, and the college graduate predictor was easily statistically significant, while the poverty predictor fell somewhat short of statistical significance (p = .14). 

Table 2: Multiple regression analysis for PISA reading
b
beta
P
Poverty
-0.60
0.24
0.14
Grad
0.51
0.51
0.02
r2 = .35

Summary and Conclusions

The states analysis for both reading and math showed that when we control for the effect of parental education, the effect of poverty is powerful.

The first international analysis, controlling for parental education, showed that poverty has a clear effect on math scores, but less on reading scores, and the impact of poverty on math and reading was not as strong as it was in the states analysis.

The second international analysis revealed that parental education and poverty are only modestly correlated.

The third international analysis, using total PISA scores, graduation rates, and poverty, showed that parental education is a much weaker predictor than poverty for math achievement, but the international analysis using total PISA reading scores, graduation rates and poverty showed a clear effect for parental education (college graduation rates), and a weaker effect for poverty.

At least some of the reading results could stem from the failure to include an obvious factor: Access to books, a significant predictor of reading achievement independent of the impact of socio-economic status using several different measures of SES (Krashen, 2011; Krashen, Lee and McQuillan, 2012). Also, before we conclude that poverty is not a good predictor of reading achievement, recall that poverty was a strong predictor of reading test scores among the states in the US when parental education was controlled.

In summary,  the results provide support for Berliner's claim that "One’s level of education and one’s level of income simply do not provide the same information" and that "Parental income and their child’s school achievement are strongly related, perhaps even more so than is parental education level and their children’s school achievement."  Poverty, as measured by income, was a strong predictor of achievement in the states of the US, independent of the effect of parental education,  and was clearly a stronger predictor of math scores, according two different analysis.  Parental education was a stronger predictor of international reading scores, but poverty also a made a clear contribution.



References
Berliner, D. 2009. Poverty and Potential:  Out-of-School Factors and School Success.  Boulder and Tempe: Education and the Public Interest Center & Education Policy Research Unit. http://epicpolicy.org/publication/poverty-and-potential
Hanushek, E. A., Peterson, P. E., and Woessmann, L. 2014. Not just the problems of other people’s children: U.S. Student Performance in Global Perspective. Harvard University, Program on Education Policy and Governance & Education Next, PEPG Report No. 14-01, May 2014.
Kids Count Data Center, 2012. http://datacenter.kidscount.org/data/tables/43-children-in-poverty#detailed/2/2-52/false/868/any/321,322.
Krashen, S. 1997. Bridging inequity with books. Educational Leadership  55(4): 18-22.
Krashen, S., Lee, SY., and McQuillan, J. 2012. Is the library important? Multivariate studies at the national and international level. Journal of Language and Literacy Education, 8(1)? 26-36. (available at www.sdkrashen.com, see "free voluntary reading" section and scroll down)
National Center on Educational Statistics, 2011. Youth Indicators 2011.  First time college gradution rates among 30 OECD countries. http://nces.ed.gov/pubs2012/2012026/tables/table_23.asp
UNICEF Innocenti Research Centre 2012, Measuring Child Poverty: New league tables of child poverty in the world’s rich countries,, Innocenti Report Card 10, UNICEF Innocenti Research Centre, Florence.
















Appendix

Table A1: PISA scores from 49 states in the USA
STATES
Math
reading
Poverty
Mass
62.3
59
15
Vermont
59.3
55.4
15
Minnesota
59
48.1
15
Colorado
58.1
52.3
18
New Jersey
57.9
55.6
15
Montana
57.5
48.7
20
Washington
54.3
49.2
19
Texas
54.2
40.4
26
New Hamp
53
47.8
16
Virgina
52.6
46.2
15
Wisconsin
52.6
45.7
18
Kansas
52.2
47.1
19
Maryland
52
51.2
14
S. Dakota
51.6
44.7
17
Connecticut
51.3
56.8
15
Pennsylvania
50.7
48.9
20
N. Dakota
50.2
39
13
Ohio
49.8
48.1
24
Idaho
49.6
44.8
21
Maine
49.4
49
21
Arizona
49.1
41.6
27
Wyoming
48.2
47.2
17
N. Carolina
48.1
39.9
26
Rhode Island
48
46.4
19
Utah
47.8
46.4
15
Indiana
46.9
40.7
22
Oregon
46.3
45.6
23
Illinois
45.6
46.6
21
Iowa
44.9
42.3
16
Nebraska
44.8
45.1
18
Kentucky
44.1
46.2
27
S. Carolina
43.1
34.1
27
California
42.6
36.3
24
Missouri
42.3
47
23
Michigan
42
40.6
25
Oklahoma
41.8
33.6
24
Nevada
41.7
35.6
24
Delaware
40.9
41.1
17
Arkansas
40.2
36.9
29
New York
39.8
46.9
23
Georgia
38.2
36.2
27
Hawaii
37.5
34.4
17
Florida
37.5
37.3
25
New Mexico
37.1
34.5
29
Tennessee
34.1
37
26
West Virgina
31.9
33.9
25
Alabama
27.8
34%
27
Mississippi
25.6
25.9
35
Louisiana
28.1
29.4
28


Table 2: International Data, At least one parent completed college
pov
math
reading
Finland
5.3
51.8
50.4
netherlands
6.1
54.7
49
Slovenia
6.3
42.8
40.1
Germany
8.5
50
53.1
New Zealand
11.7
43.4
53.3
Ireland
8.4
43.6
53.2
France
8.8
42.4
53
Belgium
10.2
50.3
52
Australia
10.9
45
51.1
Canada
13.3
51
50.6
Portugal
14.7
38.8
47.3
Switzerland
8.1
57.3
47
Czech Rep
7.4
43.7
45.9
Estonia
11.9
51.3
44.9
Luxembourg
12.3
40.3
44.3
Hungary
10.3
33.1
43.9
Norway
6.1
39.3
43.6
UK
12.1
41.3
43.4
USA
23.1
34.7
41.6
Austria
7.3
46.3
40.6
Denmark
6.5
42.8
39.1
Spain
17.1
36.9
38.3
Italy
15.9
37.4
37.7
Sweden
7.3
34.6
35.7
mean
10.4
43.9
45.8
sd
4.3
6.6
5.5


Outliers
poverty
math
reading
Japan
14.9
59.2
60.4
Poland
14.5
59.3
62.6
Iceland
4.7
45.6
33.9



Table A3: Total scores, graduation rates , and poverty
poverty
math score
reading score
grad rate
Finland
5.3
519
524
62.6
Netherlands
6.1
523
511
41.4
Germany
8.5
514
508
25.5
New Zealand
11.7
500
512
48.3
Ireland
8.4
501
523
46.1
Portugal
14.7
487
488
45.3
Switzerland
8.1
531
509
32.4
Czech Rep
7.4
499
493
35.8
Luxembourg
12.3
490
488
5.3
Hungary
10.3
477
488
30.1
Norway
6.1
489
504
41.5
UK
17.1
494
499
34.9
USA
23.1
481
498
37.3
Austria
7.3
506
490
25
Denmark
6.5
500
496
46.8
Spain
17.1
484
488
33.1
Italy
15.9
485
490
32.8
Sweden
7.3
478
483
39.9
mean
10.7
497.7
499.6
36.9
sd
5
15.8
12.5
12.05



at

No comments:

Post a Comment