21st century boondoggle?

High Tech Testing on the Way: a 21st Century Boondoggle?

Stephen Krashen and Susan Ohanian

http://blogs.edweek.org/teachers/living-in-dialogue/2011/04/high_tech_testing_on_the_way_a.html

When the plans to create Common Core Standards were announced, Secretary Duncan told us that it would be accompanied by assessments to enforce the standards. We were also told that developing standards would be relatively inexpensive, but developing assessments, by contrast, will be a "very heavy lift financially" (USA Today, June 14, 2009).

It is gradually becoming clear that the lift will be extremely heavy. The new tests will be computer-based, administered online, and "will make widespread use of smart technology. They will provide students with realistic, complex performance tasks, immediate feedback, computer adaptive testing, and incorporate accommodations for a range of students" (Duncan, 2010). Duncan noted that "with the benefit of technology, assessment questions can incorporate audio and video. Problems can be situated in real-world environments, where students perform tasks or include multi-stage scenarios and extended essays."

An example

The National Education Technology Plan 2010 (U.S. Department of Education; Office of Educational Technology) describes one kind of testing that is being developed, testing that takes place "in the course of learning" (xvii) and that tries to find out what students are thinking while doing projects:

"As students work, the system can capture their inputs and collect evidence of their problem-solving sequences, knowledge, and strategy use, as reflected by the information each student selects or inputs, the number of attempts the student makes, the number of hints and type of feedback given, and the time allocation across parts of the problem" (pages 29-30: "Assessing during online learning").

Aside from the mind-control aspect of this kind of testing, how much will it cost, in addition to the cost of developing, testing and revising the new tests?

If we are going to have computer-based tests, and if they are to be delivered to students via the internet, the first requirement is that all students need to be connected to the internet. A recent article in the New York Times gives us some idea of what will be involved. The article begins by noting that money is scarce these days:

"Despite sharp drops in state aid, New York City’s Department of Education plans to increase its technology spending, including $542 million next year alone that will primarily pay for wiring and other behind-the-wall upgrades to city schools … and $315 million for additional schools by 2014…" (New York Times, "In city schools, tech spending to rise despite cuts," March 30, 2011)

Buried deep the article is a statement by "city officials" that the huge expenditures for technology are primarily to make it possible for students to take computerized national standardized tests.

We can expect this to happen nation-wide. If the New York figure is extrapolated to the entire country, the cost to connect all children to the internet will be at least 50 times the cost of connecting New York City alone, or $25 billion (New York City enrolls one million students, the USA as a whole, over 60 million). This is only to connect students to the internet. The whistles and bells needed to do "computer adaptive testing" with audio and video will cost more.

Technology, of course, continues to develop all the time, and consumers have repeatedly demonstrated their willingness to discard the old and embrace the new, even at considerable expense. We can expect that after every student is connected, sooner or later the set-up will become obsolete and need to be replaced, either in part or totally. The schools, we predict, will cheerfully pay up, eager for the "newest" technology, and the computer companies will cheerfully accept their money.

The billions spent so that students can take national tests will have a huge payoff for the entire computer industry in other ways. This was enthusiastically announced by Education Secretary Duncan's Chief of Staff and former CEO of the New Schools Venture Fund, Joanne Weiss. Weiss noted that because all students will have internet access in order to be tested, technology companies can now profit from one giant national market for all their educational products:

"The development of common standards and shared assessments radically alters the market for innovation in curriculum development, professional development, and formative assessments. Previously, these markets operated on a state-by-state basis, and often on a district-by-district basis. But the adoption of common standards and shared assessments means that education entrepreneurs will enjoy national markets where the best products can be taken to scale" (Weiss, 2011).

WHY THIS IS A BAD IDEA

We have nothing against private enterprise making an honest profit for providing a needed service or useful product. But in this case there is no evidence that the product is needed nor is there evidence that it will be useful. Just because something is high-tech doesn't mean it is good. As Gerald Bracey pointed out, computers have made it possible to do in nano-seconds what shouldn't be done at all. We are about to waste a gigantic amount of money when it is badly needed elsewhere, and where it can be put to much better use.

No Evidence for Standards/Test Approach

First, there is no evidence supporting the idea that tests to enforce national standards, no matter how subtle and refined, will have any positive impact on student learning. In fact, the evidence we have suggests that it will not: States that use more high-stakes tests do not do better on the NAEP tests than states with fewer (Nichols, Glass and Berliner, 2006), and the use of the standardized SAT does not predict college success over and above high school grades (Bowen, Chingos, and McPherson, 2009; Geiser and Santelices, 2007).

Of course, the administration has argued that these will be new and better tests, more sensitive to growth in learning, able to chart student progress through the year, and able to probe real learning, not just memorization. Before unleashing these "improved" tests on the country, however, there should be rigorous investigation, rigorous studies to show that these measures are worth the investment. Right now, the corporations and politicians insist that we take on faith the claim that these tests are good for students. Such claims exhibit a profound lack of accountability.

Second, there is overwhelming evidence that dealing with poverty is an excellent investment, one that will not only improve school achievement but also affect quality of life and personal happiness.

· There is very good evidence that our international tests scores are "low" because of poverty. Studies show that middle-class American students in well-funded schools score at the top of the world on international tests. Our overall average is less than spectacular because we have such a high percentage of children living in poverty, at least 20%, the highest among all industrialized countries (Berliner, 2011). In urban areas, where test scores are the lowest, the poverty level is much higher: 51% in Cleveland and Detroit, 37% in Miami, and 35% in Dallas and New Orleans (http://datacenter.kidscount.org/data/acrossstates/Rankings.aspx?loct=3&by=a&order=a&ind=43&dtm=322&tf=38) (These figures are based on the federal poverty level. If we consider the percentage of children eligible for free and reduced lunch, between 130 and 185% of the federal level, the figure is much higher, with 81% of children in Detroit and 68% in Miami living in poverty. According to the National Center for Children in Poverty (see e.g. http://www.nccp.org/faq.html), families need an income of twice (200%) the official federal level to meet basic needs. Nearly 40% of children in the US live in families with incomes of less than 200% of the official federal level.)

Studies show that children living in poverty suffer from conditions shown to impact educational attainment and school performance, such as "food insecurity," environmental toxins, lack of health care, and lack of access to books. The impact of these factors is enormous: No matter how good teaching is or how carefully a curriculum is put together, it will be of little value when students are hungry, malnourished, in poor health, and when they have little or no access to reading material.

· When these conditions are dealt with and alleviated, school performance improves. Providing food for hungry children has been shown to produce dramatic differences in behavior and performance in school (Berliner, 2009), having medical insurance improves school performance (Berliner, 2009), and increasing access to books as well as providing time to read for pleasure results in better literacy development (Shin and Krashen, 2009).

The obvious cure for poverty is full employment, with a living wage paid for honest work. Our society today provides neither of these, with unemployment high and with wages low: As of this writing, the average pay for a retail sales position, about $20,000 per year, is well below the federal poverty line for a family of four (Gibson, 2011).

The obvious step to take now is to make sure children are protected from the effects of poverty. A great deal of this, e.g. improving libraries in high poverty areas to the point where children of poverty have access to a print-rich environment, can be done at a fraction of the cost schools will be required to spend just on the technology aspect of the new testing program.

The Department of Education plans to use American students as experimental subjects to try out an extremely expensive, time-consuming and dubious testing program that will engulf classrooms. If it fails, the effect on students will be devastating, with schools robbed of money, and a generation of students poorly educated, teacher professionalism subsumed by data management, and schools robbed of funds for anything but technology repair. But the testing and technology companies will win, profiting regardless of the success or failure of their products and always ready to convince us that the next versions will be better.

The DOE's stubborn refusal to even consider dealing with poverty as a means of improving school performance, their insistence of "rigor" and "accountability" in testing students but willingness to give testing and technology companies a free ride, and their enthusiasm about opening new and profitable opportunities to "education entrepreneurs" is highly suspicious behavior. It raises the possibility that DOE policy is designed to profit a small part of the private sector, not the 60 million students in our public schools.

Sources:

Berliner, David C. (2009). Poverty and Potential: Out-of-School Factors and School Success. Boulder and Tempe: Education and the Public Interest Center & Education Policy Research Unit. Retrieved [date] from http://epicpolicy.org/publication/poverty-and-potential

Bowen, W., Chingos, M., and McPherson, M. 2009. Crossing the Finish Line: Completing College at America's Universities. Princeton: Princeton University Press.

Duncan, A. 2010. Beyond the Bubble Tests: The Next Generation of Assessments. http://www.ed.gov/news/speeches/beyond-bubble-tests-next-generation-assessments-secretary-arne-duncans-remarks-state-l

Geiser, S. and Santelices, M.V., 2007. Validity of high-school grades in predicting student success beyond the freshman year: High-school record vs. standardized tests as indicators of four-year college outcomes. Research and Occasional Papers Series: CSHE 6.07, University of California, Berkeley. http://cshe.berkeley.edu

Gibson, E. 2011. Retail sector adding jobs, but not always careers. Associated press. Ventura Star, April 6, 2011.

Nichols, S., Glass, G., and Berliner, D. 2006. High-stakes testing and student achievement: Does accountability increase student learning? Education Policy Archives 14(1). http://epaa.asu.edu/epaa/v14n1/.

Weiss, J. 2011. The innovation mismatch: "Smart capital" and education innovation. Harvard Business Review. http://blogs.hbr.org/innovations-in-education/2011/03/the-innovation-mismatch-smart.html