A weekend interview about testing with Education Sector founder Thomas Toch

Florida has struggled this year to get its FCAT results, dealing with a company (NCS Pearson) that has proven problematic in dealing with other state tests, too. Perhaps this turn of events should not surprise, given the state of the testing industry nationally. To get some perspective, we turned to Thomas Toch , the current executive director of the Association of Independent Schools of Greater Washington , who founded the think tank Education Sector and has written widely about the testing industry . He spoke with reporter Jeff Solochek about the situation that Florida finds itself in. The big question is, as Florida is having problems with Pearson, a lot of people are starting to suggest something nefarious is going on. But I've read some of the things that you have written. I've seen that there are problems everywhere with the testing model nationally, with few testing companies and lots of demand. So I was just curious, first off, what you see nationally with the system now and whether things are likely to get better or worse than what we have currently. The testing demands of the federal No Child Left Behind Act have overwhelmed the testing industry. It has increased the amount of state level testing dramatically, forced the testing industry which traditionally had provided a small number of tests nationally that states and school districts would voluntarily use in their districts and NCLB has replaced that model with the demand that every state have its own test that are aligned with the state's own standards. This has caused the industry to have to create many many more tests and at the same time states have been reluctant to spend the money needed to build high quality tests and to employ the number of people needed to really respond fully to the demands of NCLB testing. So the industry has had to try to respond to this dramatically increased demand for testing with not that many resources. The industry has been only able to work with a limited amount of funding because the states have not wanted to invest heavily. What has happened is that the states have required the industry to create what are in effect tests of mostly lower level skills because those types of questions … are easier to create and less expensive, therefore, to develop, administer and score. At the same time the industry has had to face the logistical challenges of printing, shipping, scoring and then reporting on many many more tests. Those logistical challenges have been overwhelming as well. That in particular is what has led to the sorts of scoring errors and delays that you see in Florida and in many other states. People here in Florida are getting really frustrated. The commissioner announced he's going to be fining Pearson at least $3 million, and that's just for the first set of tests. Is there any sort of relief? Is Pearson the only game in town? Well, there are maybe eight or nine full service testing companies that provide everything from test development to test printing, administration, scoring and then reporting. … The largest companies, and the oldest companies, including Pearson and two or three others have now been joined by four or five new companies including some nonprofits like the Educational Testing Service, which administers and scores the SAT test. … But there are still a relatively small number of players and none of them in my view are adequately prepared to respond to the logistical demands of all this additional testing. Pearson as it turns out is the biggest player in the field and has the most infrastructure. It is the company that in the past did most of the back-end work … whereas some of the other companies like CTB McGraw Hill, Houghton and a couple of others focused on test development, actually creating test questions. So should we be surprised that Pearson is failing so miserably for the state of Florida? I think that it's discouraging that the single largest player in the field, the company with the most logistical resources, is struggling in states like Florida. Some people say that there's politics at play here. Have you heard anything about the ties that bind Pearson or other testing companies to politicians in Florida? Is that a factor here? It's not something I am aware of. Obviously, there is a lot of money at stake. And vendors, whether they're testing companies, publishing companies — and many of these testing publishers have textbook divisions, although it's my understanding they maintain pretty strong firewalls — but in any sort of vendor environment vendors are going to seek to promote their products and find sympathetic audiences in different ways. That's just doing business. That shouldn't be taken to mean there are backroad deals being cut. … Presumably and I know in a number of states these have to be bid, and they're public.

That would seemingly undercut the notion of sort of favoritism. So would the fact … there has been another feature of the marketplace in the NCLB era and another reason companies have struggled to get the resources necessary to build good tests and to administer them in a timely and efficient way is the intense competition that has emerged over the last seven or eight years under NCLB. Because there are four or five new players in a market that was traditionally dominated by Harcourt and Riverside Publishing and CTB McGraw-Hill, there has been intense bidding for contracts and there have been a number of instances, insiders tell me, where companies have bid less than what it cost them to deliver the test on the assumption that eventually there will be economies of scale once they get enough business from enough states. That's proven not to be the case. Companies are lowballing their bids in an effort to compete with one another and it's just driving down their profit margin and creating more incentive to create these low-level tests and making it more difficult to hire the people, buy the equipment necessary to execute their contracts effectively. And thus you get the types of problems you see in Florida. So what's the answer then? Is it to go away from these types of tests, because nobody can do them? I don't think that is going to happen. I think the political climate in education is demanding accountability, which means there will be continued strong demands for standardized testing on a statewide basis. Where we seem to be heading … is toward the development of state testing consortia. That is, a number of states would band together to develop a fewer number of tests that would be administered across state lines. That could reduce the cost of testing on a per state basis and thus allow these combined state resources to be spent on fewer but better tests that are also administered in more effectively in a more timely way, avoiding these serious scoring errors and delays. The federal government … is in the process of awarding up to $350 million in grants to these sorts of consortia. I understand that only three groups applied. Yeah. there were seven or eight initially and through a series of mergers and acquisitions, if you will, they're down to three. Two of which are bidding for contracts to provide so-called comprehensive assessment systems … and a third one is focusing on end of course high school exams. … The question is, how much consolidation there would really be under a consortium model. Because states are able under the terms of the federal grant competition to preserve up to 15 percent of their tested topics to be state based. And that effort is being done in close combination with the development of the common core standards which 45 or so states have embraced, at least on paper. And the states are allowed to establish up to 15 percent of the standards on their own, to not be part of the common core model. So when you take those things together … you create the possibility of fairly substantial individualization on a state basis, which would undermine this notion of efficiency that I just described to you that the consortia model could provide. I know Florida is leading one of those efforts . I think the other is led by Washington state. Does being a leader state mean you have a better opportunity to get what you want on the test? It's hard to tell. What's going to have to happen is representatives of the various states in each consortium will have to come together and basically hammer out the content of the test, which will in turn be based to a significant degree on the content of the common core standards and as I said the extent to which each state wants to retain for itself up to 15 percent flexibility in both the standards and the test items. That's going to be a very complicated negotiation. It will be difficult and it just remains to be seen what will come of it. Do you have any advice for the people of Florida as they look at the FCAT scores that are going to be coming out next week? FCAT test scores are coming out. I think people can be confident that they are relatively accurate measures of student achievement but they have to understand that these tests are being developed and administered with modest resources and substantial time restraints. The sorts of problems that you see in Florida on delays and what have you don't inspire confidence in the testing system. Additional reading: How Will the Common Core Initiative Impact the Testing Industry? , by Toch and Peg Tyre

See the rest here:
A weekend interview about testing with Education Sector founder Thomas Toch

Leave a comment

Name (required)

E-mail (required)

URI

Your Message