Bearing in mind the evidence from Ken Boston at the Children Schools and Families select committee, about which I hope to make a posting later in the week, I thought I’d post the article I wrote on the Sutherland inquiry at the end of December (for publication in January) for Education Journal.
The text follows below:
There was something deeply unsatisfying about last month’s report on the biggest administrative shambles I covered in my nine years as a reporter with the TES.
The Sutherland Report, ordered by ministers and their new testing regulator, Ofqual, largely cleared the Government of any responsibility for the catalogue of mistakes which led to the late publication of 1.2 million children’s national test results this summer.
Was that a fair verdict? Well, a close reading of the report, and an understanding of the way the Government, the Qualifications and Curriculum Authority and its test marking contractor ETS have operated, suggests not.
ETS and the QCA, and particularly the former, clearly must bear the bulk of the blame. But the Government was hardly uninvolved in decisions which helped to shape this summer’s events.
The report rightly heaped opprobrium on the American testing firm, which was in the first year of a five-year contract to mark the key stage 2 and 3 tests. Its conduct, it said, had been “wholly unacceptable and lacked professionalism”.
This, perhaps, was the only possible verdict when one considers the events up to and including the summer’s marking delays.
From the botched trial of a new system of online training for markers in late 2007 to many examiners receiving incorrect contracts and wrong information on training events, and schools complaining of faulty on-line registration systems, administrative bungles characterised the build-up to the marking itself.
This was then blighted by papers being sent to the wrong markers’ homes; a helpline staffed by people who appeared to know little about the marking process and which was overwhelmed by the sheer number of calls; and software systems for checking markers’ work which broke down.
“The primary responsibility must therefore rest with the American organisation, ETS Global BV, which won the public contract to deliver the tests and failed its customers,” said the report.
But the QCA had also failed to deliver its remit, it added. This was to “select a first-rate delivery contractor, oversee its work and ensure that the Department for Children, Schools and Families [sic] policy objectives were met”.
Moreover, Sutherland said the QCA, and its testing division, the National Assessment Agency, should have realised sooner that the results would not be released on time, only informing the DCSF of this a week before they were due, on June 30th.
This “it’ll be all right on the night” sense of complacency at the NAA may be well-founded.
Yet, among the report’s 46 recommendations, only two suggested failings by Ed Balls’s schools department: one proposing that the role of DCSF observers at QCA board meetings should be clarified; and the second stating that Government officials “may not have challenged QCA sufficiently on its project and risk management”.
Yet a further consideration of the evidence might prompt a different view. For example, the DCSF had oversight of the process of appointing ETS as a contractor.
At least one of its senior officials was present at a meeting when the procurement was discussed, and the Office of Government Commerce, an arm of the Treasury, approved the process.
Ken Boston, the QCA’s chief executive, told the inquiry that officials were closely involved at all stages leading up to the marking of the tests.
He said: “Throughout the process of procuring the contract and delivering the tests according to DCSF specification, ministers and officials had access to exactly the same data and information as the NAA and the QCA; they were active participants in the process.”
The report does not comment on this observation. However, it is widely known that DCSF officials shadow the work of their counterparts at the QCA. The detail of what ETS was setting out to do had to have been known to the Government, as part of the procurement process. And so, too, must have been several major changes to the system, introduced all at once this year. These included alterations to the arrangements for delivering scripts to markers; the removal of double-checks on the marking of papers where a pupil’s result left them just short of achieving a certain national curriculum level on first marking; and the requirement for examiners to enter each individual mark at the computer screen.
All of these would appear, to this observer, to have added to the risk of failure this summer. But the report offers little sense that this danger should have been on ministers’ radars.
Sutherland also concludes that the ambiguous nature of the NAA – it is a division of the QCA, but the report said that it was viewed by ETS and Ofqual as a separate organisation – got in the way of proper risk management.
But the report fails to point out that one of Mr Balls’s predecessors, Charles Clarke, approved the NAA’s creation.
Making these points might seem, to some, like quibbling. Yet they are, I believe, important. The central point is that, in its relationship with the QCA, it is the DCSF that often wields the power.
It sets very tightly defined remits for the authority. It sets the QCA’s budget. And the QCA chairman and chief executive can expect a dressing down from ministers if they step out of line.
Given all this power, where is the responsibility when things go wrong?