Ofqual: grade boundaries were changed to make results “right”

Wednesday, October 31st

A response from Ofqual to a Freedom of Information request, published last week, offers fresh insights into this year’s GCSE English grading controversy.

Followers of my regular blog on the NAHT’s website will need no reminding that I’ve been taking quite a close interest, having posted several lengthy pieces on this and the related issue of Ofqual’s “comparable outcomes” policy for controlling apparent grade inflation.

Ofqual itself is due to publish its final report on this year’s problems on Friday (November 2nd).

This latest set of correspondence has been released under FOI to blogger and tweeter Antony Carpen, following an earlier request by him which also generated a lengthy correspondence trail and which was covered in my last two blogs on the NAHT site.

This one focuses on correspondence between Ofqual for the Department for Education as the controversy was developing. And it is interesting, right from the start.

In the first set of emails, which date from August 17th – six days before national GCSE results would be announced at a press conference –   an unnamed Ofqual official tells the DfE that:

“On GCSE English, there is a potential story because in order to make sure the overall subject grades are right/comparable with last year and across the boards, some of the Controlled Assessment units sat in the summer have higher grade boundaries than the units sat in January.”

The email goes on: “Policy colleagues have been talking to DfE at your place and are due to talk again early next week.”

“Controlled Assessment” units are tasks set by the boards for pupils to take  in-class, with the work marked by teachers but with the boards deciding later how many marks are needed for each grade.

For the AQA board, which has by far the highest number of pupil entries for English, the number of marks needed for a C grade was changed from 43 marks out of 80 for pupils submitting work in January for 45 in June. This has proved extremely controversial, since the tasks set for the pupils did not change over this period.

Other boards, however, also changed CA boundaries between January and June, according to data provided in this Association for School and College Leaders document, matched to data provided at the end of Mr Carpen’s FOI.

Edexcel changed the grade boundaries for two controlled assessment papers between January and June: one from 55 marks out of 96 to 65 and another from 60 to 64 marks out of 96, the ASCL document suggests.

A third board, OCR, changed the marks on five CA papers between January and June, in all cases, again, moving the boundary upwards, according to the ASCL document. The two other boards included in our regulatory system – the Welsh and Northern Irish boards – seem not to have allowed early entries for controlled assessment units. (For more on the data, see note below)

The significance of the quotation from Ofqual above, I think, is that it is the clearest statement I have seen yet that what the boards, under supervision and in at least one case pressure from Ofqual, did in moving grade boundaries in this way was driven very much by the need to get the overall pass rate “right” in the end.

This can and is justified by Ofqual in terms of it being necessary to combat “grade inflation”. But it does raise some problems, such as the clear risk that boards raise grade boundaries not, in reality, driven by the need to ensure fairness to all candidates taking different modules of a course – or the same module at different times – but by the need to produce overall headline statistics which are comparable to the previous year’s. Again, this may seem unproblematic, for me, it deserves further debate and scrutiny.

So, if one set of candidates one year is advantaged and another disadvantaged by the setting of grade boundaries, but their overall effect is to produce a total number of grades which is similar to previous years – ie the results of the two groups cancel each other out – it may be deemed satisfactory under this policy but is it really being fair to each group of pupils?

In an earlier NAHT blog, I referred to comments from a senior exam board official in a paper dated July 30th, unveiled through Mr Carpen’s previous FOI request.

The official said: “If asked by [schools and colleges] and the press to explain the rise in controlled assessment [grade] boundaries, the rationale has to be based on [examiners’] qualitative judgements of work seen, not on a statistical fix.”

That quotation above suggests to me that we  may be more in the territory of “statistical fix” than qualitative judgement.

-There is plenty more in this latest set of FOI correspondence, as might be expected given that it runs to 148 pages. I don’t have time to blog any more now, though, except to note that it does show Ofqual and the DfE co-ordinating their communication very closely. For example, on 22nd August, a Department for Education official emails Ofqual about a story in the Independent saying “it would be really helpful to have your line on this issue so that we can craft our own around that”.

Similarly, in an email the same day from Ofqual to the DfE, Ofqual indicates it is trying to match its message to that put out by the Joint Council for Qualifications at the GCSE press conference, where results would be announced the following day.

The email from Ofqual says: “We have refined the line on GCSE pass rates to make sure it is a bit closer to the message JCQ will be issuing at the press conference.”

-Note: Candidate entry data provided at the end of this FOI appear to indicate, if I have understood it right, that while changes in controlled assessment grade boundaries were in some cases quite dramatic, relatively few pupils will have benefited from what Ofqual says were “generous” pre-June grade boundaries by submitting controlled assessments before June.

The data show that all the grade boundary changes to controlled assessment discussed above occurred in exam specifications where low proportions of candidates submitted in January 2012 or previously. The only instance in which large numbers of pupils submitted CA entries early was for an Edexcel course for the controlled assessment grade boundaries did not change between January and June.

The significance of this, I think, is that relatively generous grade boundaries for CA in January 2012 are unlikely, in themselves, to have pushed up overall pass rates by much.

However, there were more conventional externally-assessed papers – including a much-discussed foundation paper set by AQA – where there were substantial changes in the grade boundaries between January and June and where substantial numbers of candidates were entered before June.

Leave a Reply

Your email address will not be published. Required fields are marked *