'A' Level grades - how about a sensible discussion....

[motorbiking]

They did - didn't they?

but one reason would be size - Scotland, while very noisy politically is less than 10% of the population of England.

 

[Ihavenojob]

They did - didn't they?

Exactly, but no one seems to mention it.

 

[JohnD]

The goal was to avoid a grade spike by being over generous. It assumes that the pool of pupils is roughly as bright as the previous pool of pupils. Therefore rank them and allocate according to the ranking based on historic grades. Seems fine. But at an individual level your grades up or down are based on other people's performance and that needs adjustment.

it also assumes that if you are an exceptionally bright scholar, in a school that has not previously had a person of such exceptional talent in your particular subject, you cannot be allowed to get a higher grade than the dullards of the past. And that if you are an middling student, but the lowest of them, and your school previously had some very poor ones, you must be given a grade as low as them.

Good explanation here.
https://next-media-api.ft.com/renditions/15976895127300/1280x720.mp4
I don't know if you can watch it without registering.

The algorithm did what it was supposed to. It produced results in accordance with the wishes of the politicians in charge. And their unelected boss.

 

[motorbiking]

No disagreement. The missing bit was to apply some logic to the outliers. Kid A predicted an 8, gets a 6 because of the allocation. Those needed to be flagged as low confidence bunged in to a bucket to be manually assessed. The revers allocation can also be true. Decent teacher for maths in school A gets everyone to a high standard, moves to a new school and even the kid who can't add gets a 5, based on the performance of his predicessors.

They talk about this algorithm like its some advanced AI running on GPUs, its just a ranking and a list. Probably all on a spreadsheet.

 

[ellal]

The goal was to avoid a grade spike by being over generous. It assumes that the pool of pupils is roughly as bright as the previous pool of pupils. Therefore rank them and allocate according to the ranking based on historic grades. Seems fine. But at an individual level your grades up or down are based on other people's performance and that needs adjustment.

Again...

Ministers have admitted that the 'over generous' results by algorithm benefited fee paying students!

So on a historical basis, state school students were going to get screwed over yet again!

Only the outcry this time has been too great, so even the 'elite' have had to back down a bit!

 

[motorbiking]

which minister? source?

I think the model disadvantaged rapidly improving schools and probably favoured those in decline. Those with stable grades would be in neither category.

 

[IT Minion]

No disagreement. The missing bit was to apply some logic to the outliers. Kid A predicted an 8, gets a 6 because of the allocation. Those needed to be flagged as low confidence bunged in to a bucket to be manually assessed. The revers allocation can also be true. Decent teacher for maths in school A gets everyone to a high standard, moves to a new school and even the kid who can't add gets a 5, based on the performance of his predicessors.

They talk about this algorithm like its some advanced AI running on GPUs, its just a ranking and a list. Probably all on a spreadsheet.

Depending on the data sets they might not have been able to use Machine Learning. I don't know if predicted grades and class rankings are normally a thing. Otherwise they wouldn't have the material to do a meaningful machine learning process. They'd probably run a shed load of runs but ML is only as good as the data set it's fed.

For this many results they'd need lots of sheets or to run 64bit office.

It's also not a normal ML problem, they had all the data they needed for weeks so they'll have run lots of different algorithms against each other and then picked the one that gave the output they liked most.

 

[motorbiking]

For this many results they'd need lots of sheets or to run 64bit office.

how many schools have more than 1,048,576 grades to asses?

I think this would easily have been possible with a modern analytics package. These things support highly collaborative models. They would easily have been able to do some what if analysis and attacks the outliers. I suspect as usual anyone with any IT talent, doesn't work anywhere near a school or exam body and definitely not in some govt dept. I suspect it was downsourced to some management consultant with a 2:2 from a polyversity.

 

[IT Minion]

how many schools have more than 1,048,576 grades to asses?

I think this would easily have been possible with a modern analytics package. These things support highly collaborative models. They would easily have been able to do some what if analysis and attacks the outliers. I suspect as usual anyone with any IT talent, doesn't work anywhere near a school or exam body and definitely not in some govt dept. I suspect it was downsourced to some management consultant with a 2:2 from a polyversity.

Professionals would have just chucked it into one of the ML suites or started pulling it apart with R.

In this case it doesn't need IT skills, it needed strong statisticians, maybe with a bit of knowledge of R, but MATLAB or even Excel would have gotten them there.

ML might have been ok, chuck in the starting data, pick a minimal requested grade inflation and set bounds about grade variance and then sit back and wait so see exactly what it spat out.

 

[EddieM]

Depending on the data sets they might not have been able to use Machine Learning. I don't know if predicted grades and class rankings are normally a thing. Otherwise they wouldn't have the material to do a meaningful machine learning process. They'd probably run a shed load of runs but ML is only as good as the data set it's fed.

For this many results they'd need lots of sheets or to run 64bit office.

It's also not a normal ML problem, they had all the data they needed for weeks so they'll have run lots of different algorithms against each other and then picked the one that gave the output they liked most.

Don't be daft, this is a government IT project and as such probably never got an algorithm to compile. Let alone multiple ones, no one that can walk and talk at the same time works on Government IT projects.

 

[Notch7]

Don't be daft, this is a government IT project and as such probably never got an algorithm to compile. Let alone multiple ones, no one that can walk and talk at the same time works on Government IT projects.

I've heard not good things about govt IT.

I remember an IT contractor talking about when he did a 6 month contract for the NHS.

Apparently they were running some software that costs millions but they could've got the same thing using an open source platform that was better supported.

 

[EddieM]

I've heard not good things about govt IT.

I remember an IT contractor talking about when he did a 6 month contract for the NHS.

Apparently they were running some software that costs millions but they could've got the same thing using an open source platform that was better supported.

No, they're invariably utter sh*te. The only time you'd work for them is on a contract basis, as you'll be paid a similar amount to industry standard, but the contract will probably last significantly longer, and your delivery can be minimal / zero. Though that's not a very professional way of working I concede.

 

[Doug99]

A 25% increase in the number of students receiving top GCSE grades. That's OK then? No doubt the Education Secretary will be promoted for that result!!

 

[charliegolf]

A-levels were never going to be easy to sort out, but less so for GCSEs. Again, 'they' knew months ago. One way would have been to use metrics we use in Wales (not sure about elsewhere) known as L2 Threshold (5 gcses at >C grade); and Level 2 Inclusive (same thing including Maths and English)*. This might have been used as a one-off 'Year 11 Leaver's Cert'. Why? These pupils ONLY have to convince their own sixth form teachers to let them in (easiest); OR the local college. There is no, 'application' or, 'clearing' for this. EVERYONE gets in, they are catchment based. Pupil level interviews and moderation would easily sort wheat from chaff at entry to specific A-level courses. A one off. Granny doesn't get to brag about Jacinta at bridge. So what? Thick Charlie's Mum gets a free pass to say at Bingo he was 'as good as' Jacinta, he got the same cert. No UCAS points involved, no offers, no preferences to navigate.



Instead we ridicule everyone for the kids being boosted by 'for free'. Hey ho.



CG



* we use these for school effectiveness reporting in Wales. Or did.

 

[JohnD]

I wonder who's responsible for that scheme our government decided to go for, that was "robust and reliable."

Is there any way of findng out?

Yes there is.

https://www.theguardian.com/educati...ummings-hired-to-work-with-ofqual-on-a-levels

But they must have chosen the best people for the job, right?

Nope

"Public First was given communications contract without competitive tender"

Not the same government who handed a Ferry contract to a company with no ferries?

Yep.