In my first job out of university, I had the task of developing a tool to calculate “minimum target grades” for FE students based on GCSE scores and regression statistics. These grades would be used by teachers as a basis for setting target grades for their students.
I’m no statistican. When I did my maths A-level I chose Decisions for my applied units. However, looking at the regression statistics, began to notice that some some of these statistics looks suspect. Some of the sample sizes were tiny. Some of the standard deviations were larger than a whole grade boundary.
After realising this, I felt like it would be irresponsible for me to use these datasets to calculate target grades. I built safeguards in to the system so that it wouldn’t calculate grades if the data was of insuffient quality, and target grades would have to be determined by other means.
I hope that anyone designing algorithms to determine students’ final grades would take a similar level of care.