Documents obtained by The Ferret about the Scottish Government’s standardised testing policy reveal a range of potential risks including data losses, invalid results and a potential boycott by teachers.
The concerns – 35 in total – are highlighted in a document submitted by ACER International UK Ltd which won the tender to manage the testing system.
The Scottish Government – whose policy is to have pupils in P1, P4, P7 and S3 complete new standardised tests to help teachers form judgements on their progress – finally released the documents more than six months after the information was requested under freedom of information legislation.
In response, Scottish Labour branded the government’s handling of the issue as a “shambles”.
Information on 35 separate risks has been revealed although the likelihood of each risk and any actions taken to address them remain secret.
Some of the risks – such as database failures and security breaches – relate specifically to the operations of ACER International but others reflect long-standing concerns about one of the Scottish Government’s flagship education policies.
In the area of “Admin & systems”, the document states that schools “may lack sufficient technical capabilities and/or capacity” and may therefore be unable to access online testing.
One IT specialist with extensive knowledge of Scottish schools – who asked not to be named – pointed to ongoing problems with out of date computers and insufficient bandwidth, both of which could have a significant impact upon any online assessment system.
These concerns were echoed by George Gilchrist, a recently retired headteacher who now writes about education issues. He said: “A lot of schools don’t have the hardware to allow a whole class or year group to access computers at the same time, so this presents challenges regarding organisation, whilst still delivering learning experiences.”
In the “Skills Knowledge Capability” section, the risk of teachers becoming “over-familiar with the test” and able to “predict items” is identified alongside the recognition that this could compromise the “reliability of assessment and progress information.”
Critics of the government’s policy – and of national standardised testing systems more generally both here and abroad – have consistently warned about the prospect of increased “teaching to the test”.
Under “Comms & Engagement” the document states that “assessment outcomes will need to meet the varying needs of stakeholders”, with a failure to achieve this potentially leading to a “lack of confidence in outcomes” and the tests being seen as “pointless and/or a burden on schools and pupils.”
ACER International also warns of “possible boycotting by teachers as happened in England”.
At its 2017 AGM, the EIS teaching union agreed that “if the Scottish Government implements national standardised testing which the EIS determines as detrimental to learning and teaching in schools, all members in primary and secondary schools will be balloted on a boycott of the administration and reporting of the test results.”
EIS General Secretary Larry Flanagan reiterated this position. He said: “The EIS remains unconvinced about the need for national standardised testing and we are deeply sceptical about the role of Scottish Government, or its agencies, in handling data which might arise from such tests.
“In particular we believe that national standardised testing of pupils in Primary 1 is a fundamentally flawed approach to supporting children’s learning at that critical age, which reflects a lack of pedagogical insight from Scottish Government.”
“The EIS accepts that progress has been made in shaping the Scottish standardised assessments away from an English style high-stakes SATS approach but our position remains that if these tests, in practice, lead to a distortion of effective teaching and learning in our schools then they will be opposed.”
Scottish Greens education spokesperson Ross Greer MSP warned that the SNP risks repeating the mistakes of English education policy.
He said: “Standardised testing from the age of six causes unnecessary stress for pupils and staff, is not backed up by evidence and now, after initial resistance from a government with a transparency problem, a whole raft of administrative and staffing concerns have been clearly laid out in the public domain.
“Why the SNP is repeating the mistakes of English education policy, despite ample warning, is inexplicable.
“Testing is not the answer and the government should instead commit to investing in more resources to reverse the cuts of the last decade, especially in Additional Support Needs, where over 500 teachers and hundreds of support staff have gone since 2010. That’s why Greens fought for, and won, an additional £160 million for local services this year.”
Iain Gray MSP, education spokesperson for Scottish Labour, said: “The SNP government’s introduction of standardised testing has been a shambles from beginning to end.
“Labour’s concerns and those of teachers and parents – for example about school league tables – have been dismissed, only to turn out to be well founded. Trials have been delayed and costs have soared.
“Now we see that the very company producing the tests is flagging up serious risks, such as teaching to the test, the reliability of results and a lack of confidence in the new system.
“No wonder Ministers tried to keep these documents hidden. They show yet again that the SNP has no clue what it is doing when it comes to education.”
Another key risk identified by ACER International is the “intense and highly political environment” within which Scottish education exists.
This, according to their assessment, could generate an “adverse impact on staff behaviours” such as the “concealment of problems within project team and from Scottish Government.”
A Scottish Government spokesperson said: “Scottish National Standardised Assessments are on track to be delivered on time and within budget – demonstrating the success of the robust governance and project management procedures we have in place.
“This includes, as routine, the identification and management of any potential risks to delivery.”
In December 2016, the government was asked to release the name of each bidder for the standardised testing contract, as well as bid documents from the chosen supplier.
The evaluation of the top three bids and the expected final cost of the new testing system were also requested.
The government initially failed to respond in time to the request, with the official responsible advising the material could not be released as they were “awaiting internal clearance” from an unidentified person.
A response was eventually received on the 17th of February – almost one month late – but some of the requested information including the Risk Register had been withheld. The Risk Register details potential risks about the system.
However, in advance of the SIC’s decision the government has released some information from the Risk Register document.
The Scottish Government’s plans to introduce a system of national standardised testing have proved deeply controversial since being announced by Nicola Sturgeon in 2015.
The Scottish Government also fought an unsuccessful, year-long battle to keep the limited written advice it received on the matter a secret from the public. With the Scottish Information Commissioner eventually ruling that the material should be released.
The winning bidder, ACER International UK is wholly-owned subsidiary of the Australian Council for Educational Research (ACER).
The above article was amended on 31st July 2017. The following par was re-inserted after being mistakenly deleted during editing – “The SNP government’s introduction of standardised testing has been a shambles from beginning to end.” A link to a Common Space piece by the author of the above was removed.
