Evidence to the Schools Exam Results Inquiry

Confidentiality

We note that the Committee intends to place all submissions in the Parliament's Information Centre for Inspection. We would welcome our submission being made available for public scrutiny. However, we are aware that there are people who work within various organisations who have a lot of knowledge about what went wrong but who may fear that their jobs are at risk if they provide evidence publicly. We therefore urge the Committee to offer potential witnesses the option of confidentiality/anonymity.

Introduction

In providing evidence to this Inquiry, we are clearly not competent to discuss the internal workings of SQA whilst individual experiences are best provided directly by the parents and pupils who suffered them. Therefore, our main focus is on the wider context and the reasons why the warnings, which many now say they clearly gave, appeared to have had no impact. In doing this we draw on our continuous involvement in the Higher Still development programme since its inception in 1994. However, as an illustration of the problems faced by schools, we start by detailing the specific but typical complaints from one of our member schools.

Specific but typical complaints from a school

1) There were indications in advance of this year that SQA was not working well. In 1998-99 the school in question had massive problems with one subject. Pupils with excellent results from Standard Grade in the subject and A and B results in their other Highers were getting Cs or failing in this subject. The school had no confidence in the examining team.

2) The school had a catalogue of problems with HSDU and SQA

• Various course materials were not delivered on time by HSDU
• There were late changes to the marking schemes for some unit assessments with consequential changes to students' unit outcomes
• Unit tests were not available on time
• Amendments were made to final exam exemplars (see also the results of SPTC's survey)
• The internal assessments did not reflect the requirements for the external assessments (see also the results of SPTC's survey)
• There were unnecessary demands made on departments for moderation. Materials were requested by SQA, prepared and sent away by the school and then returned unmoderated
• Forms for putting in returns arrived late and had inaccuracies on them
• There was a shortage of examiners for practical elements such as music
• Teachers who had not agreed to be markers were repeatedly sent scripts to mark
• SQA required the school to be manned from 3rd to 21st July and the school found itself sending the same information over and over again
• Results did not reflect school estimates in a range of subjects
• Results in a range of subjects were incomplete because of missing unit data
• For the current year, some material, due in June, has yet to arrive

Our correspondent concludes with the following comment

"Personally, I have no faith in SQA and believe that the re-shuffling of existing personnel, especially those with a Scotvec background, will not solve the problems. SCOTVEC personnel presided over a system of certification which was riddled with inconsistencies and which had very little credibility among secondary schools. This lack of rigour and consistency has now infected SQA."

Complexity of Higher Still

In considering the problems with the Exam Results, it is necessary to reflect on how much the sheer complexity of Higher Still was a contributory factor.

Higher Still was developed to serve the following main objectives. It was designed to: -

• provide meaningful courses to youngsters who were staying on at school beyond S4 but who could not take the Higher in one year
• provide a route for all youngsters to reach the Higher by making the different levels compatible
• end the divide between academic and non-academic courses by bringing both onto a common framework
• offer a single programme which was suitable for FE and schools.

Reflecting on these objectives, we would suggest that the first three could be and were accommodated reasonably easily. The development of courses at different levels but with a common framework both provided the necessary courses for youngsters who could not do Higher in one year and offered the chance for progression. Treating academic and non-academic subjects the same required that the final external assessment should be varied according to the nature of the subject. However, this approach was already in use vide "performance" as an element in music, or "painting/drawing" as an element in art. There was no difficulty in extending this principle to other subjects. The real difficulties arose in accommodating the needs of FE and schools. Schools have a captive audience of students who do courses in a single year. FE colleges have variable groups of students who start and stop courses at different times, who may be full-time or part-time. In many respects it was trying to accommodate the diverse needs of these different groups which caused the complexity because it was this which led to the development of units as well as courses, of unit assessments as well as external final assessments and which introduced the second diet of exams.

There were two further consequences of this change.

1. The first was the massive increase in formal assessment and the recording of such assessment - on a crude analysis it went up by a factor of four for each pupil. We would suggest that the sheer volume of assessment which had to be recorded contributed to the problems with the exam results and that whilst this quantity of assessment remains, then the risk of problems in the future will remain.
2. Secondly, it meant that both the school and FE sector were operating in ways that they were not totally familiar with. Although schools had, in the past, some experience of SCOTVEC modules and internal assessments, the unit assessments on this scale took them into unfamiliar territory whilst, for the FE colleges, the external assessments for all students similarly posed new challenges. This is an issue which could be overcome in time, as the two sectors become more familiar with the changes in approach, but we would suggest that it could remain a source of problems in the current year.

Surveys of pupils, parents and teachers, undertaken before the examination results were issued and clouded peoples' views, showed liking for the units but reservations about the assessment burden. The units were seen as helpful in keeping youngsters working steadily all year and avoiding the last minute cram. However, criticisms of the assessments were that there were too many, those for different subjects all came together, they took away from teaching time and they were not good predictors or indicators of the final exam. We attach a copy of our survey results to support this evidence.

"Refusal to listen" attitude.

A significant problem in the development of Higher Still was the way the voices and views of those outside an inner core group were largely ignored, even when they were persistent in what they said, when everyone said the same thing or they were right.

Higher Still was a product of the HMI. It was written by an Inspector and it was steered through by the Inspectorate. To the extent that it was their initiative, HMI had a vested interest in seeing it implemented. We would suggest that this meant they tended to disregard difficulties and we offer a number of concrete illustrations of this.

Higher English

Whilst the developments in many subjects were accepted by those working in the subject, English teachers objected from the very beginning to the changes to Higher English. It is worth mentioning that the English panel was deliberately put under the chairmanship of a non-English specialist - George Smuga whose background was in Modern Studies. Here, one of the problems was the attempt to marry the SCOTVEC courses in communication with the old Higher English. School English teachers felt so strongly that what was being proposed was wrong that they set up an organisation, Scottish Association of Teachers of Language and Literature (SATOLL), to campaign against it. They kept to their line despite considerable vilification. Eventually, it was conceded that there had been so many changes to Higher English that it was agreed both to allow a year's delay in the implementation of the new course and to make modifications. Then, at the end of last session, it was agreed that there could be a further year's delay to the implementation of Higher Still English although many teachers complained that by the time the announcement was made (23rd June 2000) it was too late to be helpful, as many schools had already embarked on their courses for the 2001 diet of exams.

Burden of Assessment

As with English, teachers raised objections from the beginning about the burden of assessment in schools. This was a permanent theme which was never really addressed. However, as the course was implemented, evidence emerged to support their case.

• Students were not at the right level to sit the unit assessments at the end of each unit and so there was more re-sitting than anticipated.
• In some subjects the solution was to deliver all the unit assessments together, at the end of the course, immediately before the diet of external exams.
• Students were asked to do unit assessments in several subjects in the same week.

Computer provision in schools

During the development of Higher Still it became increasingly evident that ITC was going to be very important for how schools managed the courses, for delivering the core skill in ITC and as an element of various subjects. However, schools were increasingly nervous as many had very poor computer provision. Aware of this, in 1996 SPTC undertook a survey of computer provision in secondary schools. Of the 405 secondary and primary/secondary schools that we sent the survey to, 213 replied - a response rate of 53% which is good for an organisation such as ours. Perhaps the most significant finding was that of the computers for use by pupils, 25% were "old" BBCs and 60% were more than 4 years old. Our then convener, Judith Gillespie, presented the findings to the Higher Still Development Strategy Group. She recalls the experience as similar to being torn limb from limb by rottweilers. The information was dismissed and rubbished with such memorable comments as "a computer is a computer and it does not matter what machine you use". Interestingly, following the General Election in 1997, the Government itself undertook a survey of computer provision in schools and came up with remarkably similar results. At that point huge amounts of money were poured in through the National Grid for Learning to rectify the situation. So, when computer provision was not Government policy, any problems were denied. Once it became Government policy, action was taken and checking computer provision became a feature of school inspections. It would seem that those at the centre will only listen to that information which coincides with their agenda rather than assessing it objectively.

Implementation

In 1998 it became apparent that there were going to be considerable difficulties in introducing Higher Still. SPTC recognised that Higher Still was urgently needed for those youngsters who could not take the Higher in fifth year and we were concerned that the whole programme would be lost with the effect that the needs of these youngsters would not be met for some considerable time. We therefore proposed that one solution was to introduce Higher Still in the order in which it was most needed, i.e. starting with the lower level courses and working up to the Higher as there were no actual problems with the Higher itself. We put our proposal to two different groups. Alison Kirby, SPTC's then convener, wrote to Douglas Osler asking for the point to be considered by the Higher Still Implementation Group of which she was a member. The minutes show that her letter was merely read, but not in any way discussed. Judith Gillespie, then our Development Manager, wrote to officials within the SOEID. This correspondence shows that we were concerned that the whole Higher Still programme was in danger of failing and we made the following point:

We identified a number of other benefits from introducing the programme in this way, including the following.

The response we got was that the decision had already been taken and that it would be too hard to reverse the development process. When we pursued this to ask when and why the decision had been taken, we were told that it had been announced by Brian Wilson at a COSLA conference in June 1998. We were also told that the reason for introducing it this way was that, because the change to the curriculum content of the Highers was minimal, teachers would only have to concern themselves with the changes in procedure, and not with course content. However, the weakness of this argument was revealed when the implementation was finally undertaken. It was determined that eight subjects, including two of the five most commonly taken Highers - Chemistry and English - had had such significant curriculum changes that schools were allowed to delay their introduction for a further year. Perhaps the true reason was the concern of one official, expressed to us and recorded in our correspondence, that if Higher Still were introduced this way round, then the Higher would never be implemented.

Awareness of problems

There are a great number of people who, with the wisdom of hindsight are saying that they knew, back in April, that the exams would lead to this kind of fiasco. We have tried to pin these people down to ask whom they told of their concerns. It would seem that they were widely voiced at meetings attended by SQA, HMI and local authority education officials. The question has to be asked why HMI did not act on this information. Indeed, a serious question has to be asked about the HMI report presented to the Higher Still Liaison Group on 15th February which gave a very positive view of the introduction of Higher Still. If there was such chaos in the schools either why did the HMI not notice it or why did the schools not make it clear during their inspections? It has been suggested to us that people at every level were afraid to say anything negative. Staff were "constrained in their complaining" and the "the machine is geared to keep people in their place". This brings us full circle back to the original point we made about allowing people to give evidence in confidence. However, it is worrying if the system is not open enough to allow teachers to report legitimate concerns for fear that this will somehow have repercussions on their employment.

It is worth pointing out that even on 15th February, when the HMI report was presented, reservations were expressed about the findings. Item 12 of the minutes of the meeting of the Higher Still Liaison Group (15.02.00), reads as follows:

Attitude of HMI

One reason why pronouncements of problems are not heard is because there is a tendency in HMI to regard any problems as arising from flawed practice on the part of teachers rather than as the result of flaws in the system. For example, in the Standards and Quality report on Modern Studies in Secondary Schools (1995-1999) the quality of courses in S1 and S2 is deemed to have important weaknesses or be unsatisfactory in 60% of the schools inspected. When such a large proportion of schools are judged to be failing, it would be reasonable to consider that what was being asked of schools was at fault rather than that schools were at fault for the delivery. The same attitude was revealed over a survey of the implementation of Higher Still English in those schools that went ahead with the course in 1999-2000. Questions were raised about how these schools had responded to assessment and whether class size affected their response. Douglas Osler replied (23rd July) that "a number of departments with both large and small class sizes commented that they found the requirements of assessment and re-assessment, including assessment of the oral presentation, to be time consuming". But he went on to add "where national advice had been implemented on integrating oral work with other work, such as literature, assessment time was not "lost" but contributed well to progress in other aspects of the course"

Again the approach seems to be that it is the schools which are failing to deliver what they are being asked to deliver and no consideration is given to the possibility that the demands on them might be unreasonable.

Role of politicians

The first meeting of the Higher Still Development Programme was in August 1994 when Lord James Douglas Hamilton was the Minister with responsibility for education. He was succeeded by Raymond Robertson, Brian Wilson, Helen Liddell and now Sam Galbraith. The Westminster Ministers all had other major responsibilities such as Health or Housing. Only Helen Liddell had a relatively light load in terms of departmental responsibilities, but she had Labour Party responsibility for spearheading the election for the Scottish Parliament. On top of these diverse responsibilities, Westminster Ministers also had to be in Westminster several days of the week. There was never any ministerial presence at any of the Higher Still Development, Implementation and Liaison Committees attended by SPTC representatives until the Liaison meeting on 13th September 2000. (A further meeting also to be chaired by the Minister has now been set for 29th September 2000). Despite this, references were periodically made that "this is a decision for the Minister to take". On one occasion the simple statement was expanded to include the comment "and he's not minded to take it" suggesting either that mind-reading is a job requirement for HMI or that Ministers were given very strong advice as to what decision they should take.

It is clear that, given the other commitments on Ministers' time, the many changes in personnel and the complexity of the development, it would be very hard for any Minister to be fully aware and informed about Higher Still.

Changes in Personnel

Over and above the many changes in Minister during the development and implementation of Higher Still, there were a lot of other changes in key personnel.

• The Senior Chief HMI changed although Douglas Osler, the new Senior Chief Inspector had, as Depute, been a member of the Curriculum and Assessment Committee which, in many respects, was the key committee in the development programme.
• Most, if not all, officials in the relevant departments of the variously named SOED, SOEID, SEED changed. Whilst this is common practice within the civil service and officials argue that incomers are well briefed, new people inevitably deal with the situation as they find it and have no real knowledge of all the arguments involved in reaching that current situation.
• Local Government underwent reorganisation resulting in changes to authorities and directors of education.
• The heads of SEB, SCOTVEC and SCCC all changed.

This high changeover in personnel is common to all developments, but the sheer complexity of the Higher Still programme meant that it resulted in very few people having a full grasp of everything that is involved. In so far as this Inquiry intends making recommendations, it is a process which might be looked at for the future.

Multiplicity of committees

Another aspect of the programme which is probably a feature of all such developments but which helps explain why messages from the grass roots seemed not to reach the decision-makers, is that the programme moved along through a multiplicity of committees. The initial development had been undertaken by four massive committees supported by a series of subject panels. Once the programme reached implementation stage, the original idea was to replace the development committees by one implementation committee. However, this proved ineffective and so there grew up a proliferation of committees, working parties and focus groups, some set up by SQA and some by HMI. Remits and personnel overlapped with Ron Tuck and the same union representatives often serving on more than one committee. However, the complexity of the arrangements meant it would not have been very clear to anyone not directly involved where information could be placed most usefully. Moreover, this complexity of committees was compounded by the multiplicity of organisations - SQA, HMI and the Higher Still Development Unit. All of these bodies had different responsibilities and sent materials to schools which were then left confused as to which body they should approach with complaints. For their part, it was extremely easy for the different bodies to say a problem was not their responsibility and then take it no further.

Resources - problems of the bid system

An ongoing argument from schools was the lack of resources for implementing Higher Still - the lack of materials, money and time for staff development etc. At the end of 1998, Helen Liddell announced that considerably extra resources would be available. However, despite her reassurances, headteachers continued to complain that it was not reaching schools. An early meeting of the Higher Still Liaison Group established that one problem was the bid system now used to distribute resources to local authorities. It has become common practice for central Government to require local authorities to put in bids detailing how they propose to spend new money earmarked for special projects. This system was used to distribute the extra money for Higher Still and at an early meeting of the Higher Still Liaison Group it was established that bureaucracy had taken over - that rather than money being paid out as quickly as possible, local authority bids were being returned to them for refinement. One of the successful actions of the Liaison Group was to speed up this process.

Scottish Parent Teacher Council

September 2000