Mark Scaling Policy and Procedures

Policy

The purpose of scaling is to rectify anomalies in mark distributions that arise from unanticipated circumstances and should be used in exceptional circumstances only. Hence, the assessment criteria and practices for any module that has its marks scaled should be reviewed in order to reduce the chance that scaling will be necessary in subsequent years.

Where scaling is employed for adjusting agreed assessment marks within a module to correct abnormal group performance, the following rules must be adhered to:

1. The raw marks, together with the rationale under which they were awarded; must always be made available to the Assessment, Progression and Awarding Committee;
2. Scaling must not unfairly benefit or disadvantage a subset of students (e.g. failures). This means that any scaling function applied to a set of marks must be monotonically increasing, i.e. it must not reverse the rank-order of any pair of students. The definition of any scaling function used (its domain) must encompass the full range of raw marks from 0 to 100%. For example, 'Add 3 marks to all students' or 'Multiply all marks by a factor of 0.96' are both valid scaling functions. 'Add 4 marks to all failures and leave the rest unchanged.' is not acceptable because it would cause a student whose raw mark was 39 (a fail) to leapfrog a student who got 41 (a pass).
3. External Examiners should always be consulted about the process;
4. The rationale for scaling and the impact on marks must be clearly recorded in the minutes by the Assessment, Progression and Awarding Committee;
5. The system used to identify modules as potential candidates for scaling must be transparent.

See also: University TQA Handbook - Scaling of Marks.

Note: The University's Assessment Handbook Working Group is recommending University-wide adoption of the above policy to the June 2014 meeting of the Faculty of Taught Programmes. The APAC replaces the discipline-level functions of the Board of Examiners

Procedures

Before the External Examiners Arrive

The APAC will consider the mark distribution statistics of all modules and identify those with anomalous outcomes that, in the Committee's judgement, merit further investigation. Criteria may be based on the statistics provided by SITS. Where feasible, however, a more sophisticated analysis should be used, for example by using the

• Student Performance Report Generator

The Committee must avoid using criteria that inevitably cause grade inflation/deflation, e.g. by considering only modules with a high number of 'fail' grades but ignoring modules with a high number of 'excellent' grades.

The APAC will appoint appropriately experienced member(s) of staff to review the assessment of the module(s) identified and to recommend what, if any, scaling of the mark distributions is necessary to align the module with the applicable University marking criteria.

A pro-forma report is required for each module reviewed:

• Mark Distribution Review Form [PDF]  [DOCX]

The reviewer will consider the evidence available and recommend whether, in their opinion, adjustments are necessary and of what type. Reviewers must keep in mind University policy that scaling should be used in exceptional circumstances only and it should therefore be recommended only when there is no reasonable alternative.

When the External Examiners Arrive

In order to allow time for the recommendations to be implemented, external examiners should asked to consider the Mark Distribution Forms at the earliest opportunity. Their signature on each form will confirm that the consultation required by University policy has occurred.

At the Assessment, Progression and Awarding Committee

The recommendations made on the Mark Distribution Review Forms will be proposed to the APAC and final decisions made and minuted in accordance with the above policy. The criteria used to identify modules with anomalous outcomes should also be minuted.

Follow-up Action

Copies of the Mark Distribution Review Form must be:

• archived with the affected student scripts so that it is readily available for QA purposes;
• sent to the Module Leader so that they can review the assessment practices that gave rise to the need for scaling;
• considered as part of the module review process.