Examiners are asked to provide their questions in the form of print-ready PDF completed papers, typeset in the appropriate format with title/rubric page, good quality diagrams and a full set of marking criteria. The goal is to carry out minimal revision of these papers after the initial submission, so setters/checkers should be confident that each paper is ready to present to candidates or to an external examiner. (To this end, setters and checkers are asked to sign a tracking sheet for each paper, confirming its readiness.)
A PDF file (with correctly embedded fonts) for each paper should be submitted to the Education Support Hub by uploading to the appropriate Hub Sharepoint folder. The call for questions includes: the dates, times and membership of the scrutiny meetings; a list of modules; and a list of setters and checkers – questions must be set and checked by the persons indicated, unless the Assessment Officer agrees to a substitute. (It is understood that questions will eventually be marked by the setter.) A template with preset styles is available (see below). Some templates and guidelines for preparation are given below.
Scrutiny meetings will consider the papers as submitted, without any revision by the Assessment Officer. Thus it is important that editorial matters and issues of the technical accuracy of the content are sorted out before submission - there should be no need to spend significant time on these at the meetings, where the main task will be to check the overall level of each paper, and that the distribution of marks within each question are in accordance with the guidelines below.
After the corrections required by the scrutiny meeting have been made, the papers are sent to the external examiners who may require further amendments before granting their approval.
Deviation from the deadlines in this schedule should only arise as a result of exceptional circumstances and must be approved by the Associate Dean for Education.
Note that week numbers for the preparation of May exams are currently provisional.
| Week Numbers | Actions and Notes | |
|---|---|---|
| Jan. Exams | May Exams | |
| T1:02 | T2:03 | Module leaders submit complete (draft) exam papers and mark schemes, for both the main and referred exams, to internal checkers (by end of this week). |
| T1:03 | T2:04 | Checkers review and return exam papers and mark schemes to module leaders with comments. |
| T1:03 | T2:04 | Module leaders make all necessary changes and pass exam papers to Education Support (by end of this week) who will collate the paperwork for the scrutiny meetings. |
| T1:04 | T2:05 | Scrutiny meetings (membership organised by Assessment Officer and DoE) take place, normally on Wednesday PM. Papers clearly annotated with corrections/comments to be returned immediately to module leaders. |
| T1:05 | T2:06 | Module leaders make edits / corrections indicated by scrutiny meetings. Final electronic papers to be submitted to Education Support. |
| T1:05 | T2:06 | Education Support send papers and mark schemes to external examiners (by end of this week). |
| T1:07 | T2:08 | External examiners return comments to Education Support (by end of this week). |
| T1:08 | T2:09 | Module leaders review external examiners' comments, replying and updating papers as necessary. (See Review By External Examiners below.) |
| T1:08 | T2:09 | Module leaders submit finalised print-ready PDF exam papers and mark schemes to Education Support by Thursday (for delivery to the Exams Office by Monday of next week). |
Under no circumstances must the contents of examinations be revealed to any person who is not a member of the Physics and Astronomy APAC (Board of Examiners) or an approved member of the University Staff.
Shared and/or networked computers present special security problems, in particular:
All modules need to be assessed in a manner such that the final module mark is consistent with the applicable
Examinations must be a fair, balanced and appropriate assessment of the module specified by the published module descriptor. When setting questions examiners should structure them so as to achieve the following mark distribution:
The implication of this scheme is that, within a question, no attempt should be made to assign equal marks for equal difficulty. We should, however, be guided by the principle of equal marks for equal time spent by a good candidate in answering the question. Where appropriate, examiners should attempt to include a reasonable number of calculations leading to a numerical answer within a paper.
Questions sometimes require a result obtained an early section to be used in a later part. In such cases, particularly in compulsory questions, care should be taken not to over-penalise a candidate who doesn't get the first result correct, e.g. by automatically giving zero for the subsequent parts of the answer. Fairness can be achieved, for example, by asking the candidate to 'Show that...' or giving 'method marks' for the later part.
Take special care when setting an exam for a module that comprises parts delivered by different lecturers to ensure that students can't selectively revise parts of the module and still achieve a good mark.
Examinations have to be appraised by internal and external examiners who were not involved in delivering the module. To ensure that they have the information necessary, mark schemes must indicate the level of difficulty of material by identifying it as requiring 'Recall', 'Explanation', 'Application' or 'Synthesis'. These categories are roughly equivalent to the traditional 'bookwork/seen/unseen' nomenclature, but are more generally applicable. This table gives an indicative guide to the use of these categories:
| Type of Question | Category of question/part | |||
|---|---|---|---|---|
| Recall | Explanation | Application | Synthesis | |
| Mathematical | Quote mathematical formulae or proofs. | Derive or give a physical interpretation of a result. | Solve problem of previously seen type or requiring a straightforward procedure. | Construct solution to an unseen problem, e.g. by selecting and combining single-step methods. Exercise foresight and judgement in choice of method. |
| Scientific | State laws and facts, quote formulae, describe experiments. Recall factual content of lectures and/or directed reading. | Interpret observations and explain how they are related to physical laws. | Use factual knowledge and reasoning to answer questions. Solve familiar types of problem. | Construct solution to an unseen or complicated, problem. |
| Technical | Describe standard procedures. | Explain how methods work and their precision. | Analyse results using standard methods. | Design experiments or measurement schemes. |
| Qualitative | Describe phenomena, methods, history, etc.. | Give physical interpretations and explanations of phenomena and observations. | Use familiar methods and/or ideas to analyse and/or evaluate observations. | Demonstrate independently critical engagement with primary and/or secondary literature. |
Mark schemes should indicate the parts of questions that are assessing 'self-study' elements of modules. Guided self-study is an essential part of all Exeter Physics modules and examiners need to ensure that it comprises an appropriate proportion of the assessment.
Note: Whilst it can be appropriate to recycle parts of old questions, questions from recent years should not be reused as they stand, in their entirety. Where old questions are reused in a substantially unchanged form, their source must be indicated on the mark scheme so that checkers and external examiners can verify the integrity of the examination.
The structure of examination papers is expected to follow a standard pattern as specified below. Variation of these structures is allowed for exceptional reasons, on the authorisation of the Director of Education. In cases where the rubric instructs the candidate to attempt more questions than are required to achieve full marks, this should be clearly stated: e.g, "Attempt all questions. X complete answers are required to achieve full marks (Y).".
Examinations to be taken in person under closed-book invigilated conditions have the following rubrics:
| Mathematics Levels 4 & 5 (PHY1025-26, PHY2025 15-credits) | Time allowed 2 hours. Five questions, each worth 20 marks, with the rubric: 'Answer ALL five questions. Full marks (100) are attained with five complete answers. (Marks may be subject to scaling by the APAC.)' |
| Mathematics Level 6 (PHY3062 15-credits) | Time allowed 2 hours 30 minutes. Five questions, each worth 20 marks, with the rubric: 'Answer ALL five questions. Full marks (100) are attained with five complete answers. (Marks may be subject to scaling by the APAC.)' |
| General Problems Level 6 (PHY3053 15-credits) | Time allowed 2 hours 30 minutes. Section A: twelve questions, each worth 6 marks; Section B: four questions each worth 14 marks; with the rubric: 'Answer TWELVE questions from Section A and TWO questions from Section B. Full marks (100) are attained with twelve complete answers from Section A and two complete answers from Section B. (Marks may be subject to scaling by the APAC.)' |
| Physics Levels 4 & 5 (15-credits) | Time allowed 2 hours. Four questions, each worth 25 marks, with the rubric: 'Answer ALL four questions. Full marks (100) are attained with four complete answers. (Marks may be subject to scaling by the APAC.)' |
| Physics Levels 6 & 7 (15-credits) | Time allowed 2 hours 30 minutes. Four questions, each worth 34 marks, with the rubric: 'Answer question 1 and TWO other questions. Maximum marks (100) are attained with complete answers to question 1 and two others. (Marks may be subject to scaling by the APAC.)' |
Referred/deferred papers, which are normally taken in August by a small number of students, should also include the instruction:
This examination paper must be left with the script at the end of the examination and may NOT be removed from this room.
as part of the rubric.
| Physics and Mathematics modules at levels 4 & 5 (15-credits) | Time allowed: 30 minutes. Total of 25 marks, with the rubric: 'Answer ALL questions. All essential working should be shown. Full marks (25) require complete answers to all questions.' |
| When multiple choice questions are used add this instruction to the rubric: | A correct answer to a multiple choice question gains the marks indicated. An incorrect answer is penalised by the deduction of one mark unless it is supported by reasonable working. The "Don't know" answer 'X' gets zero points. |
Notes:
A detailed set of Marking Criteria must be submitted with each paper. These will normally include:
A satisfactory set of marking criteria would allow:
The marking criteria need to match the mark allocations on the exam paper, and this should be confirmed at the setting-checking stage.
Marking criteria and/or model solutions for past examinations are not made available students. This is policy was adopted by the Board of Examiners in order to encourage students to develop their own problem-solving skills and to discourage weak students from rote-learning past answers as a substitute for more appropriate learning strategies.
Immediately after the principal examinations (i.e. Jan/May but not Aug) have been marked each year, the module leader must publish a summary sheet of solutions and hints on ELE and email a copy to the Examinations Administrator for archiving. These will normally include:
Vague references to 'bookwork' should be avoided; give specific pointers that students can use to locate the source, e.g. a lecture number, link to ELE, text-book section, etc.
The purpose of these sheets is to enable a student who has attempted the problem(s), e.g. as part of their revision, to assess whether their method has yielded a correct solution. The typical amount of detail required is indicated by this example [pdf].
No more than the most recent 4-years of solutions and hints should be published. Once this limit has been hit, remove the oldest set at the same time that the newest set is uploaded.
The style of deferred/referred examinations is the same as that of the original examination. Particular attention should be paid to the correspondence between the pass mark and the stated learning outcomes if the module includes a component from coursework and/or mid-semester tests because referred assessment is based solely on the referred examination, i.e. previously acquired marks are discounted (see the applicable Examination Conventions.
Examinations must be prepared using the current approved templates and formatted consistently with conventions given below. Depending on how your browser is set up, you may need to right-click (or click-and-hold) the link and ask for the file to be saved to disk 'as source'.
If you use Microsoft Word, you will need to turn off the 'Automatic' options which interfere with styles, paragraph numbering and formatting. In the event of problems, consult the Assessment Officer.
The template has three styles, to be used as follows:
(It may be helpful to print the template before altering it to see how it's meant to look.)
Indicate on page one if any materials are required in addition to the
sheet (for example, graph paper on request). In the case of an open-note exam, a prominent statement to this effect is required on page one. Page one is only for title, rubric, etc.. Please do not put formulae or data on page one. Start the questions on page two. Type "Turn Over", at the foot of each page except the cover page and the final page, which should have "End of Paper".
Use (i), (ii), (iii), etc. to label principal sections of the question, but only if these relate to different topics; (principal sections do not need labelling if they all relate to the same overall topic - just use separate paragraphs).
Use (a), (b), (c), etc. to label such things as sub-sections of questions, a sequence of quantities to be calculated or a list of terms to be defined. Such lists should normally use EXAM2 format.
Avoid asking several separate questions in the same paragraph with the marks given in one square bracket at the end, e.g. [2, 3, 2]. In such cases use a list labelled (a), (b), (c), normally in EXAM2 format, or give each question its own EXAM1 paragraph.
If character formatting is used to produce subscripts, superscripts, etc. ensure the size and offsets applied match the the equation editor settings. The use of symbols and notation should conform to:
This can be achieved by following the NIST guidelines:
Briefly, almost all symbols (whether Latin or Greek) should be typed in italics. The three main exceptions are mathematical functions (cos, ln, etc.), subscripts that relate to names or words (kB, Eext, etc.), and physical units (mA, kg, etc.). Vectors should normally be typed in bold-italic (without arrow or underline). Subscripts and superscripts should be 9-point size. Type 0 (i.e. zero), not o (i.e. the letter oh), for superscript or subscript 'nought'. Put a space between quantity and unit, and between elements of a unit (e.g. 59 m s−1, not 59ms−1). Do not use a hyphen ('-') as a minus sign, the correct symbol '−' (an en-rule) is longer. Punctuate formulae and symbols as parts of the sentence that contains them.
The NIST Guide has a detailed checklist in its preface.
Don't overuse emphasis but where it is needed for clarity use italics. Use boldface for highlighting specific terms that are being referred to later in the question, e.g. when asking for definition or explanation.
Every person responsible for an examination will be issued with access to a Sharepoint folder by the Examinations Administrator.
It is the responsibility of the examination paper author to upload in Adobe PDF format:
These files must be uploaded into the Sharepoint folder.
It is the responsibility of the checker - after discussions with, and revisions by, the setter - to confirm that in their opinion the examination:
In the unusual event of an unreconcilable disagreement between setter and marker the advice of a suitably-qualified mutually-agreed third person must be sought.
All external examiners' comments, including invitations to consider points, must be responded to by the setter on the form provided by the Examinations Administrator.
Where amendments are suggested by an external examiner, these should be made unless there is a compelling reason to do otherwise. When suggestions are are not followed a rationale must be given and approved by the Assessment Officer before it is returned to the external examiner.
For obvious reasons, changes arising from external examiners' comments must be made very carefully to avoid introducing new errors as side-effects. Significant modifications must be seen and approved by the original checker.
During revision week the Education Support Office will print one 'proof' copy of each exam. The module leader must review this copy and sign and date the front cover to confirm their final approval at least three working-days before the exam date.
The purpose of this exercise it to ensure that the pdf used for reproduction has all the necessary fonts and figures embedded correctly to to ensure that there is a definitive final copy available at short notice should the need arise. There should, obviously, be no errors in the paper at this stage. If errors are found, however, the necessary action (errata, reprinting, announcement, etc.) should be discussed and agreed with the exams administrator without delay.
Exams that have not been signed by the module leader will be inspected by the academic Assessment Officer in order to ensure that there are no 'obvious' printing errors: illegible diagrams or text, incorrect font substitution, missing questions/pages, etc.
Staff are expected to prioritise marking of exam scripts. In order to manage the clerical workload associated checking and uploading of marks into the student records system, a deadline for the return of marked scripts is marked on the pack using the following formula:
|
|
This spreadsheet formula may be used to calculate the return date of N_SCRIPTS scripts available for marking on COLLECT_DATE:
=WORKDAY(COLLECT_DATE, FLOOR(N_SCRIPTS+64,20)/20)
See also the Department: