UAI:Moderating

BikiCrumbs: Moderating

From Biki

--PLEASE NOTE: Could someone who knows a lot about moderation please read this an check it for accuracy. To the best of my knowledge I believe it is accurate, however I may be wrong. Once this page has been checked for accuracy could you remove this notice, --me121 --

1 Introduction
2 A Simple Walkthrough of the Process
3 Some interesting results of the moderation process
4 Conversion Graphs
5 Pseudocode for Moderating School Assessment Marks
6 More Information
7 References

[edit]

Introduction

“All assessments submitted by schools are scaled by the Office of the Board of Studies in a process called moderation. The reason for doing this is so that direct comparisons can be made between assessment marks awarded by different schools. Schools have students of differing quality, as well as different standards in their marking and in the difficulty of tasks and tests they set, so it is necessary to adjust all the school assessment marks in each course to a common scale.

After the examination marks have been scaled, the assessment marks for each course in each school are moderated according to the examination performance of the group of students in the school. The moderation process adjusts the assessments submitted by the school so that the mean (or average) of the group’s assessments is set to equal the mean of the examination marks obtained by the students in that group, and so that the top assessment mark is equal to the highest examination mark obtained by a student in the group. The moderation process maintains the rank order of students within their school group. It is important that the marks submitted by schools show not only the rank order of their students, but also the relative differences between them as these are reflected in the moderated assessment marks.” [1]

To safeguard the system students whose performance in the examination is markedly below what was expected on the basis of their assessments are not included in the calculation of the mean.

[edit]

A Simple Walkthrough of the Process

The moderation process can be explained graphically as follows. (This explanation does not incorporate the exceptions as published in [2])

The raw examination mark is a mark out of 50 on a per unit basis. It is the raw mark attained (total weighted mark) in the examination after scaling for option questions and after being converted to a mark out of 50 on a per unit basis (initial scaled mark).

The image above shows the performance range of one school for one course, in the raw examination mark on the right in purple, and that same school’s and course’s raw school assessment marks submitted to the Board of Studies on the left in blue.

After removing any exceptions as documented in [3], a mean is calculated for the raw school assessment marks and a mean for the raw examination marks. Now the raw school assessment marks are moved (translated) by a difference of raw examination mark mean – raw school assessment mark mean. This ensures that the means of the two sets of marks are the same.

The next thing that the Board of Studies does is scale (stretch) the translated raw school assessment marks about the raw examination mark mean, by a factor that ensures that the highest raw examination mark attained is the same as the highest mark in the initial moderated assessment mark, as shown below.

An animation of this process is shown below:

Image:Moderation.swf

NB: The diagrams above show marks ranging from 0 to 100. In actuality the mark is from 0 to 50.

NB: UAC uses your raw HSC mark in the calculation of the UAI. However as these marks are not usually made available to students, table A3 of the Report on the Scaling of the 2005 NSW Higher School Certificate, which shows the relationship between HSC mark and scaled mark uses the Aligned HSC mark, even though in actuality it is the Raw HSC mark that is used.

[edit]

Some interesting results of the moderation process

The person who is ranked first in the school assessment gets the highest HSC exam mark out of everyone at their school as their school assessment mark.
If you are the only student at your school doing a particular course, then your school raw assessment mark that is submitted to the Board of Studies becomes irrelevant. Regardless of your raw school assessment mark, the moderated school assessment mark will be your HSC exam mark. (Although if this is the case for you, don’t do so bad in your school assessments as your school may give you an ‘N award’. Although they need to warn you before they do this. Also doing your school assessments, including exams, will help make you better at the subject and more prepared for the HSC exam.)
As the standard deviation of the two sets of data is not aligned, and as instead the top mark is aligned, the person who is ranked last at their school, even if they do good in the HSC exam may still have a low moderated school assessment mark. (Please note that this may not always be the case. It depends how close the mean is from the max mark. The smaller the gap, the more the person who ranked last, provided they do well in the HSC exam, the more their school assessment mark will be pulled up by the performance of the group. If the gap between the max mark and the mean of the raw assessment mark is large, then low school assessment marks will not be pulled up as much by high ranking people.)
The moderation process does not change the rank order of students.

[edit]

Conversion Graphs

…this section is under construction… Quadratic curves are used. – Sorry I don’t understand this enough to elaborate. Me121--

[edit]

Pseudocode for Moderating School Assessment Marks

--Please note that there are exceptions to the general process. For instance students who have successful illness/misadventure appeals; students whose performance in the examination is markedly below what was expected on the basis of their assessments; and special cases as determined by the Board, will create slight variations in the general process.

[edit]