The standard for all MRCPCH theory exams is set using an absolute criterion rather than a norm-referenced approach; there is no number or percentage of candidates predetermined to pass or fail.
Hypothetically, using a criterion-referenced approach means all candidates could pass the exam or fail the exam. As a result of the standard-setting process, the pass mark and pass rate may vary for each diet; however, the difficulty is considered when setting the pass mark.
The specific type of standard setting used in the MRCPCH is a Modified Angoff Method. For each exam diet a panel of professional judges who are subject matter experts grade the difficulty of the exam, by item, which results in a cut-score or pass mark.
Prior to grading the exam, judges are asked to conceptualise the notion of the 'borderline candidate'. The borderline candidate is a fictitious person who has a 50% chance of passing the exam. Once this has been established, the judges are asked to independently grade each item of the exam and rate out of 100 borderline candidates the percentage of how many will answer the question correctly. Once completed these rating are compiled and prepared for a panel discussion.
The judges will then come together as a panel to examine the ratings for each item in sequence and discuss each item’s rating. Judges will discuss disparity in ratings and different viewpoints. Once these have been considered, if they wish, judges will have an opportunity to re-rate items based on the discussions, taking into account:
- the judgements of the entire panel
- the definition of the borderline candidate
- the comments of the panel.
These judgements are the last ratings provided, which when averaged become the final criterion referenced score for an item. The mean of all these final scores is calculated and the nearest achievable mark becomes the pass mark for the exam.
For each MRCPCH Theory exam, a Standard Monitoring Meeting (SMM) is held, where any anomalous items will be identified through statistical analysis and candidate feedback and discussed for suitability. If it is decided that the items were unsuitable for use in the exam, they will be removed and will not contribute to the overall pass mark.