We frequently receive questions concerning the difference between Raw Score and Cut Score grading and which one should be used. This article aims to raise awareness about these methods of reporting students' grades and help the educator decide which one to use with their courses.
What is a Cut Score?
We should begin by defining the term "Cut Score." A cut score, or cut-off score, represents the passing score of a given assessment. It is the minimum score that would indicate that a student possesses proficiency or minimum competency of the content included on the assessment. Simply stated, a cut score is the minimum score that a student must achieve to pass the test (Cut-off score, 2014).
Establishing the passing score, a.k.a. the cut score of an assessment, is known as standard setting. The passing score can be set arbitrarily (Students need an 80% to pass), using norm-referenced grading (grading on a curve), or by using a criterion-based method for establishing cut scores (Cizek, 2012).
Raw Score Grading
Most educators and students are likely most familiar with the Raw Score grading system. Raw Scores represent the percentage of items answered correctly on a given test and are easily calculated by dividing the number of items answered correctly by the total number of items on the test (Johnson, 2021). For example, Student A answered seven items correctly on a 10-item test; therefore, Student A would receive a Raw Score of 70%. Raw scores are often used with arbitrarily set cut scores where a program sets a passing score that the students must achieve to succeed. Though the raw score percentage represents the student's statistical performance, it does not consider the difficulty or sophistication of the items included on the test. Therefore, it should not be regarded as a good indicator of content proficiency or minimum competency.
Norm-referenced grading is another method that uses the students' raw scores to determine success. This method does not use a predetermined arbitrarily set cut score. Norm-referenced grading uses the students' raw scores and compares them against the performance of their classmates. This method does not establish a passing score that determines proficiency or minimum competency and potentially allows students that possess minimum competency to fail an assessment based on their percentile ranking (Cizek, 2012).
Cut Score Grading
Cut score grading systems use a criterion-based standard-setting method to establish each test item's cut scores. The assessment's passing score (cut score) is the average of the cut scores included on the assessment. Since criterion-based standard-setting methods take the difficulty or sophistication of an item into consideration, they provide a better picture of a student's proficiency based on a predetermined score that would indicate minimum competency (Cizek, 2012). There are several acceptable criterion-based standard-setting methods available. This article will review two methods, Angoff's and Nedelsky's methods for establishing cut scores.
Angoff's Method
Angoff's Method uses a panel of subject matter experts (SME) to establish the cut score for testing items. Using this method, the SMEs would review each item and estimate the percentage of minimally competent students who would answer the item correctly based on their professional opinion. The item cut score would then be calculated by taking the average of the percentages provided by the SMEs. The assessment's passing score (cut score) would be calculated by finding the average of the cut scores for each item included on the assessment, thus determining the passing score for a minimally competent student (Cizek, 2012).
Nedelsky Method
The Nedelsky Method also begins with a group of subject matter experts (SMEs) to determine a cut score for a test item. With this method, each item is reviewed by the SMEs, who assign a cut score to the item based on the number of distractors (wrong answer choices) that a minimally competent student would be able to rule out (Cizek, 2012). The more seductive or sophisticated distractors present would decrease the likelihood that a minimally competent student would answer the item correctly, thus increasing the complexity. Item cut scores are determined by the number of seductive or sophisticated distractors available. A multiple-choice item with four answer choices would have a cut score assigned using the following criteria:
# of good distracters - Item Cut score:
0 - .90 or 90%
1 - .60 or 60%
2 - .45 or 45%
3 - .36 or 36%
The item cut score would then be calculated by taking the average of the cut scores provided by the SMEs. The assessment's passing score (cut score) would be calculated by finding the average of the cut scores for each item included on the assessment, thus determining the passing score for a minimally competent student (Cizek, 2012).
Platinum Educational Group Criterion-Based Standard-Setting Methodology
Platinum Educational Group (PEG) uses criterion-based standard-setting to establish the cut scores of every item in our test banks and on our high-stakes assessments. Depending on the item type, PEG uses either a modified Angoff's or modified Nedelsky's method to establish each item's initial estimated cut score. Items are piloted, and upon completion of the pilot process, each item receives a calculated cut score based on the validation statistics determined by actual student responses.
Conclusion
Hopefully, this article provided the necessary information to help the educator make an informed decision about using Cut Score-based grading. The author's opinion is that due to their failure to consider the complexity of the items included on a test, Raw Score and Norm-Referenced grading methods are poor indicators of a student's proficiency or competency. The better choice is to use Cut Score based grade reporting.
If you have any questions or would like to talk more about these topics, we encourage you to reach out to our Education Team.
Reference
Cizek, G. J. (2012). Setting performance standards: Foundations, methods, and Innovations (2nd ed.). Routledge.
Cut-off score definition. The Glossary of Education Reform. (2014, February 11). https://www.edglossary.org/cut-off-score/.
Johnson, S. (2021, November 5). What are raw scores? The Classroom | Empowering Students in Their College Journey. https://www.theclassroom.com/raw-scores-4447.html.