Moodle Workshop Grading Evaluation Phase

After completion of the Assessment Phase, the Moodle workshop moves to the Grading evaluation phase. The checklist in this phase has four items, including the final Switch to the next phase. The other three are:

• Calculate submission grades
• Calculate assessment grades
• Provide a conclusion of the activity

The first two of these are accomplished simultaneously, while the third may or may not have been completed in the workshop setup. In this phase, participants will see a linked list of the submissions they should have assessed, and will be able to view these, but not complete or alter any assessments. They will also see a message indicating that assessments are being evaluated, and that grades are being calculated.

Grading Evaluations

The calculation of grades for submissions and assessments is done automatically based on two settings determined by the instructor, although there are ways to affect the outcome of these calculations, as detailed below. The first setting determines the type of calculation used, and the second sets the manner the calculation is made, primarily controlling the strictness of it. The first setting is Grading evaluation method. There are currently two options available: Comparison with best assessment and Participation credit. Each of these types then has an additional control under Grading evaluation settings.

Grading Evaluation Setup

Comparison with best assessment looks at all the assessments made and calculates an 'ideal' assessment based on the average response to each element of the assessment. It then chooses which assessment is closest to this ideal, and sets it as the best assessment, assigning it a grade of 100%. The other assessments are then compared against this one on an element-by-element basis (not overall assessment total) to produce a percentage, based on how far from the best each element of the assessment is. The calculation can be adjusted in  the Grading evaluation settings by setting the Comparison of assessments setting. For this evaluation type, there are 5 levels to choose from - very lax, lax, fair, strict, very strict - with very lax resulting in higher grades, and very strict giving lower marks.

Participation credit provides a way to include assessments as a requirement for the workshop, but without grading their fit or quality. In this evaluation type, participants are graded on the number of allocated assessments completed. The requirements for completion are controlled by the Evaluation mode under Grading evaluation settings, as described below:

• All or nothing - All allocated submissions must be completed to receive 100% of the assessment grade. Anything less is 0%.
• Proportional - The assessment grade is calculated as a percentage of allocated assessments completed, e.g. 2 of 3 completed = 67%.
• At least one - To receive the maximum grade in this option, one or more of the allocated assessments needs to be completed.

Once the Grading evaluation method and sub-setting have been chosen, clicking on the Calculate grades button will run the grading calculation and fill the Workshop grades report. The calculation will result in two grades being calculated, the Grade for submission and the Grade for assessment. In addition to these overall grades being displayed large in the corresponding column, the grades for individual submissions are also shown next to each author or reviewer, as explained below. 

At any point in the Grading evaluation phase, you can change either the Grading evaluation method, or the Grading evaluation settings and select Re-calculate grades to see the effect of different grading types or settings within that type.

Viewing and Modifying Grades

After calculating grades, the Workshop grades report will be filled with calculated grades.The report consists of 6 columns, beginning with the name of each participant (sortable by first or last name), followed by a the name of the submission, with a link to view it and its assessments, and then four categories detailing the grades given and received for that participant. The sections below detail how to read this information, and use the links provided for making adjustments to the scores shown.

Grades Received

The first grade column in the Workshop grades report is for Grades received. This column lists each reviewer for the submission by the author listed in the first column. To the left of the reviewer's name are two scores, the first is the grade they gave the submission, and the second (in parentheses) is the grade they received for that assessment of this submission.

Grade for Submission

The second grade column is the calculated grade for the submission (out of the total specified for submission in the workshop setup),. This grade is at its most basic, an average of the assessments made by all reviewers. There are three ways for an instructor to modify this grade, particularly if one of the assessments is not in line with the others, either too high or too low.

Adjusting Grade for Submission

• Submitting an instructor assessment - In some cases, including when instructors feel that the peer assessments don't fully represent the quality of a submission, instructors may choose to assess the submission themselves. To do so, complete the following steps:
  1. Click on the name of the submission in the Submission column of the Workshop grades report.
  2. Scroll down below the submission and click on Assess. There will also be the option at the end of the assessment form to give this assessment greater weight (from 2 to 16) than the peer assessments. 
  3. Once the assessment is submitted, you will need to click on Re-calculate grades above the Workshop grades report to see the effect of the additional assessment. 
  4. Note: adding an additional assessment from the instructor will modify the calculated Grade for assessment for all other reviewers of the submission, as it will be treated like any other assessment in determining the ideal against which all assessments are graded.
• Weighting one of the assessments - If one or more of the assessments are deemed to be a more accurate reflection of the quality of the submission, the instructor can give extra weight to an assessment, causing it to be counted double, triple, etc. To do so, complete the following steps:
  1. Click on the scores to the left of the particular reviewer in the Grades received column.
  2. On the following page, at the bottom under the assessment are the Assessment settings. Next to Assessment weight choose the desired weight for this assessment from 1 to 16 and Save and close.
  3. Once the assessment has been weighted, you will need to click on Re-calculate grades above the Workshop grades report to see the effect of the additional assessment. 
  4. Note: weighting assessments will modify the calculated Grade for assessment for all other reviewers of the submission, as it will 'increase' the number of assessments used in determining the ideal against which all assessments are graded.
• Overriding the submission grade - If all the assessments for a submission seem out of line, an instructor can choose to simply provide their own grade for the submission directly. To do so, complete the following steps:
  1. Click on the name of the submission in the Submisssions column and scroll down to the bottom of the page to Feedback for the author.
  2. Next to Override grade for submission, choose the new grade to assign to the submission, and leave any optional Feedback for the author.
  3. After clicking Save and close the new, overridden grade will be displayed in green, with the old, calculated grade displayed with a strikethrough in red.
  4. Note: the final grade for submission can also be overridden in the gradebook itself, once the workshop is close, but this will not be reflected/displayed in the workshop itself.

Grades Given

The third grade column shows all of the assessments made by the participant in that row. To the left of the name of each author whose submission they assessed are the grades they gave to their peers, as well as the grade they received for that assessment (in parentheses).

Grade for Assessment

The final grade column in the Workshop grades report is the Grade for assessment. This represents the average of the calculated grades given to a participant for all of the assessments they completed. As with the submission grade, there are ways to modify this average grade for assessment, and with it the overall grade for the workshop.

Adjusting the Grade for Assessment

• Overriding one or more assessment grades - If an instructor feels that the calculated grade for an assessment is incorrect, because of how it did or did not align with the other reviewers, but that the reviewer might actually deserve a higher or lower grade, the grade for that assessment can be overridden. To do so, complete the following steps:
  1. Click on the assessment scores next to the reviewer's name in the Grades given column, and scroll down to the bottom of the displayed page to Assessment settings.
  2. Next to Override grade for assessment, choose the new grade for that assessment, and select Save and close.
  3. The assessment score to the left of the reviewer's name will show the calculated grade with a strikethrough in red, and the new, overridden grade in green.
  4. To see the effect the new grade has on the total Grade for assessment, you must click on Re-calculate grades above the Workshop grades report. 
  5. Note: this will not affect any other scores in the workshop.
• Adjusting the weight for an assessment - If an instructor feels that a given assessment is more representative than others of the same submission, they may choose to give that assessment more weight, causing it to be counted as multiple assessments. This will affect the calculated score for that assessment, as it will adjust the calculation of the ideal assessment against which it is graded. To do this, complete the following steps:
  1. Click on the assessment scores to the left of the reviewer's name and scroll to the bottom of the displayed page to Assessment settings.
  2. Next to Assessment weight, choose the new weight for that assessment (from 1 to 16), and select Save and close.
  3. Once the assessment has been weighted, you will need to click on Re-calculate grades above the Workshop grades report to see the effect of the additional assessment. 
  4. Note: weighting assessments will modify the calculated Grade for assessment for all other reviewers of the submission, as it will 'increase' the number of assessments used in determining the ideal against which all assessments are graded.
• Overriding the assessment grade - If an instructor wishes to directly adjust the total grade for assessment, without changing individual assessment grades, this can be done in the Gradebook once the workshop is closed, and grades report to the gradebook. Note: this grade change will not be reflected/displayed in the workshop itself.

Closing the Workshop

Once an instructor is happy with the calculated or adjusted grades for submission and assessment, and any optional Feedback for the workshop has been entered, the workshop can be closed by clicking on Switch to the next phase from the Grading evaluation phase checklist, or clicking on Close workshop under Closed in the final column of the workshop table. Once the workshop is closed, both the Grade for submission and the Grade for assessment will be reported to the gradebook and participants will see the optional Conclusion text entered for the workshop, and their results for  the workshop activity. The instructor will see the same Workshop grades report in their view, and participants will see both their Grade for submission and Grade for assessment shown, out of the totals possible. They will also see links to their submission, with its assessments and feedback, under Your submission, as well as the assessments they did, with their scores and any feedback, under Assigned submissions to assess.