1. Assessment Plan
An assessment plan contains detailed information about the constructive alignment (the alignment of learning objectives, teaching/learning activities and assessment) of your course assessments, and how the assessments contribute to the course’s final grade. In this chapter, it is explained how to construct, analyse and improve such an assessment plan for constructing or improving the assessment of your course. You will also see the assessment plan in detail, and read an example of what it could look like.
1.1. Set-up of the assessment plan
To give a good overview of the constructive alignment of your course assessment, include an assessment overview (a tabulated summary of the assessment plan). The assessment overview can be included in your Brightspace course for your students to see what assessments they can expect and can be used to get insight on the following at a glance:
- Constructive alignment of assessment methods with learning objectives;
- Alignment of formative and summative assessments with feedback;
- Grading methods;
- Timing of assessments and feedback.
It is recommended to include the elements listed in Table 1. These elements will give insight on the level of validity, reliability, transparency and feasibility, in your course assessment. An example of an assessment plan can be found in Section 1.2.
In the table, you can summarise all formative and summative assessment in your course.
- Summative assessments test how well students master the learning objectives. Summative assessments may be classic written exams, digital exams, assignments that students perform at home or during a computer lab, performance, presence or attitude during for example a project, lab, excursion or class. Summative assessments usually lead to a grade (1-10), and/or a pass/fail decision.
- The goal of formative assessment is to monitor student learning to provide ongoing feedback that can be used by instructors to improve their teaching and by students to improve their learning. Therefore, formative assessments are assessments that usually do not contribute to the grade of the course. Students should receive feedback on how well they master the learning objectives, and this can be done by giving teacher/TA feedback, automated computer feedback or peer feedback. The resulting feedback is focussed on criteria that cover the tested learning objectives and the assignment is at the same level as the summative assessment. This way, the formative assessment prepares students for the summative assessment.
Let us look at the assessment overview and plan in more detail.
1.2. Example assessment plan
An example assessment plan can be found in Table 2. The main characteristics and considerations of an assessment plan will be discussed in the remainder of this section, based on this example assessment overview.
-
Assessment name (assessment type)
Assessment method
Individual or group
LOs
% of final grade
Grade type
Minimum grade
Deadline/ date of assessment
Grading method
Date of announcement of grade/ feedback
Feedback on assessment outcome
ECG analysis (assignment)
Report, code, presentation
Group
3,4,5,6
20%
Grade
5 for the weighted average of two assignments
End of week 4
Rubric
End of week 5
Rubric with a tip and a top, feedback is focused on EEG analysis assignment and on the exam.
EEG analysis (assignment)
Report, code, presentation
Group
5,6,7,8
30%
Grade
End of week 9
Rubric
End of week 10
Rubric with a tip and top, focused on the exam.
Excursion Medical Company
Attending the excursion
Group
3-8
0
Pass-fail
pass
Week 5
None
Immediately after the excursion
NA
Practice exam
2 open questions with 4 sub-questions, 40 MCQs with 3 options each
Individual
1-4, 7
0%
NA
NA
Start of week 10, in class
Answer model
Immediately after the practice exam
Exam and model answers are on Brightspace, including references per sub-question to page numbers and exercises in the book. Students can ask questions in class after the exam.
Exam
2 open questions with 4 sub-questions, 40 MCQs with 3 options each
Individual
1-4, 7
50%
Grade
5
End of week 11
Answer model
Week 13
Debriefing after exam.
Exam and model answers published after exam on Brightspace, see practice exam.
-
The reason that there is a minimum grade for the assignment, is that this is the only place where LO5 and LO6 are summatively assessed. However, the grades of the two assignments can compensate each other. Students have the biggest problems with mastering LO5 and LO6. Since both the assignments contain these LO’s, and because the second assignment has a higher weight, students can use the feedback on the first assignment to improve on LO5 and LO6 in the second assignment. Therefore, it is fair that they can compensate the assignment grades, since they partially measure the same LO’s and that redoing assignment 1 would be partially redundant and unnecessarily increase the workload for students (doing an extra assignment) and lecturers (grading these assignments).
-
Since the excursion is mandatory for finishing the course, but there may be cases where students are not able to visit the company, students who have a valid reason not to attend the excursion (to be determined by the study advisor) are allowed an alternative, for example, writing an essay on the company visited by the rest of the class.
-
Since the assignments change every year and reflect the state-of-the-art developments in the field, students cannot keep the grade the next year. Furthermore, the assignments are group work, and if students passed the group assignments, but failed the course because they had a low grade on the exam, they may not have contributed enough to the project after all, since they apparently lack some knowledge and skills. Finally, speaking from my own experience: In previous years, students did not have to retake the assignments. However, students who did not get a pass for the assignments before taking the exam, almost never passed the exam. To be able to pass the course, students would need a second chance to complete the assignment successfully.
Since the excursion is the same every year, students are not required to go on the excursion a second time.
-
For the assignments, students get their grade, rubric and tip & top one week after the deadline of the assignment. The lecturers have enough TAs and lecturers to do this. Furthermore, students get this feedback before starting the next assignment and one week before the exam and are encouraged to use this feedback to work on assignment 2, and to study for the exam. The tips & tops are only focussed on the next assessment so that the students can actually apply the feedback.
Students are advised to take the practice exam at home, once they think they are well prepared. If students get stuck on a question, they can use the hints on a hint-form, which will refer them to a page or formula in the book (the exam is an open-book exam), or to related exercises, which they can use to get to the answer or practice more. After finishing the practice exam, the students can compare their answers to the model answers. To make sure that students realise that there are more correct ways to get to the correct answer, multiple answer routes will be included in the model answer.1
The model answers will be published on Brightspace. In these model answers, each model answer to a sub-question will have a reference to a page or formula in the book and to related exercises, so that students can study and practice that part, in case they will take the resit.
Directly after the exam, students are invited to a neighbouring lecturing hall, in which the lecturers will discuss how the questions could have been answered. The lecturers will emphasise that the goal of this meeting is to enable learning after the exam, not to discuss the quality of the questions, since students will be able to inspect their work and file complaints in another meeting, after the grades have been announced.
Just like for the practice exam, the model answers will be published on Brightspace with references to the book and exercises, so that students can study and practice, in case they will take the resit.
Of course, circumstances in your course are different.
-
Whenever a minimum grade is present, it is recommended to grant students a retake, or enable them to deliver a new version of project reports. The reason for this is to diminish the number of assessment hurdles for students, since grades are not perfectly reliable and erroneous grading may keep students from progressing with their studies. That is why there is a retake for the assignments as a whole, and a retake for the exam. Another reason to average the grade of the assignments, is to lessen the workload for the teaching staff. If a crucial learning objective is only assessed in a single assignment, it would be good reason not to average the grade, and instead to require students to earn a minimum grade for a single assignment.
1.3. Constructive alignment of assessment methods
TU Delft works with the principle of constructive alignment. For your students to complete your course, they should demonstrate their knowledge and skills in some way or another. They demonstrate this by completing the summative assessments that you set for them. Once they have completed an assessment, you then evaluate/grade them based on certain predefined criteria. These criteria should be based on the learning objectives of the course.
Now, to enable your students to complete these summative assessments, you need to provide them with various learning activities to enable them to prepare. This might include course content, excursions, lectures, workshops, formative assessments etc. Lastly, you close the loop by checking that absolutely everything in your course (whether it is content or assessments) will enable your students to reach the learning objectives for the course. If so, your course is constructively aligned.
All assessments should cover at least one learning objective. If assessments do not aim towards students meeting the learning objectives for the course, they can be considered redundant.
The following two sections will discuss how the choice of assessment method as well as the balance between formative and summative assessments will influence the constructive alignment of your course. This text is adapted to the TU Delft situation from (Dunn, Selecting methods of assessment, 2018).
-
Assessment methods are, for example, written tests, presentations, and projects. It is important to select the right type of assessments for students to show whether they have reached the learning objectives. For example, if you want to assess students’ communication skills, you would rather have them do presentations than a multiple-choice test.
The main reason to choose one assessment method over the other is that it enables you to get a valid measure of how well a student masters a learning objective. The assessment should be authentic for you to be able to assess what you should be assessing.
During an assessment, students should be able to demonstrate their capabilities, unhindered by the lack of experience with an assessment method. If you use an assessment method that students are not trained in (for example oral exams, group assignments), the assessment method should not prevent students from maximum performance.
For example: When the learning objective is to (orally) explain and defend design choices for a given case, it is okay to use oral exams, if and only if students can practice orally with this during the course and receive good quality feedback on the criteria that they will be assessed on, while practicing (formative assessment). And if all measures have been taken to ensure validity, reliability (assessor objectivity, as well as creating a safe atmosphere to enable maximum student performance), and transparency, since these quality requirements for assessment are more easily violated than using other assessment methods.
Keep in mind that the learning objectives contribute to the overall aims of the programme and may include the development of (inter-)disciplinary skills (such as critical evaluation or problem solving) and support the development of vocational competencies. Ideally, this should be planned together with the relevant colleagues so there is a purposeful assessment strategy across a degree program.
To motivate students to do the assessments and to do them well, it is important to validate why any particular assessment type was chosen. This works best if the assessment is authentic, i.e., if they will perform the activity during their working life, or otherwise during a follow-up course. This will make the assessment much more relevant for your students and will also help them decide if they want to pursue a career where that type of activity is common.
Nightingale et al. (1996) provide eight broad categories of learning outcomes which are listed here. Within each category some suitable methods are suggested. Note that oral exams are not included, since they are only advised when the learning objective requires it, for example ‘being able to defend one’s ideas within a research team’
Table 3: Categories of learning outcomes and corresponding assessment methods (Nightingale et al, 1996)
Please note that these suggestions are not focussed on engineering education, and you as a lecturer and as an expert in your own field will probably have other ideas for assessment methods that are more authentic in your situation. It will hopefully expand your view on the possibilities of assessment methods beyond the classical closed-book exams
1.4. (Dis)Advantages of open and closed-ended questions
In general, multiple choice questions (MCQs) in which students have to demonstrate understanding, are very useful in a classroom setting were students can discuss their answers. This can deepen their understanding and analytical skills.
For summative assessment, there are several reasons to decide (not) to use multiple-choice questions.
If you choose to use closed-ended questions, such as multiple-choice questions (MCQs) in an exam, keep the following advantages and disadvantages in mind:
-
MC questions that test lower levels of Bloom, can be answered quickly. Therefore, you can include many questions, which can increase validity and reliability.
The grading can be very fast and will automatically provide you with data for doing item analyses.
It is possible to test higher cognitive levels of Bloom, but more time need to be spent on creating these questions. A good idea is to use case studies which the students have to analyse, and then base your questions on the cases.
-
Generating MCQs takes a lot of time and should not be seen as an easy way out. A lot of care needs to go into developing really good questions, and building a large enough library of questions can take a while. Keep in mind, for example, that all distractors must be equally probable.
If you want your students to recall facts ('remember' level of Bloom), do MCQs measure whether the students can recall the facts, or do MCQs merely measure whether your students can recognise the correct answer between false answers? Do you measure whether your students will be able to produce the answers by themselves?
The same holds for higher levels of Bloom, which has as an extra problem that students will most likely need more time to answer each question. Since you will need quite some MCQs in order to develop a reliable test, this might be problematic.
For MCQs that need a lot of thinking steps, like ones with calculation or difficult case studies, generally no partial credits are given to partially correct answers, whereas for equivalent open questions partial credit would be given. Please note that it is possible to give partial credits to partially correct answers in Contest (paper-based MCQs), and probably also in other software. However, this will influence the guessing score.
On the other hand, the student might have guessed the correct answer, without having studied the subject. In open questions, the student would probably have gotten 0 or very few points.
The latter two points will create noise in the grade, which will make the grade less reliable. That is why you will need more questions for MCQs than for open questions in order to construct a reliable exam (see 5.1.d, ‘Number of exam questions’).
1.5 Possibility for feedback: formative assessment
It is important to include a balanced combination of formative and summative assessments in your course. While summative assessment is used to collect evidence on the extent to which students master the learning objectives, formative assessment is meant to steer learning. Let us look at this in more detail.
-
The main difference between formative and summative assessment is that formative assessment does not contribute (significantly) to the final grade of the course. For formative assessments, the students should focus on their own learning (Garfield & Franklin, 2011), make mistakes and experiment with new ideas without any significant consequences for their final grade. This is assessment for learning. Furthermore, you can use the information on student performance to adjust the course to the need of this particular group of students.
-
Having feedback mechanisms in place during group work assignments is very important. If your students first complete the summative assignment and only then receive feedback, it is too late to improve their learning objective achievement. Instead, have your students for example give feedback on each other’s work half-way through, or at certain milestones in the project. If there are problems with any of the performance areas, they will still have time to correct these, instead of reaching the end when it is too late to address any issues.
-
Formative assessment has been shown to have the following positive effects (Cauley & McMillan, 2010) (Shute, 2008) (Wiliam, 2011), for example:
- Pointing out misconceptions and allowing them to be corrected;
- Providing valuable information for the adjustment, or improvement of instruction;
- Allowing students to be more actively engaged in their own learning and increasing commitment.
-
Formative assessments have to meet certain conditions to enable successful completion, for example:
- The teaching team needs to believe in the value of each formative assessment, set high expectations from the start, and follow a consistent approach throughout the course;
- The purpose and reason for each formative assessment have to be explained to students, as well as the goals and the evaluation criteria;
- Students have to want to be actively involved in their own learning;
- Feedback must be timely (as soon after completing the assessment as possible) and contain information about how the student is doing, where the student is going and what (s)he still needs to do to get there.
-
The students are not graded for these assessments, how do you get the students to complete them?
- Manage the students’ expectations at the start of the course (let them know what they can expect and what will be expected of them);
- Make the formative assessments their gateway to performing well on the summative assessment (it has to be worth their time coming to class);
- Clarify what kind of feedback students can expect and how this will help them;
- Coordinate the assessment methods, deadline, and bonus point arrangement with other courses in the programme that are running that period and year, so that the assessment activities do not clash;
- Adjust the type of feedback to the year the students are in.
Effective feedback
The most important thing is that you offer students the opportunity to get feedback on their performance, per learning objective, at the level of the summative assessment, before grading them. You can do this, for example, either by writing general feedback, personalised feedback, using rubrics, or a combination of these.
-
One purpose of giving feedback to students is always to steer their progress. This means that feedback should at least answer the following questions for the student:
Feed up:
- Where should I go to?
- What is the required level?
Feedback:
- Where am I right now?
- What is my current level?
Feed forward:
- What is the first step I need to take in order to get closer to my goal?
- What can I do now to improve your level?
The student should know what the goal is, and why it is important to reach this goal. This should also be made clear before they start with the assessment.
-
Furthermore, the specificity, practicability and respectfulness of the feedback can be ensured by using the ‘Observation, Result, Advice’ structure in the formulation of your feedback, no matter whether the feedback is positive or focused on improvements:
STRUCTURING FEEDBACK
Observation
What did you observe? Start with ‘I noticed.../I observed that.../In question 2 I see that...’ and describe your observation. Your observation should be based on evidence.
Effect
What was the result? Describe the effect it had on you, or the effect it might have on other readers/listeners/professionals.
Advice
Give a concrete hint on how to improve or do things differently, or (if correct) encourage the student to maintain this behaviour.
By following these steps, you will both indicate why (i.e., validations) and how the improvement could be made, in an objective way that is specific, respectful, and actionable.
Here are three examples of how to apply the ‘Observation, Result, Advice’ structure:
Feedback example presentation:
- I noticed that during the presentation, you talked quite fast.
- For me, this made it hard to follow your talk.
- Maybe you could practice on speaking slower. If you are talking fast because you are nervous, you could try doing some breathing exercises before the presentation. There are plenty of examples on the Internet.
Feedback example code:
- I noticed that you did not use section headers or comments.
- This made it very difficult for me to understand what part of the code is doing what, and it took me a lot of time to understand it. As a result, your grade for ‘code readability’ is low.
- You can improve your code’s readability by using logical section headers and adding comments. You can find some examples on page 13 of the book.
Feedback example report:
- I could not find a critical discussion of your research method in your research paper.
- Therefore, I could not check how you have considered the limitations of your method in your conclusions. As a result, you have a low grade for ‘reflection on methodology’.
- Please add a critical reflection on your methodology in your discussion. You can have a look at the example research paper on Brightspace, which has a good example of what is expected.
-
Watch the video below that explains four characteristics of effective feedback.
As you can imagine, giving such comprehensive feedback to large classes can become laborious. This could partially be automated for online assessments though. A good alternative would be to use rubrics (assessment grids), because they will tell the students exactly what was expected of them and on which level they performed. How to go about this will be discussed in detail further on in this manual.
1.6. Digital assessment tools
There are tools that help you to grade paper exams online. Some of these tools allow you to divide the grading work amongst graders, grade anonymously, grade per question, and grade simultaneously with your fellow graders.
-
The following digital exam tools are centrally supported and follow the archiving regulations for you:
For summative and formative digital exams, Ans is the recommended solution at the moment. Ans is designed to support paper, digital and hybrid exams.
For more information and support, contact ExamSupport@tudelft.nl for summative exams and Teaching-Support@tudelft.nl for formative assessment.
For a current overview of centrally supported tools, take a look at this page.
-
The following tools are not centrally supported and therefore the course’s examiner is responsible for securely archiving and destruction of the data after the retention period has passed.
- Zesje (open source, Latex based)
- Work2grade (TBM, Pieter Bots)
-
There are also centrally supported peer evaluation/feedback tools:
- BuddyCheck (to improve behaviour and group dynamics, follow-up of Scorion)
- FeedBackFruits
Reminder: the examiner is responsible for giving the grades!
In addition, the following tools are available for evaluating and giving feedback to student deliverables like reports and presentations:
- Brightspace Assignments
- In Brightspace Assignments, you can switch on the plagiarism scanner:
- Ouriginal (will be replaced in 2023)
You can contact Teaching Support if you need more information, or if you want to use other tools for assessment.
For more information on all centrally supported tools, including (peer) feedback tools, please see this page.
1.7 Regulations and guidelines for assessment
Your assessment plan should be in line with the various regulations in place for your faculty. In this section you will find some basic information on which laws, regulations and policies might apply and where to find them. These are listed in hierarchical order:
-
The Wet op het hoger onderwijs en wetenschappelijk onderzoek (WHW; Law on Higher Education and Scientific Research, unfortunately only available in Dutch) is the law that determines how the universities in the Netherlands are organised. It also states that each programme should have a document with the teaching and examination regulations (TER).
-
All regulations regarding admission, tracks, education, exams, etc. can be found In the TER (in Dutch: Onderwijs- en Examenregeling, OER).
Article 4 in the TER describes the programme’s exit qualifications. The exit qualifications are the ‘learning objectives’ of the entire programme. The combination of learning objectives of individual courses should cover the exit qualifications of the programme. It is up to all lecturers at TU Delft to ensure that students meet all exit qualifications by the time they receive their BSc or MSc diploma. It is therefore important to take note of the following:
- Which exit qualifications (also called final attainment level, or intended learning outcome of a programme) should your course contribute to;
- Whether there is a number of courses that contribute to an exit qualification;
- If yours is the only course contributing to a specific exit qualification.
This has implications for the level at which you need to assess specific learning objectives and the importance of the assessments in your course. Furthermore, it influences with which course coordinators and lecturers you will interact to align and finetune your learning objectives and assessment plans.
For each subject that could be relevant to your assessment plan, the applicable section (§) and article number(s) (Art) are given for Bachelor and Master programmes in Table 4. The numbers are based on the model TERs and actual numbers can vary slightly per programme. Here is also a link to all TERs, IRs and R&G of BoEs for all bachelor and master programmes at TU Delft.
Table 4. Overview of assessment related subjects (first column) that are covered in the Teaching and Examination Regulations (TER, second column)
Teaching and Examination Regulations
Obligation to participate in practical exercises
§3, Art 11.2
§5, Art 23
Number and times of examinations per year. Refers to the IR.
§5, Art 16 & Art 17
Validity duration of examinations (and sometimes of partial examinations)
§5, Art 22
Type of examinations (assessment method): refers to the appendix (IR).
§5, Art 16
Oral exam: number of students that is assessed at the same time, number of examiners, public nature of the exam, identity of the student
§5, Art 18
Announcement of grades (when, how and possibility for appeal against grade)
§4, Art 19
When students are allowed to inspect their assessed work, the questions/assignments and the criteria used for grading (answer models/rubrics) (and make a copy).
§4, Art 20
When and how a discussion of oral or written exams takes place
§4, Art 21
Take the time to make sure that your course assessments are in line with the requirements.
-
The ‘Board of Examiners’ (BoE) appoints the examiners to conduct examinations. Secondly, it checks the quality of the assessment of a programme. In addition, it grants exemptions to individual students and decides what measures will be taken in case of fraud.
In the ‘Rules and Guidelines Board of Examiners’ (R&G BoE), you can find a lot of information that is applicable to many stages of the assessment cycle (see page 19 of ‘How to assess students through assignments’ by Evelyn van de Veen, 2016), namely on test design, construction, administering and marking.
Table 5. Overview of assessment-related subjects (first column) that are covered in the Teaching and Examination Regulations (TER, second column), and Rules & Guidelines of the Board of Examiners (R&G BoE, third column)
Rules and Guidelines of the Board of Examiners
Fraud
Art 7
Multiple examiners examining one examination
Art 10.1
(re)taking exams in different forms
Art 1.2-10.4
Online proctored examination
Art 11
Quality requirements of examinations
Art 12
Procedure during examinations
Art 13
Grading, rounding, partial grades, minimum grades, answer model
Art 14
Registering results in OSIRIS
Art 15
Archiving of work and results (duration)
Art 16
Projects
Art 20-21
Graduation projects
Art 22-25
-
At TU Delft, each faculty has developed their own assessment policy document that is based on the central assessment policy. The guidelines in these documents are usually very broad and general, but in some cases, they contain very practical information that needs to be followed step-by-step. For example, it might contain regulations on when to exclude questions from calculating the final results, based on outcomes of the test analysis, or how to calculate a score (grade transformation). The assessment policies of the faculties and, if applicable, of programmes, can be found on intranet.
1.8. Quality requirements for assessment
In chapter 1 ‘Principles of assessment’ in Van de Veen (2017), you will find a detailed description of the quality requirements for assessment.
The following table provides a checklist that you can use to evaluate whether your assessment plan, and your individual assessments meet the quality requirement of your assessment. Some of the requirements are explained in more detail in the following checklist:
Using the quality requirements to improve your assessments can improve the quality of your course as a whole. You might find, however, that optimising your assessment for one of the requirements compromises the level of quality according to another requirement. There will almost always be a trade-off, so it is up to you to decide what is most important for your students and your course.
For example, medical students might not always get the opportunity to perform certain procedures on real patients during their studies. However, they still have to be evaluated. Mock-ups are usually used to simulate scenarios (making the assessment practically feasible), but this compromises validity of the assessment.
References
For a list of references used in creating this manual please visit this page.
1. Some lecturers choose to publish the real answers from several students who used different approaches that led to a correct solution. This will stimulate students to find their own creative solutions.