Transcription of video "Introduction to online assessment"
What I’d like to do as part of this introduction video is give an overview of the different types of online assessment mechanisms there are. These are very broad categories, but as far as I know – and there’s always the chance that there’s a new practice or technology developed at any time – everything out there falls into one of these types.
The first is computer-based assessment, i.e. quizzes. Multiple choice questions, complete the sentence, matching words, - a whole slew of different formats but all pretty much come down to the same thing, a series of questions which the student can answer to assess their learning at a formative stage. The advantages are that they’re usually easy to set up, the marking is automated and can be repeated again and again by students so that students can test their knowledge. They’re usually really liked by students as they’re effective for developing rote learning, and are easily gamified. Gamification just means turning something into a game, so you can have leaderboards or unlock badges, and that can be rewarding for many students. The downside is that for many subjects you can only test very basic learning, memorising facts and so on with this format. For some subjects you can set up higher level tests, such as finding out information to test information literacy skills, or to undertake complex maths or engineering calculations for example. Coding is also very effectively assessed automatically, the computer runs the code to see if it works.
On the whole though, if you want to assess learning, it needs a human to do it.
Another form of online assessment is to set students the task of making an artefact. We’ve already included that as part of this course. I came across a paper by Janet McDonald and Peter Twining about this as a mechanism for assessment and they had this to say about it:
"Some students might be attracted to the “fun” element in activities, others might be motivated by the online interaction, whilst others might see activities as intrinsically “lightweight” in comparison with traditional course content. Obviously, this issue of participation is of critical importance for an activity-based approach, because the activities form a central part of the course, reflecting its philosophy. Where activities involve online communication, the issue of participation is even more critical, because non-participation in an activity by one student impacts on other students." P.605
Assessing activity-based learning for a networked course - British Journal of Educational Technology Vol 33 No 5 2002
Now making artefacts doesn’t have to be collaborative, but hopefully you’ve seen the additional benefits of doing so. Sometimes these artefacts can be accompanied by a presentation. Both of these options present difficulties and advantages with assessment. From my own experience the most pressing of these difficulties is “what is being assessed?” We should – I would argue – be focused on the quality of the learning that has taken place, but it can be difficult to disentangle this from the quality of the production. So for example, students making a video articulating the viewpoint of a character in a novel – we should be looking at how well they’ve understood the character, but camerawork, editing, acting, might all influence our decision.
If we’ve asked people to submit a presentation, are we looking at the PowerPoint/Prezi as a standalone artefact, or judging the content of that, plus the articulation by the presenter? And if the latter, can we take into consideration whether this is done with or without live sound, or no sound? There’s no one answer, but this needs to be made clear before setting the task, and to the other markers.
With collaborative activities then the opportunity for peer assessment arises. There are plenty of online tools to facilitate this – WebPA, PeerMark , PeerWise, iPeer and SparkPlus. The thing they all have in common is that, on the whole, students really dislike using them. Peer assessment is still seen as a deeply problematic process, with assumptions about its use being that it’s there as a means to punish non-performance and so on. It’s an important means to encourage collaboration and reflection, but I think needs to be carefully introduced and its role explained.
So, quizzes, artefacts and next; discussions. Online discussion is a really useful learning mechanism, but often is not always engaged with by all the students and actually being marked on something is a sure-fire way to encourage participation. Assessing online discussions is still not something we’re entirely successful at. It can be done numerically very easily, and automatically; post a comment, reply to another’s comment, and so on can be measured. However, if we’re looking at the level of contribution, it becomes more difficult to do objectively.
It can also be extremely laborious going through all the posts ensuring that the rubric for the different levels of engagement are met, which is why it’s still more popular to actually use an essay that draws on the learning from the engagement with the discussion boards, rather than the discussion boards directly. The plus points from this are that it’s a format everyone is familiar with, everything you need to mark is in one place, and you don’t need to revalidate anything in the move to doing the course online.
The downside is that it continues to reward students more for traditional skills rather than accentuating some of the newer skills that can occur with newer technologies.
A format that draws on the strengths of the traditional and the online environment is that of the patchwork, portfolio or hypertext assignment. The assignment for this course is a patchwork text, and it’s explained in more detail in the assignment brief, there’s also an example.
The rationale for these sorts of assignments – and they are widely used in online courses – is that throughout a course many different documents may be produced. For example in education courses teachers may develop lesson plans, blog posts, teaching observations, examples of student feedback and so on. A portfolio format means that each of these can be drawn into the overall submission, in their original form. Normally an overarching summative document is produced, with these separate pieces of evidence linked to from that document, providing a single reflective narrative, but also a wealth of useful supplementary information. The value of these documents is that they produce an excellent collection of someone’s practice, and in the stitching together of the separate pieces, they again can be reflected upon and drawn together. Also they can make the best use of pre-existing artefacts produced by previous tasks, so the entire course can build directly towards the final assignment.
The downside can be that with all of the separate aspects, they can become enormous documents, and their fragmentary nature can sometimes make them difficult to mark. If the student doesn't properly link to something, then it can be missed when going through it. The brief needs to be written in a way that ensures that the entire piece does not get out of hand.
So those are the main formats, quizzes for formative work, creating artefacts, analysing or counting discussion board postings, patchwork texts or the standard reliable essay. It would be good to see what your opinion is of all of those and so I’m looking forward to reading what you are saying in the discussion forums. Good luck with the rest of the course.