eAssessment Scotland 11 – A conference review from an accessibility perspective

The third annual eAssessment Scotland conference has just finished – as an event conceived, born and brought up in Dundee it’s now firmly established in the calendar for anyone interested in educational assessment and how technology can be used to optimise the quality of the assessment process.

Using ICT, particularly the Web, for assessment has particular potential to improve the accessibility of the learning experience for disabled learners and tutors, hence my interest. Here are a few notes on the keynote talks and parallel sessions I attended, from an e-accessibility and inclusion perspective.

Steve Wheeler: Assessment in the Digital Age

The opening keynote by Steve Wheeler (Associate Professor of learning technology at Plymouth University, @timbuckteeth on Twitter) was entertaining and though-provoking. His theme was on the differing ‘dialects’ of assessments and how the differences in the way learners and assessors  focusing on how assessments need to be focused on “preparing learners for a future we can’t describe”. He described different approaches to assessment that may become more relevant and appropriate. Examples include adoption of personal web tools to demonstrate competencies, and triadic assessment (involving the learner self-assessing, the tutor and peer-assessment).

In each case, the key for inclusive assessment seems to be the tools and methods that facilitate creation of content, document progress through a learning process, and allow communication between learner and tutor. If these tools have usability or accessibility problems that prevent them from being used effectively – e.g. limit what content can be added and when, or limit an individual’s ability to participate in a discussion, then disabled learners – and tutors – will be adversely affected.

Becka Colley: The student perspective on assessment

Becka Colley, (Dean of Students at Bradford University, @veggieg3ek on twitter) gave a student perspective on assessment, based on personal experience in gathering student feedback at Bradford. The nature of the student population is changing and becoming more heterogenous, with more fragmentation into different groups; more students staying at home, increasing diversity of entry routes, and increasing numbers working while studying. Learning styles are also changing; expectations of the quality of the learning experience are increasing, so assessment methods need to recognise this.

For me, this is an extension of the definition of an inclusive learning environment – dealing with different learning styles and different access circumstances and methods (including devices such as assistive technologies). So, evidence that demonstrates the benefits of recognising and accommodating, for example, the rise in use of smartphones and tablet devices gives strength to a more inclusive approach to the design and use of ICT in the learning environment.

Becka also emphasised the need for authenticity in assessment – the capability of a learner to perform meaningful tasks in the real world. Again, I saw a parallel in the accessibility world – a move from a guideline- or legislation-driven approach to inclusion that might create a technically but artificially accessible experience, towards one that focuses on an inclusive and genuine user experience.

Morning Seminar – Learning Management Systems and WordPress

Then, I attended a parallel session with two presentations:

First, Ian Pirie and Stewart Cordiner talked about the development of Edinburgh College of Art‘s strategy for assessment in the digital age. This included the design of a bespoke learning management system (LMS) to reflect the institution’s evolving approach to assessment. Time was short so we saw only a few snapshots of the system, but if it can avoid the complex navigation and information architecture that seems to bedevil the usability and user acceptance of so many LMSs and VLEs, then it deserves to succeed.

Then, contrasting this approach, Natalie Lafferty (@nlafferty) talked about the increasing use by Dundee University’s School of Medicine – tutors and students – of a free, out-of-the-box-but-extensible content creation tool – WordPress. WordPress provides an easy way for authors to produce digital content and start conversations in a relatively constrained environment. I’ve previously talked about WordPress Accessibility from a content consumer’s perspective – so, if accessibility is sorted at the start through appropriate templates and policy on using plugins for embedded third-party content, there’s relatively little room for significant accessibility issues to be introduced by editors.

Pamela Kato – Simulated stress for young medics

The after-lunch keynote was delivered by Pamela Kato (Visiting Research Fellow, University Medical Center, Utrecht), and discussed how serious games can be used to enhance medical education, particularly from the perspective of addressing and reducing incidence of medical error. We saw a demonstration of how a game can be used to simulate stressful clinical situations, encouraging to medical students learn how to control physiological state and hence improve performance.

Serious games – which in contrast to simulations rely more on fantasy or drama, and have enjoyment (and possibly even addiction!) as an explicit goal – have clear potential to make inaccessible real-world experiences available to those who couldn’t otherwise experience them. However, the complex visual and aural representation of an environment that a serious game might present means that people with sensory impairments may be unable to effectively engage with the game. If fine motor skills are required for successful performance, this will obviously affects people with reduced manual dexterity and with more severe physical impairment.

That’s not to say that gaming and accessibility are mutually exclusive. In some cases, excellent hand-eye co-ordination may be considered an essential attribute of a player – so there are limits to what can be made “accessible”. However, there is also an active games accessibility research and development community, looking at how the gaming experience can be made more accessible – see for example the Game Accessibility project. This involves pushing technical boundaries, but also a thoughtful user experience perspective – what can we reasonably expect the game player to experience, and how can we make that as inclusive as possible?

Using Google Forms for Assessment and More

I then attended a workshop on using Google Forms and Spreadsheets for data capture and management. The workshop was hosted by Martin Hawksey (@mhawksey), educational technologist from the sadly now-closed JISC Regional Support Centre N&E Scotland. The tasks, focusing on form design and data management using Google Forms and Spreadsheets, was well-prepared but in the hour available we really didn’t have enough time available to work through everything. Useful homework to complete later though!

The ubiquity, functionality and (lack of) cost make Google Apps incredibly attractive as tools for creating assessments. Frustratingly, from an accessibility perspective, there are a number of issues across the suite of Google Apps that prevent them from being as useful as they could be. Keyboard access and screen reader usability in particular are known to limit the ability to independently use Google Apps -the JISC Techdis Web2Access resource has provided detailed reviews for Google Docs, Google Calendar, Google Maps and Google Reader.

Accessibility problems with Google Apps are also illustrated in a series of videos by the US-based National Federation of the Blind (NFB); who take a more fundamental view, issuing a statement in March 2011 accusing educational institutions who adopt Google Apps as discriminating against blind people.

In response, Google – who ironically employs some of the brightest accessibility researchers around – has announced an accessibility survey to take place in September 2011, in association with the American Council for the Blind (ACB). Until Google addresses these accessibility issues, though, it should not be assumed that all learners will be able to independently and successfully use all functionality – and accommodating this limitation needs to be built into a strategy for using Google Apps in teaching and learning.

Donald Clark: When assessment goes bad

The closing keynote from Donald Clark (who blogs at Donald Clark Plan B, @donaldclark on Twitter) was refreshingly provocative, and generated a fair amount of discussion on the Twitter backchannel (see #eas11 for conference tweets – not yet sure if these have been archived?). The role of simulations, and the importance of recorded lectures were both raised during the talk; both of course have clear accessibility benefits and implications.


Overall, I thought it was a terrific event! As someone who currently is involved in e-assessment from a slightly unusual perspective (i.e. I’m not a full-time educator and have no formal qualification in education), I learned a lot…and discovered I have a lot more to learn. It’s clear that emergent technology and technology adoption, and a world of learners who are increasingly Web-enabled digital content producers and not just consumers, is influencing the theory and practice of assessment. Hopefully this (r)evolution leads to the destruction of as many barriers to inclusion as possible.

Roll on next year’s e-Assessment Scotland!

This entry was posted in Basic, Review and tagged , . Bookmark the permalink.

Leave a Reply

Your email address will not be published. Required fields are marked *