Design

Feedback from ePortfolio users

Following our year end user group meetings last month (announced here on 19th October) we have collated our feedback into this document.   We have circulated this to out version leads for future discussion.  We will be looking to act on this feedback in the near future in conjunction with the commissioners of the various ePortfolio versions.   We will post updates on this process on this blog in the future

Download the YEUG2012 collation document (PDF format)

UKFPO Request for Recommendations to Improve the Foundation e-portfolio

Background

  1. There are two e-portfolio providers for foundation doctors in the UK: NHS Education Scotland (NES) and Horus.
  2. The Foundation Programme Curriculum Delivery Group (UKFPO) is responsible for setting the core content, functionality and security for both e-portfolios.
  3. Working with stakeholders, the e-portfolio is performance reviewed every year. Recommendations to improve the e-portfolio are subsequently made, with changes being deployed on an annual basis (i.e. July/August).

National core content – have your say!

  1. To ensure that e-portfolio core content recommendations for August 2013 are completely user focused, the Curriculum Delivery group are seeking your input.
  2. If you have any improvements relevant to the national core content and functionality of the e-portfolio, now is the time to suggest these.
  3. 6.    Please complete the recommendations form and return to Stacey.forde@foundationprogramme.nhs.uk by 31st December 2012.
  4. All core content recommendations will be considered by the FP Curriculum Delivery group in January 2013. The approved changes will be embedded into the July/August 2013 e-portfolio.
  5. Please note that if recommendations are not relevant to national core content (e.g. no changes to actual form content etc.) these cannot be taken forward or managed by the Curriculum Delivery Group. Please see the paragraphs below outlining how these should be managed.

 

Mechanisms to improve the e-portfolio throughout the year

  1. If your recommendations are not relevant to national core content and/or functionality, there are other means by which you can request changes throughout the year.
  2. The NES ePortfolio has a Team Advisory Group (TAG). This group meets twice a year and should include representation from every UK foundation school.
  3. TAG considers all suggested improvements to the NES ePortfolio which do not impact the national core content or require FP Curriculum Delivery group input.
  4. The next TAG meeting is scheduled for February 2013 and will be held in Northern England (exact date and venue TBC). Participation with this group is strongly encouraged by the UKPO.

[Download as a Word document (including recommendation form)]

ePortfolio Analytics: what to do with it all that data?

Recently I was at a conference (AMEE 2012) in Lyon on the subject of education in medicine and the healthcare professions where I presented an analysis of web usage data derived from Google Analytics and from a snap-shot (96 hours) of internal tracking data (i.e. data collected within our system).

Google data is vast but difficult to unpick the differing activities and page flow of the various different types of users that use ePortfolio. Internal tracking data is also big (which was why it was necessary/practical to use a snap shot of data) but allows for more granular analysis of the type of users and be able to map what they do during the first few steps into their ePortfolio account. The presentation can be found at this link or at the end of this piece:

http://prezi.com/pryuoscrnbyl/amee-2012/

As a web designer my first instinct was to try to understand what users most want to do, or rather, what they actually do irrespective of how the site is laid out and what their training programme might want them to do.

The conclusion drawn from the data was perhaps predictably that trainees mainly log in to check up assessment form completions and request new assessment forms (56% of the trainee log ins went to directly to this activity). So to me as a designer it would seem wise to improve access to these areas for users so making their life a little easier. A simple starter for ten there. A broader interpretation might be that e-portfolios (especially professional e-portfolios) are inclined to be box ticking facilities: important to a professional body such as a Royal College who need to know that their trainees are safe to practice medicine, but less satisfactory to trainees who want something less prescriptive [no pun intended]. Exactly what trainees want is something that we are starting to get a better idea of more recently thanks to the wonders of social media. There is a balance between college needs and trainee needs in the design of an e-portfolio and possibly this type of data can help inform this debate.

There were of course deeper issues relating to the underlying process of learning and development as doctors progress through training. Differences were seen between trainee activity as they become more senior (from graduate Foundation years through to specialty training) that related engagement with curricula and log forms that were not necessarily reflected in their respective training programmes.

So to summarise, while I’m happy to ask simple questions about user flow, user experience and so on, questions about the educational implications of this data have not been well addressed as it’s not really in my sphere of knowledge (or in my job description). I would see the research I presented at this conference as a “this is the kind of thing we can do” exercise that should lead on to better designed questions that will allow us to understand how best to develop an e-portfolio that supports effective learning and development through the effective delivery of a training programme.