In July 2012 an online survey was conducted of all Foundation ePortfolio users at the request of the UKFPO. The survey ran for three weeks and generated 2,425 responses. Although the results were available immediately, the survey coincided with ePortfolio Review and as the information was commercially sensitive the results are only being made widely available now. This paper summarises the most common comments of respondents, but is not a systematic evaluation of the data.
Down the ePortfolio Survey Summary (PDF format)
During the summer the NHS ePortfolio experiences a significant increase in the level of usage and activity in the lead up and during the ARCP process for Foundation and Specialty trainees.
As shown in the Figure 1 below, the number of page loads due to extensive user activity peaks towards the end of May with 3.1 million page views across the entire ePortfolio. Over half of these are specifically Foundation ePortfolio users.
Of interest is the average page load times that peaks in mid-June. This is particularly marked in Foundation-specific data and can be accounted for by the change in the type of pages being accessed during June compared to May. During ARCP reviews the server will be generating a higher proportion of complex report pages that require the server to have to process more data.
Figure 1. Page views (blue lines) and average page load time (red line) for all users (a) and just Foundation Users (b)
The performance of the server (Figure 2) showed spikes in the data transfer rate (blue lines) and measured overall server response time (elapsed time, red lines) with a peak coinciding with the date of highest usage (22nd May). Greater daily spikes are seen seen during June while transfer rates flatten from the end of May onwards. The occasional transfer rate troughs coincide with scheduled downtime for software updates. Users who may have experienced delays in opening pages or submitting forms during May and June may have been subject to network issues related to local IT systems or infrastructure that we can’t detect or influence.
Figure 2. Server performance during May and June 2013: data transfer rate (blue lines) and overall server response time (red lines)
Overall, there was no reported or detected performance problems with the ePortfolio that could be attributed to high levels of general activity (as seen prior to ARCP) or by high levels of complex processing activity (as seen during ARCPs). This can for the most part be attributed to recently implemented server architecture that can be “powered up” during peak times but also to ongoing efforts by the development team to identify and remove software and data “bottle necks”.
In conjunction with improvements to our search functionality for Foundation Administrators (see below) we have also improved the ePortfolio so it is easier for a Foundation Supervisor, Foundation Programme Director, Dental Trainer or Dental Advisor to review multiple trainee accounts.
When a trainee account is viewed via ‘My trainees’ (if you are attached via a post/block) or ‘Programme Trainees’ (if you are attached via a programme/scheme), you can now select a trainee’s page and view it before clicking the new ‘Switch selected user’ button found on the left of the screen.
If you select a different trainee you can click directly on the name to switch them, or (where you have a long list) you can start to type their name into the text box to get to them more quickly. You will be taken to the same ePortfolio page, but for that next trainee.
If you ever need to refresh the trainees listed just return to your home page and start with a new trainee selection.
The functionality for searching and selecting trainees has been significantly improved in the most recent update (release 8.8), though currently only available to Foundation and Dental Administrators.
Previous to this update, when an administrator filtered for a set of trainees using the “Search Trainees” page, after selecting a particular trainee they then wished to choose a different trainee from the same list they needed to repeat the search process.
The new functionality will allow the list of trainees to persist so that it is possible to switch between them without having to repeat the search process.In fact this facility also works when using “Administer Posts” and “Administer Users” (so not just trainees)
The steps are shown below:
1. Search for the trainees as normal
2. The search returns a list of trainees
3. Select a trainee to work on. Note the new button on the top left “Switch Selected User”
4. Clicking “Switch Selected User” produces a pop up box where up to the first 50 trainees from the search list are shown. You can click directly on the name to switch them, or (where you have a long list) you can start to type their name into the text box to get to them more quickly.
ePortfolio’s growth since 2005 has been a collaborative effort between a wide range of organisations and professions. Ongoing feedback has suggested that many users would appreciate clarification on where the responsibilities lie for ePortfolio.
Broadly, all content and how and when it is presented to you is determined by your Organisation (e.g. Medical School, UKFPO, Royal College, Professional Group, Postgraduate Deanery). Some organisations collaborate formally to agree what is available, so that there is a standardised approach across distributed groups or geographies e.g. Postgraduate Deaneries, Foundation Schools and Medical Schools. NHS ePortfolio provides the ePortfolio software i.e. the code that underpins the website, and the secure storage of the data.
What is your Organisation responsible for?
|Content and format of all assessments
|DOPS, miniCEX, CbD, Supervised Learning Events, Core Procedures, MSF TAB|
|Reflective practice||Reflective notes
Categories for reflection
Headings in reflection forms
|Curriculum||Length, content & structure of the Curriculum
Whether you are required to link certain items
|Timelines & frequencies||How many assessments are required during each placement
Any minimum requirements eg every SLE/assessment must be linked to n curriculum items
|Supervision & progress||Who can complete supervisor reports
Content & structure of supervisor reports
ARCP process & documentation
|Permissions||Which ePortfolio roles have read & write access to your data within the ePortfolio, and for how long|
|Declarations & agreements||Health & probity declarations; educational agreements|
|Courses & alerts||Adverts and alerts for events or deadlines|
|Data entry||Adding and updating user information|
|Data exchange||Links to and from any other systems eg College database, exams, e-Learning for Healthcare|
|Non-technical user support||Password reset, form editing/moving, information|
|Change requests||Timing, frequency and funding of changes to content|
|User acceptance testing||Final approval of any changes or modifications requested by the organisation|
|Engagement with users||To determine the requirements and priorities|
|Training and user guides
|Face to face, written, video and/or audio guidance and/or training on how to use the ePortfolio system.|
|Communication||Dissemination of information about content and processes (including changes) to trainees, supervisors and administrators|
What is NHS ePortfolio responsible for?
|QA and technical testing against requirements specified by your organisation|
|Deployment (scheduling & uploading of code from test to live environment)|
|Monitoring the infrastructure (eg environmental monitoring, network monitoring, load balancing monitoring, web server and data base monitoring, firewall monitoring and intrusion detection) to detect and correct abnormalities.|
|Database backup & security|
|Technical support including resolution of bugs|
|Providing data extracts and reporting from the database as requested by the relevant organisation|
|Layout of material displayed in the main content area (fonts, graphics, pictures implemented following advice and guidance of organisations)|
|Engagement with Organisations and users to encourage dialogue and collaboration with the development of the software|
What do you do if you have ideas, suggestions or other feedback about any of the above?
If you have feedback about any of the items in the first table above, please contact your organisation. If you are unsure which organisation to contact, your organisation’s logo should be visible on any page of your ePortfolio.
If you have a technical support issue, contact firstname.lastname@example.org
You can follow us on twitter @NESePortfolio