Monthly Archives: June 2013
When is the best time to perform site updates and do maintenance that requires the ePortfolio to go offline for up to an hour? The graph below shows the activity of the ePortfolio during a normal 24 hour week day. As you can see the quietest period is between 2 and 4am. So why not do this work then?
The problem is that that we haven’t sufficient resources such that we can have highly qualified development staff working a night-shift. Similar issues surround working over weekends. The site may be offline for 30 to 60 minutes but the technical team involved in the work must prepare code and database scripts, then take the site offline to implement the updates, test the updates, and then monitor and run checks immediately after the site has been brought back online.
Another reason to perform updates in the evening is that if we encounter a significant technical issue with an update we have several more hours to bring about a fix before the majority of users come online at 8am.
There is no ideal time to take the site offline but we have to work within the limits of resources and also ensure that we can minimise the risk of downtime affecting normal working hours.
We endeavour to provide as much warning of downtime as we can, and hope to be consistent in when these scheduled updates occur and so allow users to work around our deployment times.
The ePortfolio will be updated on Tuesday 25th June at 10pm, to be offline for about 60 minutes while we deploy updates and perform routine maintenance. This is release 10.5, the highlights of which are detailed below:
Update Items included in this build are:
• 7569 – PHARMACY – VT2 Care Plan
• 8339 – CEM – STR revalidation update
Fixes included in this build are:
• 8375 – RCPI – Non-admin trainee search issue
• 8441 – FOUNDATION – ARCP Assessor with correct TAB permissions
• 8686 – RCPI – Admin target Timeline: required field error
• 8736 – DENTAL – Add LePID to analyse forms return from LEPs
• 8772 – ALL – ReturnURL paramaters getting dropped when redirecting after a form CRUD
• 8781 – RCPI PCS – Dynamic Form Builder Admin – Default Activity Date Question not required
• 8887 – ALL – Ensure mime types for the main Office XML file formats work
• 8932 – FOUNDATION – Update PT/FT status on existing posts
As part of the pages to support the Foundation Annual review of Competence Progression (ARCP) process a number of new features have recently been released onto the ePortfolio. The “Summary of evidence for satisfactory completion” acts as a central portal of ‘quick links’ to relevant evidence in accordance with the core requirements for satisfactory completion of F1 and F2 (Foundation Programme Reference Guide 2012).
This document will detail how this page can be used to support the ARCP review process. This example is form the point of view of a foundation administrator viewing a trainee’s ePortfolio, but the pages work the same if viewed by supervisors, trainees or a programme director.
How to use the summary page
1. Log in or select your trainee as usual. You should see the trainee home page.
2. Use the menus to navigate to the “Trainee home > Forms > progression ARCP” page
3. Click the “View summary of evidence for satisfactory completion of F1/F2” link. This can be seen in a box marked “Optional resources to review your e-portfolio evidence.
4. Select “F1” or “F2” to show evidence from either the Foundation trainee’s FY1 year or FY2 year. The summary page will pull through all data for that training grade.
5. You will see a list of all the sign off requirements for satisfactory completion of the year. Some of these requirements pertain to evidence that is not held on the ePortfolio, for example the requirement that a trainee Completes 12 months of F1 training without exceeding the maximum allowable leave period. In this case the summary page cannot provide any evidence on this requirement, so the ‘View evidence’ column will remain blank.
6. However if the requirement does pertain to evidence held in the ePortfolio it will be pulled though and summarised in this page. For example the requirement to have A satisfactory educational supervisor’s end of year report will pull through that report if one is present for the training year. The page even summarises if on that report the person filling it rated the trainee as ‘Met requirements’ or ‘Did not meet requirements’ to make it quicker to review the evidence.
7. There is also an embedded link to the form in the summary page, so clicking on the ‘magnifying glass icon’ will cause the form to open in a pop-up page so the details can be reviewed immediately.
The ePortfolio will be updated on Tuesday 18th June at 10pm, to be offline for about 60 minutes while we deploy updates and perform routine maintenance. This is release 10.4, the highlights of which are detailed below:
Update Items included in this build are:
• 3832 – DMS – Basic site build – Phase 1
• 6972 – UNDERGRAD – MiniPAT
• 7713 – PHYSICIAN – Add spreadsheet export function to exam result report
• 8002 – ALL – Convert Warehouse SPROC – uspReport_TrainingLogbook
• 8224 – PHARMACY – PRPS Weekly Review Meeting Record Form to be amended
• 8382 – FOUNDATION – ARCP 2012: revisions to ‘All current trainee’ report
• 8554 – DMS – Logo to be added
Fixes included in this build are:
• 7897 – FOUNDATION – Trainees have the wrong Courses & Seminars menu link
• 8463 – CEM – Reflective Notes Summary Page Issue
• 8480 – ALL – Investigate Slow Pages
• 8533 – FOUNDATION – Moving filled TABs between posts also moves all TAB tickets
• 8606 – RCPI – Hypens in posts table list
• 8723 – DENTAL – Prescreening form – directors comments not visible to admins
• 8729 – DENTAL – Trainee has permission to create ‘certificate of satisfactory completion’
• 8731 – DENTAL – Run script to update filled ToK and MME certs with directors names
• 8738 – DENTAL – Certificates – Pdf has large section of whitespace at the foot of the page
• 8787 – ALL – Can’t edit usertrainingprogramme details for UTPs with no curricula attached
• 8816 – FOUNDATION – Broken url in probity declaration form
• 8869 – ALL – Change to Permission Caching
• 8308 – FSRH – Update to multiple Forms
The article below was presented in June 2013 as a poster presentation at the 17th Annual International Association of Medical Science Educators (IAMSE) at St Andrews:
The Foundation Programme is a two-year generic training programme which forms the bridge between medical school and specialist/general practice. In August 2012 a change in the approach to trainee assessment was introduced in the form of Supervised Learning Events (SLEs) for 3 key tools, the Case-based Discussion (CbD), Mini- Clinical Evaluation Exercise (MiniCEX), and Direct Observation of Procedure (DOPs). These tools directly replaced the equivalent work-based assessment (WBA) forms that were used in previous years and included Likert scales. SLEs are not individually summative; the intention has been to encourage trainees to use these tools to support ongoing learning, reflection and discussion with assessors and supervisors.
The NHS ePortfolio is a web-based tool used by 15,000 (93%) UK trainees as part of the Foundation Programme that follows graduation from medical school. The purpose of this study has been to compare the level of trainee engagement with the new SLE tools (session 2012 – 2013) with the equivalent assessment tools during the previous year (session 2011 – 2012).
To measure engagement levels we have analysed aspects of behaviour relating to the submission of these forms to their ePortfolio account during a fixed, equivalent period of time, including the time-dependent pattern of form submission. We have compared the number of form submissions made by Foundation Year 1 (FY1) trainees and Foundation Year 2 (FY2) trainees. Data was generated from the NHS ePortfolio database counting the number of forms submitted each day, starting from day 1 at the beginning of the start of the Foundation session through to 6 weeks after the start of the trainees’ third post in May, 43 weeks after start of the training year. The total number of form submissions and the rate of form submission were compared between WBA and SLEs forms.
For reasons of brevity and to avoid data “noise” we analysed data taken from 3 representative Foundation Deaneries: consisting of 1600, 750, and 900 trainees, respectively.
A total of 119,019 form submissions were analysed. Comparison was made between WBA form types against equivalent SLE forms for the same time periods during the training year. This indicated that no appreciable differences exist between WBA and SLE form submission counts were observed with respect to the number of forms submitted or the rate of submission. Minor variations in number of DOPs versus SLE-DOPS forms were observed though this may be attributed to changes in training requirements and the use of “Signed Procedure” forms.
During 2011 – 2012 SLE versions of MiniCEX, CbD and DOPs forms were not available. On 1st August 2012 the SLE forms were introduced, replacing their WBA equivalents.
The accompanying figures show the total weekly form submissions during the training year, beginning on 1st August 2011, and with form submission during the training year beginning on 1st August 2012.
DISCUSSION AND CONCLUSIONS
The results presented here have shown the pattern of creation of SLE forms does not significantly differ from the worked based assessments (WBA) forms that they replaced. The total number created would largely be governed by the curricula requirements of the Foundation Programme; in the sample deaneries used in this study 69,000 WBA forms were completed compared to 50,000 SLE forms.
Anecdotal criticism of WBAs has been that assessment (and associated scoring) of procedures may not best support a learning process throughout training as trainees might wait until nearer the end of placements to be assessed thereby gaining better scores. It has previously been observed that the end of each placement is characterised by a significant rush by trainees to submit the required number of WBA forms, commonly within days of the placement end date. With the introduction of Supervised Learning Event (SLE) based forms it might be expected that these tools would result in more measured and timely submission of SLE forms during a trainee’s placement. However, no significant shift in trainee behaviour was observed when examining the number of forms. Indeed, the pattern of SLE form submission was remarkably similar to WBAs.
Of interest was the fact that having previous experience of WBAs did not result in FY2 trainee behaviour when switched to SLEs, as compared to FY1 who were entirely new to SLEs.
To conclude, the lack of any behavioural change in form filling patterns might be attributed to either (i) the lack of training/understanding by trainees (and supervisors) of how to use the SLE tools, or (ii) the nature of form-filling during training in a work environment is be its nature going to result in trainees leaving completion of WBA or SLE forms until late into a particular placement.
Following discussions during poster sessions the issue of the trainees being able to get SLE forms completed by assessors is problematic – the process of trainees having to request assessment/SLE completion using a ticketing system can result the “moment has passed”. The development of a mobile app that allows for offline completion of tickets and forms may have an impact on this so repeating this data analysis in 2014 may be telling
This is available as a Prezi slide presentation at the link
and is embedded below:
NES would like to thank the Academy of Medical Royal Colleges, the UK Foundation Programme Office and the representatives of the Postgraduate Deaneries for their continued support