From the very late 1990’s up until the time I retired, I worked in an office. I chose a different career path that involved sitting in front of a computer doing things with other computers and servers for the British Healthcare system: the NHS.
This post relates the process of implementing an NHS IT Department Risk Register. Lots of other places, organisations and businesses have Risk Registers, this post is about ours – the one for the IT Department.
The old crap
Clinical investment
I’ve commented a few times throughout this site that in terms of IT equipment, we used to run some reasonably ancient hardware for the most part. The reasons for this were manyfold: this being the cash-strapped NHS, any money that did become available usually got spent in areas of clinical importance, rather than on IT hardware. That in itself is OK (after all, hospitals are there to treat people), but there will come a point at which technology is going to make the clinician’s life easier, by either performing a task or getting better results a lot quicker than they did with the old equipment.
This meant that on the odd occasion there was indeed some investment in IT equipment for clinical areas. It happened, but it didn’t happen often.
Background services
What most people don’t realise however, is that in order for the doctors, the nurses and the admin staff to be able to access and use the clinical equipment, read test results and communicate with patients that need treatment, there are various non-clinical IT-orientated services that work away in the background.
For example, most doctors, wards and admin staff will have access to a hospital PC. They will have a logon ID to be able to access the PC, where they can use different (and varied) programmes loaded onto the PC to perform their specific tasks. From clinical applications, email and printing to ordering systems, 3D design software and safe, filtered internet access, there are many background systems that run on servers and (nowadays) are supplied by third parties, accessed through remote networking.
Speaking of remote networking, there’s also the ability for a clinician or admin staff to work from home: using their business laptop from their kitchen, as if they were at work. This bring with it the ability to participate in a meeting using teleconferencing software (something that became very popular due to us all being locked in our own houses for a year!).
All these services run either on local server hardware (based in the hospital server rooms) or at a third party location. Either way, something will run on a server, somewhere and will require maintenance or IT administration.
Virtualisation
I discussed virtualisation in the post Office Life: 4 – Physical vs Virtual Servers. And whilst virtualisation offers a reasonable amount of redundancy and fault tolerance, it can only be as good as the software that’s running on it.
“Dodgy” Software
OK, maybe a bit unfair to say dodgy, it’s more likely old, rather than dodgy. Let’s look at an example:
Hospital systems are many and varied. Some can be large (e.g. a Radiology X-Ray system), but some can be very small, e.g. a database that logs results from a particular type of blood test, performed by an analyser connected to it, in a Pathology lab. This analyser will have been performing this particular type of blood test for years – just chugging away in a corner of a Pathology lab, logging the results to a database – usually an Access database. When I say an Access database, this will be Access 2003 (or older).
The PC that the Access database runs on may have been kept up to date with security patching (it may even be a currently supported O/S version), however the version of Access will need to be the Office 2003 version it’s always run on. And the database will more than likely have grown over time to become quite large, both in stored results and in size.
- “Why hasn’t it been virtualised” would be the cry from IT Management, every so often (when one of them woke up). “Because the analyser needs to be physically connected” was usually the answer. (Probably the one time centralised virtualisation was actually useless!)
- “Can’t we get an updated version of the programme? One that runs on a modern O/S” was usually the next question. “We can’t, as the software authors went out of business/got taken over/died” was the usual answer.
- “Is there any other product that can do this”? This was the clincher question. And usually the answer was “Yes there is. It’ll cost “X” amount of pounds to buy/rent it, plus “X” amount of pounds per year for maintenance”. Usually “X” amount of pounds was far too expensive and things would fall mysteriously quiet.
And the PC – with its Access 2003 database and ancient analyser – chugged on.
Enter the Risk Register
Senior IT Management decided that it would be best if we had our own Risk Register. They’d seen the main hospital risk register and decided it would benefit “us” in having our own. Apparently, “we” didn’t put some of the identified risks (like the Access 2003 PC) on the main hospitals Risk Register, so we would have our own and transfer the ones that were deemed “important” to the main hospital Register.
In the meantime, we were told to identify some risks, make a note of them and then we would be trained how to use the hospital Risk Register software to log them.
“You what?” we asked.
The hospital had procured a software package that managed risks. We were actually acutely aware of it, as we’d had a bad time installing the main server and the client software. It had its own contained database system and its own authentication. There was no integration with any other system, bar the email system that would be used to send notifications from the risk management software. That in itself was quite a fight, as the risk management software was blissfully unaware of any security issues arising from freely sending emails to all and sundry. (There was plenty of eyes rolling on the day that little nugget came to light.)
You logged details into it and it would automatically flag them up at the required time to the salient people. This was great… but it didn’t work very well. Sometimes not at all. The software was clunky (the interfaces had the look and feel of the old Windows 3.11 O/S), it was very complicated and had so many fields to fill in, it was almost impossible to use. It wasn’t intuitive in any way, shape or form and was quite slow in places.
Full of optimism
However, we sucked all that up and the relevant team leaders began to log risks into the system. It was quite an exciting time, as we thought that finally, we can make people other than our managers aware of some of the big risks we carried as an IT department, in terms of old hardware, faulty software, bad installations, out of date security updates… the list went on and on. And on!! We (the team leaders) thought “this is great! The hospital management will look at this, recognise that investment was lacking and money will flood our way to correct all of these risks”.
The next team leader meeting
Once a month we would have a meeting, where the all the IT team leaders gathered with the IT Department management to discuss things. Things were items for discussion placed on an agenda. Items on the agenda were put on there by the IT management. Very rarely did the team leaders get a look in, however there was always an “any other business” section at the end that we could raise any issues.
The very first team leader meeting after the register had been introduced had (at the top of the agenda) “Risk Register Review”. The head of IT opened the risk register on his laptop and we all had a look in the meeting at the risks we’d placed on there.
That meeting lasted seven hours and we only covered that one subject.
A “fine tuning” happened
Shocked (and very probably dismayed) at the sheer quantity of risks the team leaders had entered on the risk system, a good deal of meetings happened! Team leaders met with their relevant managers, who directed the leaders to remove a portion of the risks from the register.
I objected to this quite a bit and said that we would remove them when they were fixed. Sadly, I got overruled and some risks that were deemed as “minor” were removed.
We were also given “guidance” on what risks we were supposed to log on the register. Any risks that we did identify, we would have to submit them to our management for approval, prior to entering them on the register.
At that point, there was no point
I spent a good deal of time arguing against leaving risks out of the register. The whole point of the register was to highlight areas (risks) where something wasn’t happening, or something needed replacement – which in turn might require the clinical department investing some funds to fix it. If we weren’t going to log that risk, the clinical department may be aware of it, but the main hospital management wouldn’t. If that was the case, funding to fix the risk would not be forthcoming and the risk would remain active.
The IT management response
Most of the IT management responses consisted of “they know the risk is there, it’s up to them to log it in their own register”. We knew that would never happen because it required money to fix things.
What about the risks we were allowed to log?
The analyser example above was a prime example of this. The risk was logged and was raised at a hospital management meeting. Part of the risk record was to list the actions required to mitigate the risk and an approximate cost. The risk was put on hold due to lack of funding (so therefore wouldn’t be fixed).
When this was reported back, I expressed my dismay about the rejection in no uncertain terms. The fix was rejected on cost alone, but what about the security implications? Running old unpatched equipment on the hospital network was risk enough, surely, to warrant an upgrade?
And I was told (in no uncertain terms) that “we (the IT department) had raised the risk (with hospital management) and therefore it was up to them whether they accepted the risk or not. It was now to be classified as none of our business”.
My answer to that was this: “how much of our business is it going to be if somebody puts a virus on that unpatched PC (with no anti-virus)? It’ll cause havoc if it spreads and we would have a major incident on our hands, with a possibility of critical systems being compromised”.
“Well, it wouldn’t be our fault, as we told them”. Came the response.
No point
That point was the point of no point. There was no thought about the amount of work we would have to do in the event of a major incident, let alone the impact and disruption to the hospital in both staff and patients. It was all about finger-pointing and blame association.
At that point, dear reader, I decided to make plans to be elsewhere.