One thing that has struck me as I have been browsing blogs, reviewing links on twitter, and talking to colleagues across different organisations, is that there is a lot of negativity around national clinical audits. Some of this is well founded – some do take too long to report, some are just continuous data collection, some don’t have robust standards. However, there are some criticisms, which I find very hard to accept – the biggest one being that national audits don’t do enough to support local change.
Before you shout at your tablet/computer/phone, let me clarify… I accept that national audits don’t always provide a neat toolkit of change that we as audit facilitators should go away and implement (but some do – for example the tools that are provided by the Prescribing Observatory for Mental Health). National audit don’t always provide an opportunity for staff to meet, discuss and reflect on findings at a national or bench-marked level (but some do – for example the IBD National audit).
The problem comes with a lack of consistency over national audits and what we as local audit staff can expect from them. This can only be rectified by outstanding commissioning processes, an ability to effectively scrutinise these (audit if you like), and change the practice of the national audit providers – in an enforced manner if needs be. Call me naive, but if you called a decorator into your home and asked them to paint the room blue, you wouldn’t accept it and pay if they painted it purple ‘because it’s almost the same colour’. You would expect blue. A transparent and consistent expectation of what the national audits should deliver, and holding them to account for this, is the only way that local audit staff will receive a consistent offer to support them to effect change.
Which takes me back to my first point – some of the negativity is well founded, but actually some of it isn’t! Why should we as local staff be waiting for national audits to provide us with the tools to change practice? Why shouldn’t we be facilitating change ourselves? The recent blog post from CASC re: their “Clinical Audit Summit for Mental Health and Community professionals” noted in the SWOT analysis that “results may not be as high impact as local audit” and that national audit “takes the focus away from local audit”. My question back would be – so what have you (staff working in audit, not CASC!) done about it? We each have a responsibility to drive change – if the results aren’t high impact as local audit, why is that? Is that because they haven’t been given enough support from facilitators? Did they have good clinical engagement from the outset? Was the right clinical lead in place? Did we use improvement tools once we had the findings – such as root cause analysis, five whys, PDSAs? Have we asked those questions (and more) of ourselves and our teams. To paraphrase the words of Ghandi we have to be the change we want to see…
Audits (be they local or national) will never be valued if they don’t deliver improvement. And they won’t deliver improvement if they don’t get to the bottom of what isn’t right about current practice and implement a change to improve this. The national audits have come a long way in the last 5 years, but they still have a way to go. They aren’t going to go away though, so it is up to each of us to work together – through organisations like NQICAN and regional networks, alongside HQIP to commission methodologically sound, consistent projects which we can then act upon the findings of, and deliver the end result of improving care for patients.
Footnote from Editor – Carl Walker, NQICAN Chair
The author of this guest blog has requested to remain anonymous for reasons that I respect. Please let us know know if you would like to write a blog on a topic that you are passionate about or what to generate discussion on – anonymously or not.
Email: nqican@gmail.com Twitter @nqican
Stellar, balanced blog. Well done to the mysterious author and NQICAN for starting an acutely needed debate on NCAs.
The author states ‘national audits have come a long way in the last five years’. If a hospital CEO said that we’d verify via many metrics: CQC reports, national audit data, FFT feedback, etc. Given we work in the ‘improvement business’ where is the hard evidence that NCAs are improving? Certainly the recent HSJ article citing extensive work by YEARN and CASC suggested NCAs are treading water while others drown under the increasing burden. The previous blog highlights NCAs have not diversified and cons outweigh pros in many minds.
In answer to the blogger’s key question ‘whose responsibility is it to improve national clinical audit?’. One answer: the Healthcare Quality Improvement Partnership.
HQIP hold the contract to deliver the NCAPOP and are funded by NHS England. HQIP have sole responsibility for procurement of NCAs and in most cases award contracts to their Academy of Royal Medical College associates. HQIP manage national audits and control the end product we local audit staff implement. Unless mistaken, local audit staff have zero input or say in this mystic process. It is a nice idea for the author to suggest we all work together but where has the appetite for partnership and collaboration been from those at the top?
The author correctly asserts ‘the problem comes from a lack of consistency over national audits’. Who is responsible? I agree we should have a transparent and consistent expectation of what national audits should deliver’. Why is this not happening in 2017? We auditors expect clinicians to standardise their care but the NCAPOP isn’t just blue and purple, it features all the colours of the rainbow.
Further to this excellent comment, I would point out that is not NOR HAS EVER BEEN the responsibility of clinical audit staff to ensure change as the result of either local or national clinical audit. It is the duty of clinical staff, supported where necessary by trust management to act upon the results of audit and the responsibility of commissioners either nationally or locally to be assured that this done. Simples!
I would agree with this reply, HQIP have the money, the commissioning authority and the clout, it is them and the royal colleges who run most of these audits who need to improve the quality and the impact of many of the national audits so they match the best. Pushing this on to local staff and overrun clinical leads who have little resource and time is not sufficient. Many national audits are poor, plain and simple, they are too long, too onerous and the change part od the cycle is given scant attention or resource. They do not respond to calls for improvement and ignore the work of local colleagues and notable commentators like Nancy Dixon, frankly most of them would fail the tests we set local audit. Some of the patient and staff questionnaires in particular are very poor and I believe border on unethical
To the mysterious author: what data and metrics are you using to conclude that NCAs are improving? Thank you
Thanks for your query – we have contacted the author and they have stated that they are using on the job evidence of working with NCAs over the last 10 years to make this conclusion.
I’d like to ask why NQICAN speak on behalf of the blog author? What started as a promising debate on NCAs has descended into a farce. The unnamed blogger’s ‘ask not what national audit can do for you, but what you can do for national audit’ stance was refreshing. There is nothing wrong with the author directly challenging us local audit staff and asking for our ideas and feedback. Yet when the author hides behind NQICAN and gives a cursory reply to a simple question asking for more details then they stop playing by their own rules. The author exhorts us to learn from Ghandi and ‘be the change you want to see in the world”. Would Ghandi have asked a third party to speak on his behalf or would at the least he write a detailed reply using (as I do) the anonymity option?
The more I read this blog the more I come to the conclusion that this is written by a representative of a national audit. The approach certainly stands up: provide lots of information but then refuse to reply to anything other than a positive response. I prefer a different Ghandi quote ‘To believe in something and not live it, is dishonest’.
Thanks for your comments – I can assure you as NQICAN chair and editor of this blog that the author is an experienced clinical audit professional. I will ask the author to provide a more detailed response asap.
Firstly, let me apologise for the lack of a full and considered response to the previous blog posts. Like most of you who are reading this, I have to put ‘the day job’ first, and huge recent pressures on my team have meant that I have not had time to look at this much since my initial post, and I didn’t want to respond with what may be considered a flippant or ill-considered response. I asked Carl to note a quick response, so it didn’t seem like I was completely ignoring all responses to the blog.
I have worked in local audit for over 15 years, mainly in a mental health Trust, but more recently in community services. I’ve not worked in acute personally, but have had close working relationships through my regional network, and liaising with colleagues in my area to know a little about the acute national audits (and just how much work they take!). That’s where I am basing my experiences from. I’ve never worked for a national audit team (nor do I plan to – my heart is in local Trusts). I’ve seen the national programme come from a few standalone audits, to the huge programme of work (both in terms of numbers of projects, cost, and levels of staff involved) that are today’s national audits. I was careful in my blog to state not that National audits have improved, but that “national audits have come a long way”. My personal perception, is they have changed a huge amount in the last 10 years – whether change is an improvement is a whole PhD’s worth of subject matter… Via the commissioning process of HQIP, national audits involve NQICAN in the specification development, and gain local audit staff feedback via regional networks, about what is working and what isn’t. The idea of that 10 years ago was real ‘blue sky’. Some of them do presentation days where they share their national reports, and work to develop resources to support clinical staff to change process, they’ve worked to cut their timescales to reporting from data collection – based again on feedback that we local staff have given. I’m by no shape saying they’ve got it made, but they have made significant changes, and I think we need to recognise that.
I do think the national audits need to continue to change – and I think that is best supported by effectively commissioning those audits – for example, if HQIP put in their specification that national audits needed to issue their reports within (say) six months of data collection, and attached a financial penalty if that didn’t happen, then I’m pretty certain most national audits would find a way to make that happen. As a previous responder said, HQIP have this sole responsibility, and I personally am keen to see them continue to improve from this commissioning side.
The reason I wrote this blog piece though, was because of my own frustrations of us as a body of local audit staff. I understand we need to feedback where the national audit programme needs to improve, but we have to take responsibility for supporting this ourselves. An earlier poster said that “it is not…the responsibility of clinical audit staff to ensure change as the result of either local or national clinical audit.” Correct – but in my opinion, it is our responsibility to support and facilitate clinical staff to be able to identify where change is needed, what changes can be made, and support them to work with others as needed to make those changes. I have been lucky enough to attend various events (training, conferences, workshops, etc.) over the last couple of years (some as a delegates, and some as a speaker), and the think that really struck me, was how much we like to moan (and I include myself in that!). We moan if an event starts to early we had to be up at the crack of dawn, we moan about our teams we work with, we moan about the weather (too hot/too cold), and we moan about how we are doing a job that is just about collecting data, and doesn’t make a difference to patients! My challenge was to make readers think – just a little – about if they personally could be doing anything differently, in their own practice, to help make things better. Do you always use all the tools available to you? Do you always involve the right stakeholders in a project? Do you always think about what else is happening in your Trust that might tie in with your action plans (classic example in my Trust recently where we had completed a POMH audit on valproate – did a great action plan (or so we thought), involving the right people (or so we thought), then realised there was a whole other team working on a parallel plan related to a CAS alert on the same topic, and none of us had joined those dots)? My point was, we can’t just moan about it not being right ‘from the centre’ if we aren’t thinking about using all we have in our own toolkit of effectiveness. It will not only help us to manage the increasing workloads we all seem to have, but will also support clinical staff to improve outcomes for patients, which is why we do this job.
I hope that this response adds to the initial post, and the comments that have come so far. Again, apologies I didn’t have time to add this earlier…
Regards, The Author.
Although the author has asked for feedback from the audit community and ‘not CASC!’ we just wanted to add briefly our perspective. A couple of points:
We agree with the author. Certainly, those working in local clinical audit could be more positive with regard to NCAs. Over the years we have attended many regional meetings where the main theme seems to have been ‘lets bash the latest NCA’. YEARN have demonstrated what can be achieved if local audit staff unite rather than moan and their recent review of NCAs is an excellent piece of work all other networks should learn from and try to emulate.
The author is also right that at a local level there should be more integration and sharing especially if Trusts are largely working on the same NCAs. Given where technology stands in 2017 it would be easy and cheap to create a simple portal whereby Trusts could upload questions on NCAs for all to see and action plans of how NCAs have changed care for the better. NCA providers could share replies and stimulate collaboration. Win-win.
Our main observation is that local and national audit seems divided into two distinct camps. At one end there are the Royal Colleges and national bodies who create and run the NCAs and at the other are the clinicians and audit staff that implement them. There isn’t much middle ground and our view is that this doesn’t particularly help. We suspect most audit staff probably feel disengaged from NCAs other than their role in delivering/co-ordinating the work when NCAs ‘land’.
And when NCAs do ‘land’ untried and un-tested and full of basic mistakes in the data collection form that a clinical auditor in a Trust would have picked up on (e.g. the 16 page End of Life Care Audit), tensions inevitably increase. For us the solution we have always advocated has been that NCAs need to get more local audit staff who have the expertise in delivery and implementation of NCAs involved in their design and planning. That would solve a lot of problems. Ditto an open and a facilitated meeting of 50 staff working in national audit with 50 who work in local level. We believe, honest and open discussion from both camps could move NCAs years forward in one day.
Funnily enough, we looked back recently at a national debate from 2007 where we were asked to suggest how NCAs could be improved. Our main suggestion was that NCAs were over-complicated and could be made much more sophisticated. To achieve this we recommended much shorter data collection forms pinpointing key information and not trawling for data. Our argument was that this approach would reduce the burden on clinicians and result in more rapid reporting (less data = faster feedback). Reduced workload would also allow clinicians more time to see patients and cut the cost of NCAs. Reinvestment of saved monies could be used to diversify the NCA portfolio, for example following Europe’s approach to auditing care homes.
Whether we are right or wrong we would like to thank NQICAN for publishing the blog and congratulate the author on not shying away from a contentious topic area. One thing is for certain, for NCAs to improve we need more debate like this.
[…] (NCA) to drive improvement which follows on nicely from a couple of our previous blogs – whose responsibility & making QI core […]