Profs doubt that disruption and anxiety caused by closing emergency departments is worthwhile.

Last year Dorset Clinical Commissioning Group’s Chief Officer welcomed a Sheffield University analysis of the effects of closing 5 emergency departments in England between 2009 and 2011 [https://doi.org/10.3310/hsdr06270], on the grounds that:

“this research…independently supports the national research that giving people care from specialist sites, which may mean further to travel for some people, will not increase death rates…”

Bournemouth Echo report

He added hopefully – in complete disregard of the research findings:

“…and we believe will ultimately save lives.”

On the contrary, the study’s co-author, Professor Jon Nicholl, went so far as to warn:

“It is important to highlight that we didn’t find the better outcomes for patients that planners hoped to see from closing these small departments either.

“This means it isn’t clear that the disruption and anxiety that can be caused by closing emergency departments is worthwhile.”

NHS Detectives have read and commented on the study, which is online here.

What follows is all the key points we understood, as well as some we didn’t – not being statisticians. These are included in case anyone reading this can make sense of the bits we couldn’t get our heads round.

The study’s headline findings

The study’s overall findings were as follows:

  • There is evidence of an increase, on average, in the total number of incidents attended by an ambulance following 999 calls, and those categorised as potentially serious emergency incidents. (This suggestas to me that people understandably get more anxious when they can’t go to their local A&E, so call an ambulance for the longer journey.)
  • There is no statistically reliable evidence of changes in the number of attendances at emergency or urgent care services or emergency hospital admissions.
  • There is no statistically reliable evidence of any change in the number of deaths from a set of emergency conditions following the ED closure in any site, although, on average, there was a small increase in an indicator of the ‘ risk of death’ in the closure areas compared with the control areas.

Other key findings

  • There was a statistically significant reduction in A&E attendances in areas with a big increase in travel time to A&E
  • There was an analysis of out-of-hospital deaths following local A&E closure – but I can’t explain this because I’m not a statistician – any one out there who is and can interpret this analysis – reported below – please help!

Omissions and caveats

As far as I can see the study comes with two important omissions and some big caveats.

The omissions are:

  • The effects of A&E cuts/centralisation on people without cars/on low income
  • The effects of ED closure on neighbouring EDs and hospitals.

The big caveats are:

  • The burden on individual neighbouring hospitals may have increased with unknown consequences.
  • The study’s findings may not transfer to other areas where EDs are being considered for downgrade/closure, because the 5 A&E closures that were the subject of this study had all been run down for some time before the closures.
  • There are huge uncertainties about whether centralising emergency care centres may be achievable without harm.

These findings, omissions and caveats are outlined below.

Statistically significant reduction in A&E attendances in areas with a big increase in travel time to A&E

There is an analysis of the impact of closure in high-dose areas (ie where there is a a large change in time to an ED) compared with low-dose areas (ie areas expected to have a small change in time to an ED).

This showed that although overall there was no reliable evidence of a change in emergency and urgent care attendances following the closure of a local ED, there was a statistically significant reduction of 8.3% in areas with a big change in time to ED.

Figure 13 p 81 is about this measure, I can’t read what it means cos not a statistician but the blurb underneath says

“Over all three sites with usable data, there was no statistically reliable evidence to suggest a change in emergency and urgent care attendances following the closure of a local ED. It was estimated that there was a reduction of –8.3% (95% CI –21.5% to 5.0%; p = 0.22) in emergency and urgent care attendances in high-dose areas compared with low-dose areas following the closure of the local ED.”

p39 also refers to evidence that those living closer to an ED are more likely to use it than those living further away.

Analysis of out of hospital deaths following local A&E closure

The Profs analysed out-of-hospital deaths following closure of local EDs.

But because of the way they define out of hospital deaths, I don’t know if this will have picked up any increased deaths because of people who didn’t go to A&E, but who maybe would have gone before their local A&E closed/was downgraded.

We’ve seen that the study found that the further it is to A&E, the fewer people make the trip. So does that mean more deaths among people who didn’t go to A&E? This is clearly important and it’s a bit frustrating not to understand the findings. We should maybe write and ask the Profs to help us out.

For example, Lincolnshire Councillor Charmaine Morgan told Grantham Matters:

“At the United Lincolnshire Hospitals Trust board meeting in December 2017, Mark Brassington advised board members that there was ‘no impact’ of the [Grantham A&E] night closure in August 2016.

“This simply could not be determined without a holistic look at all possible scenarios. For example, if a patient dies at home because their ambulance arrives too late this would not be recorded by ULHT. People using Nottingham QMC, Peterborough or Leicester A+E units, all over 23 miles away were not considered by ULHT either.”

If you feel like wracking your brains, our summary of the research findings about out of hospital deaths following local A&E closure is right at the end of this blog post, together with a list of the things that we don’t understand.

Omissions and caveats

As far as I can see the study comes with two important omissions and some big caveats.

The omissions are:

  • The effects of A&E cuts/centralisation on people without cars/on low income
  • The effects of ED closure on neighbouring EDs and hospitals.

The big caveats are:

  • The burden on individual neighbouring hospitals may have increased with unknown consequences.
  • The study’ s findings may not transfer to other areas where EDs are being considered for downgrade/closure, because the 5 A&E closures that were the subject of this study had all been run down for some time before the closures, so they may not have been providing the full range of type 1 blue light 24/7 A&E services.
  • There are huge uncertainties about whether centralising emergency care centres may be achievable without harm

Omission 1 – the effects of A&E cuts/centralisation on people without cars/on low income

One omission is that the study doesn’t investigate or take into account what we know anecdotally – that when A&E services are centralised, an effect on people without cars and on low income is that they just won’t call 999 in a health emergency because they don’t want to be taken to an A&E in another town or area.

They don’t want their family and friends to have the unaffordable expense of visiting them – and they can’t afford the expense of getting home once discharged. Public transport is expensive and often takes hours. Taxis are extremely expensive.

A&E centralisation makes health inequalities worse.

This urgently needs attention before it’s too late. This might not immediately lead to deaths, but it might well lead to increased suffering, delayed treatments and worsening health.

We need to investigate numbers of patients who DIDN’T call an ambulance after the closure/downgrade of their A&E

The study’s main outcome measures were

  • ambulance service incident volumes and times,
  • the number of emergency and urgent care attendances at EDs,
  • the number of emergency hospital admissions,
  • mortality, and case fatality ratios.

It would have been better if they’d found a way to investigate numbers of patients who DIDN’T call an ambulance after the closure/downgrade of their A&E. But of course that data is not collected by ANYONE and for sure not by the sources they used.

Maybe that is something campaign groups should start collecting and ask the Sheffield Profs to design a way of collecting that info?

The Conclusions kind of allow for the need for this. They say that future research should ask:

“How do patients experience reorganisation? Do patients change their help-seeking behaviour following reorganisation and how does this affect their satisfaction with care?”

(p 113)

Too damn right.

The study also references a study from the USA that found that:

“geographical distance has an impact on ER utilization, especially by reducing utilization in disadvantaged block group areas. Disadvantaged persons living near ER hospitals (<5 miles) were found to be more likely to utilize the ER services. Geographic distance should therefore be considered when planning state-wide ER programmes for disadvantaged populations.”

(Lee JE, Sung JH, Ward WB, Fos PJ, Lee WJ, Kim JC. Utilization of the emergency room: impact of geographic distance. Geospat Health 2007;1 :243– 53. https://doi.org/10.4081/gh.2007.272

The other big omission is the absence of data about the effects of ED closure on neighbouring EDs and hospitals

This is despite campaigners’ concerns that the remaining EDs would struggle with additional attendances/admissions, evidence that in some cases this has happened, and a reliable study showing that inpatient death rates increased in Californian hospitals with A&Es, after A&Es in neighbouring hospitals were closed.

(The research does include a reference to this: Liu C, Srebotnjak T, Hsia RY. California emergency department closures are associated with increased inpatient mortality at nearby hospitals. Health Aff 2014;33 :1323– 9.)

The study identifies that the effects of ED closure on neighbouring EDs and hospitals needs to be studied. Future research should ask:

“What is the effect on neighbouring areas? Are hospitals able to absorb any additional workload? what are the implications for the population in these areas? Is there any impact on the workforce in these areas in terms of satisfaction?”

P 113

The study reported that in some of the areas studied, emergency hospital admissions were no longer accepted at the NHS acute trust of the closure site, following the closure of the ED, and that this may increase the burden on the neighbouring A&E hospital:

“Although difficult to ascertain from the documentary analysis, this appeared to be evident in Rochdale, where all emergency admissions appeared to be transferred to a neighbouring hospital, and to some extent in Hartlepool, where a reduced emergency admissions facility appeared to operate following the closure of the ED. Therefore, either all or some of the emergency admissions were transferred to a neighbouring acute hospital and an emergency admissions unit was effectively closed. Added to the existing emergency admissions population in the neighbouring hospital, this reorganisation creates an additional emergency admissions population for the neighbouring hospital and may increase the burden on this hospital.”

P107

There is an important caveat associated with the absence of data to identify the effects of ED closure on neighbouring EDs and hospitals.

Caveats – “ the burden on individual neighbouring hospitals may have increased with unknown consequences.”

Lack of precision in the available data meant that the study was unable to identify the additional numbers of type 1 A&E attendances that might fall on neighbouring emergency departments, following closure of a type 1 A&E.

(Type 1 attendances are patients with life-threatening emergencies and a type 1 A&E is a full blue light 24.7 A&E that is equipped to deal with patients suffering life-threatening emergencies.)

The language is a bit confusing, but the Study reports (key bits in bold italic):

“With regard to attendance at an emergency or urgent care service, the implication of any additional workload is less likely to be evident. Unlike emergency admissions, in each of the areas where a type 1 ED closed, a replacement urgent care service was provided that would have been able to manage a proportion of the patients who were attending the ED before closure. These patients would not need to go to neighbouring EDs, and so we might expect that the impact of closing an ED is less for neighbouring EDs than for neighbouring emergency admission services. In our study, emergency and urgent care attendances combined both type 1, 2, 3 and 4 ED attendances and those attendances recorded as ‘ unknown’ .

P107

We were unable to isolate type 1 attendances so we are unsure exactly what the impact for type 1 attendances that move to neighbouring EDs was. However, any increase in attendances at a neighbouring type 1 ED may then have implications for waiting times, crowding, staffing levels and, ultimately, safety. We were unable to identify, through documentary analysis, how the neighbouring hospitals absorbed any increased workload and whether or not this had implications for their performance.”

P107

It adds,

“ the burden on individual neighbouring hospitals may have increased with unknown consequences.”

P108

One reason for this (p108) ‘is because the 5 A&E closures that were the subject of this study had all been run down for some time before the closures, so

Another important caveat – the research findings may not transfer to other areas where EDs are being considered for downgrade/closure

“Given this, patients with serious clinical conditions (such as MI, stroke or trauma) may have already been diverted away from the ED that closed to a regional specialist centre and, therefore, the EDs we studied may have been delivering care within a less clinically acute environment at the time of the closure.”

Sound familiar, Grantham, Huddersfield, Chorley and others?

Relatedly

“The analysis of data in the time series spanned the 24 months before and 24 months after the closure. It is likely that local health systems began to adopt different ways of providing care prior to the closure… It appears that some EDs were already operating at a ‘reduced’ care level prior to closure but it was not possible to establish this with any precise detail or certainty. Given that some of the EDs closed up to 5 years after their first public consultation regarding the reorganisation, there is a possibility that NHS providers within the emergency and urgent care system began to introduce small changes to their practice in anticipation of the closure ahead of our 24-month pre-closure time series. We also found some evidence that ambulance services may have anticipated the closures, for example by transporting patients to other hospitals prior to the closure. Our analyses have focused on the impact at the time of the closure, not on the impact of the closure. Of course, there is a clear argument that this is what we should focus on because if the other changes were made prior to the closure then they could be made whether or not the ED closed, and so are not strictly a consequence of the closure.”

(p112)

Caveat 3 -‘Huge uncertainties about whether centralising emergency care centres may be achievable without harm’

Plus, although the research found that mortality did not increase following the 5 ED closures, the study warns that although:

“Therefore, operating fewer, larger centres for emergency care that are geographically dispersed may be achievable without harm… there are huge uncertainties about this, and, importantly, the selection of such centres would require careful consideration in terms of quality of care, staffing, distance to hospital for the catchment population affected by reorganisation, the needs of thelocal population and the current, or required, level of service provision elsewhere in the emergency and urgent care system.”

“Future work: Understanding why effects vary between sites is necessary. It is also necessary to understand the impact on patient experience. Economic evaluation to understand the cost implications of such reorganisation is also desirable.”

The study also admits that consideration should be given to the need to analyse the wider health system (e.g. to understand if the reorganisation of care has any impact on primary care).

Lack of reliable data is part of the reason for omissions and caveats

Reliable data about changes to emergency and urgent care attendances was only available for 3 of the 5 towns.

Bishop Auckland

“There was strong evidence of a large decrease in the number of emergency and urgent care attendances. This appeared to be driven by a decrease in the number of attendances by patients arriving by ambulance. This decrease remained evident when compared with the control area. However, there may be a coding issue with these data.”

(p68)

So what happened to the people who would have called an ambulance – but didn’t?

AND:

“There was strong evidence of a large decrease in both overall emergency and urgent care attendances and minor attendances when compared with those experiencing a smaller change in journey times.
However, there may be a coding issue with these data.”

Again, so what happened to the people who would have come in for emergency/urgent care, before their A&E was closed/downgraded?

Rochdale and Newark

This decrease in number of patients arriving by ambulance and decrease in emergency and urgent care attendances wasn’t found in Rochdale, Newark (where there was some evidence of an increase in the number of urgent and emergency care attendances).

Hartlepool

“There were no reliable data to inform the analysis of total emergency and urgent care service attendance. Analysis of this measure by mode of arrival indicated that there was strong evidence of a decrease in the number of attendances by patients arriving by ambulance. No comparison with the control area was possible.”

(p70)

Bit staggering that there were no reliable data for this, and the same question as for Bishop Auckland: what happened to the people who would have come in for emergency/urgent care before their A&E was closed/downgraded?

presents the results of the individual site analyses that estimated the impact of closing the EDs on the total number of emergency and urgent care attendances for residents of the catchment areas of the EDs that closed.

The report says reliable data were only available for three of the five sites: Rochdale, Newark and Bishop Auckland.

Figure 11, p79

I can’t read Figure 11 – I don’t know what it means, But the blurb underneath says,

“Over all sites, there was no statistically reliable evidence to suggest a change in the total number of emergency and urgent care attendances following the closure of a local ED. It was estimated that there was a reduction in the total number of emergency and urgent care attendances at the time of closure of–8.4% (95% CI –23.5% to 6.7%; p = 0.27).

Figure 11, p79

So are they saying after an initial fall in attendances following closure of local ED, on average the numbers recovered?



Help us understand the statistics for out-of-hospital deaths following local A&E closure! This matters!

It seems at first sight that the Profs define out-of-hospital deaths as deaths that are related to A&E attendance/admission but that occurred out of the hospital. eg

“ in our study, ‘out-of-hospital’ deaths may occur at home prior to making contact with a service, in an ambulance on the way to hospital, in the ED prior to hospital admission or when a patient is recorded as having a SEC incident and are admitted, discharged alive and subsequently die out of hospital (i.e. without being readmitted). We estimate that the majority of these deaths occur within a pre-hospital setting.”

(p111)

But then they seem to say that the out-of-hospital deaths also include those of people who had no contact with emergency services:

“For some of these ‘out-of-hospital’ deaths, it is worth noting that there was little that the emergency care system could do to prevent this death (e.g. if a person died without making contact with an emergency service).”

So did they include all serious emergency case deaths in the out of hospital deaths, whether or not they were related to 999 calls?

Whatever, their conclusion was:

“Over all sites, there was no statistically reliable evidence to suggest a change in the total number of out-of-hospital deaths following the closure of a local ED. The estimated effect was a decrease of –4.2% (95% CI –15.9% to 7.7%; p = 0.49).”

Figure 26, p94

The confusion just got worse! Does this mean there was actually a decrease in out-of-hospital deaths? Any statisticians out there who can interpret this, please?

I can’t make head or tail of this, because the study also says that out-of- hospital deaths in the catchments of EDs that were closed, when compared with control sites, showed that

“the estimated effect was a slight increase of 2.2% (95% CI –6.9% to 11.3%; p = 0.63).”

(Figure 27)

But the blurb under Figure 27 also says

“Compared with the control sites, over all sites, there was no statistically reliable evidence to suggest a change in the number of out-of-hospital deaths from Serious Emergency Conditions following the closure of a local ED.”

So does this mean that the slight estimate increase is statistically insignificant? This NHS Detective is confused.

2 comments

  1. It is telling that poor people are not one of the protected groups in the equality laws. This means that the NHS doesn’t have to consider the effects of proposed significant service changes on people on low incomes, although they do for all the groups that are protected under equality legislation. Yet NHS England is banging on about reducing health inequalities.

    Like

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

This site uses Akismet to reduce spam. Learn how your comment data is processed.