Involving multiple stakeholders in assessing and reviewing a novel data visualisation tool for a national neonatal data asset

Struggling to consume information

Parents described difficulty with comprehending health information due to the emotional strain of their baby being on the neonatal unit. Indeed, one felt the amount of data featured in the tool was excessive:

‘That is way too much data… I think I would sit down… look at that… and say, “Oh my god”, and… panic’ (Mum).

However, while another participant concurred that the tool contained a large amount of information, they indicated that some parents desired even more than this. They described a complex balance between the volume of data required for decision-making and the type or amount of information capable of worsening their distress, particularly when the baby’s survival is precarious:

‘…We had [him at] 23 [weeks]… [the doctors] said: “Baby is going to be born. We can either take them away from you for… assessment, but by the time we hand him back to you, he might already be dead. Or you can give him a cuddle as he passes away. We’ll come back in 30 minutes…” It is horrific. A decision needs to be made, ultimately, so you need… to know these things…’ (Mum).

Dads and data

Across all parent and preterm adult consultation groups, fathers expressed interest in statistics on neonatal conditions and complications, saying: ‘I wanted the stats,’ or: ‘I wanted numbers.’ They described a gendered division of parental labour, with mothers handling tasks ‘closer’ to the baby, such as expressing milk. These fathers regarded affinity for data as supporting their baby and combating their inability to fulfil stereotypical masculine ideals:

‘They were telling us to stop looking at the machine and look at the baby. I couldn’t help [it]. I stood with the machine, I was analysing numbers… As a man, you want to try and protect them. You want… to try and do something, and it’s completely useless’ (Dad).

Bringing the data to life

Several participants desired to have individual case stories that represented their experiences; for them, the scale of the data muted how they related to the experiences of other families:

‘You really need to provide actual stories of patients and parents, so that those statistics are not [just] statistics. So when you click on some of those dots, they take you to parent voices, interviews – good and bad’ (Preterm Adult Woman).

Yearning for predictions

Many participants expressed a need for, and limited access to, clear statements explaining risks, in a format such as: ‘X out of 10 000 preterm babies will experience this.’ They recognised each baby’s outcomes would be ‘different’, complicating prediction. However, they believed accessing large-scale data visualisations could help alleviate this:

‘I asked for kind of stats because I needed to prepare myself… I had four days… so I think being armed with some form of stats [would have been helpful] ’ (Dad).

Although the tools were not designed to predict outcomes, one parent perceived data—specifically probability data—as being particularly helpful in navigating the swiftly changing risks that neonatal babies face:

‘You think “we’re breezing this”… then it really hits the fan… [data] gives you something to hold onto… it allows you to manage your expectations… [the consultants’] knowledge isn’t going to be anywhere near as broad as the data… the conversation I had with them all the time was, “ what are the chances of X?” “ I don’t know, all babies are different.” “Okay but if you’ve got 10 k of these babies, what are the chances?” “We haven’t got that information”’ (Dad).

Parents pointed out that accessing the tool would offer an opportunity to reduce encroaching on clinicians’ time with requests for detailed explanations.

Comparativeness of data across units

The most salient theme for this group was to understand differences between their unit’s outcomes and those of other comparable units:

‘Something that allows us to make sure that we’re comparing like for like and that we’re not comparing a cohort of babies in… [our network] and thinking we’re doing terribly because [another unit is] doing much better… [when] actually their data is different…’ (Data Manager).

Presently, units define their standards by routinely monitoring data and comparing themselves to trends set by the collective work of units, but this can be challenging:

‘I can pull things out from [electronic patient record system] but it might be very complicated… Rather than me having to constantly monitor data… if every year, I’m going to look at these lines illustrating differences across units, I know where I am heading to and where I need to work’ (Data Manager).

‘Allowing us to identify where those differences in practice may be occurring, and pointers as to where we can go to either to review ourselves compared to peers around the country or which units we need to… focus on’ (Data Manager).

This group also focused heavily on the opportunities the tool would offer such as direct comparisons with other units. They recommended these be made between overarching operational delivery networks (ODNs) and between individual units within an ODN. One network manager reported that they were happy to feed back to their wider network, and expressed interest in sharing public-facing network-level data if anonymised and password protected:

‘…I can see this taking off, it’s very detailed and it provides a lot of flexibility… there’s room for it to grow and develop… there’s lots of things like audit and parent engagement, I can imagine they’d like to see this data to kind of reassure them; it covers all the bases’ (Network Manager).

One network manager discussed specific ODN needs:

‘It will certainly aid the standardisation of review… the ODNs all have a data manager, but there’s variance… each staff member is very different, and the needs of networks are quite different: it’s a genuine comparison with like for like… a huge step forward’ (Network Manager).

Another agreed, saying:

‘Benchmarking is one of the key things we’ve struggled with for such a long time. Locally, we can compare apples with apples but my data and other managers’ would be apples and pears. That’s where the tools come into their own [sic]’ (Network Manager).

Additional outcomes and transfers

Discussion of additional outcomes beyond survival and length of stay focused on understanding reasons for admission and its variation between units. However, participants also expressed the desire for data on a wide range of outcomes, including breast feeding, intraventricular haemorrhage, modifiable factors, tube feeding in hospital and on discharge, probiotics, necrotising enterocolitis, lung disease, ventilator use and transfers. Transfers were also discussed as a particularly difficult phenomenon to track because of the varying reasons for why they occur:

‘…I know there are two units whose babies should be coming to me – if they’re not, why? We don’t capture that… so we have to rely on audits…. It’s a big exercise capturing that, and the focus nationally is on trying to reduce transfers. What is their trajectory?’ (Clinician).

Socioeconomic status and ethnicity

Central to comparing units and ODN was identifying areas of disproportionate need, particularly how socioeconomic status and ethnicity shape their neonatal unit populations and admissions:

‘…Some of the sliders in the tool would filter and say, “Out of your population of your 24–26 weekers, view that by ethnicity or deprivation”… It can help us understand where the maternity drivers need to be and to help some of those mums in a more focused and targeted way to understand getting better antenatal care…’ (Clinician).

Timeliness of data updates

Updating neonatal data was an integral complexity, caused by the length of time before results of clinical practice adjustments were observable:

‘ If we made a change, we’re not going to see it for another year…. If we’ve been doing the wrong thing, it takes us a long time to find out’ (Data Manager).

Participants said the visualisation tool could provide ‘standardisation’ of and ‘more frequent access to’ data on a regular basis, but neonatal care processes cause a ‘lag’ in data access:

‘Anything we do monthly we’re not going to have any outcomes unless they died within a few days. We don’t know lengths of stay for 3 or 4 months in some cases… Having it too quickly is detrimental, we don’t get the true picture…’ (Data Manager).

One participant emphasised the solution would lie in ‘access to timely data but not too timely data.’

Widening availability of the tool

One clinician described themself as a ‘technical dinosaur’ and, like the parent and preterm adult groups, said the visual nature of the tool was helpful. They also emphasised the importance of sharing visualisations with parents as PDF printouts and expressed strong interest in distributing the tool demonstration to data manager groups. They added that the inclusion of explanatory notes or supplementary material within the tool instead of simplifying existing content could support future interpretation. The network managers did not anticipate parental concerns regarding data use:

‘ …They trust that professionals are doing that judgement call. If it’s good valid data, that’s why they’re showing them; as opposed to parents questioning data, then questioning what the answers are…’ (Network Manager).

The only exception was where there were very low numbers, for example, rare congenital anomalies, which could, coupled with geographical location or gestational age, identify babies. They felt confident they could reassure parents that data were insufficiently fine-grained for this.

Interrogating the data

Trialists and epidemiologists enquired about the tool’s versatility, praising its usefulness for grants and study planning. They asked whether data will be broken down by conditions, and if capability exists for cross tabulations of more than two variables, such as filters exploring birth weight, gestational age and status 1 year later. One said:

‘That would be really helpful for planning any study, not just trials, to get down to that level’ (Epidemiologist).

Trialists, like network managers, advocated for supplementary information including a data dictionary, suggesting this could be ‘hover and click.’ Like data managers, they cautioned against fine-grained attributes, explaining that, for example, the small number of 500 g birthweight babies ‘etch themselves on your brain.’ Another triallist expressed desire for data categories, ‘given the data spans ten years’ (Triallist).

Another broadened this point, emphasising the inclusion of missing data in categories:

‘It is still useful to have categories where there is a lot of missingness, even if they are not usable in a trial, as an indication of what is not currently routinely collected’ (Triallist).