What characteristics of clinical decision support system implementations lead to adoption for regular use? A scoping review

Quality assessment

Of the 22 CDS tools, 18 had evidence of over half the STARI checklist domains. Many publications associated with the projects focused exclusively on either implementation or intervention. See online supplemental appendix 2 for full scoring.

Synthesis of results

The questions in each NPT domain answered from the published information for each CDS (online supplemental appendix 4) were colour-coded with red for a ‘no’ or negative response; amber for a ‘maybe’, neutral or ambivalent response and green for a ‘yes’ or positive response (table 3). If there was insufficient information, this was deemed ‘not reported’ and colour-coded grey (figure 3). See online supplemental appendix 4 for full definitions of terms and synthesis explanation.

Coherence

‘Is there a clear understanding of the new service?’ scored well with 64% positive ‘Yes’ reports (figure 3). Publications provide explanations that are clear to the reader, however, there is limited reporting of the explanation given to clinicians involved. Frequently, the CDS replicate published guidelines.

‘Do individuals have a shared understanding of the aims, objectives and benefits?’ was variable. It appeared that mandating use to circumvent this aspect has been used in some cases (F, Q, V, Z) . In others, the CDS was ‘communicated’ to staff (figure 4) but there was no evidence of understanding its purpose (A, H, UU). Training and targeted advertising appear to have been successful in many cases. Where training was used, the emphasis was on bespoke training by profession or clinical role.

Only 36% of studies said ‘yes’ in response to ‘Do individuals have a clear understanding of their task or responsibility?’, and several studies reported ‘no’ that is, that during implementation individuals explicitly expressed a lack of understanding (A, F, MM). Understanding role-specific tasks and responsibilities was problematic at the outset of most of the implementation projects that reported on this (figure 4).

‘Do individuals understand the value, benefits and importance of the service?’ was variably reported. Where uptake of the CDS was low, implementers in some cases mandated use to achieve uptake. Learning from inappropriate requests was mentioned as a benefit among multiple CDS projects. Where clinicians were aware of improvements in costs, efficiency or compliance with guidelines, this was well received (A, B, S,KK).

Cognitive participation

There were 45% ‘yes’ responses to ‘Do individuals buy into the idea of the service?’ (figure 3). Five projects relied on mandated use of the CDS to achieve uptake, rather than encouraging buy-in (F, Q, V, Z, FF). There were significant barriers to engagement in many cases. Training and teamwork appeared to be facilitating factors (H,X) along with time-saving interventions embedded in the CDS pathway or workflow (JJ).

‘Can individuals sustain involvement?’ scored 59% ‘yes’ responses. Across the board, integration with existing electronic health record (EHR) was key to sustaining involvement (Figure 5) (B, K, GG).

41% of CDS did not report any evidence for ‘Are key individuals willing to drive the implementation?’ Of those that mentioned key individuals, management support was key (Z, GG, MM) and having a small team of known team members to discuss and encourage usage (F, H, CC).

‘Do individuals feel it is right for them to be involved?’ was the least reported element of the framework (73% not reported). Of the five projects that reported against this question, two reported a negative response (Figure 5) (CC,OO). Only four projects contained any report on this domain (HH, MM, OO, UU). Where problems were identified by the team undertaking the implementation, it was clear they had a mandate to be involved.

Collective action

Organisational support for the CDS project was reported positively in 45% of projects (figure 3). Where it was not explicitly reported, indications of organisational support included enabling significant changes to a local care pathway (F). Support came from a wide range of sources, including clinical commissioning groups in the UK (F, CC, KK), insurance companies in the USA (Z) or professional bodies such as the Swedish Pharmacy Association (M). Organisational incentives to introduce a CDS were usually cost and efficiency savings (figure 6). These priorities did not necessarily align with those mentioned by the users: safety and improved patient care. Even where the CDS had no key individuals to drive the implementation (K, RR), the organisational support led to success.

‘How does the innovation affect roles and responsibility or training needs?’ was positively reported in 41% of cases. There are examples of implementation leading to positive role change within a team (Figure 7). For example, a change in the pharmacists’ and microbiologists’ roles from being exclusively lab based to having a patient-facing role was reported (KK). Where the CDS replicated best practice guidelines, subsequent service audits demonstrated improvement in adherence to guidelines long term (K, KK, NN).

There were only 32% ‘yes’ responses to ‘Does the service make people’s work easier?’. Additional time burden was a perceived disadvantage (Figure 7). Some factored in incremental changes, and therefore, the burden was mitigated in response to feedback. Examples of mitigation included increasing integration with the EHR,32 improved automation of inputs (B) or removing another burdensome task to make the CDS timesaving or time-neutral (JJ).

‘Do individuals have confidence in the new system?’ was answered ‘yes’ in 45% of projects, however, 50% was not reported. Where there was feedback from clinicians, the studies reported confidence in the new system almost universally (B, MM, OO, UU).

Reflexive monitoring

‘Do individuals try to alter the new service ?’ had 14% ‘yes’ answers (figure 3). Lack of flexibility for clinical judgement was a clear negative aspect of implementations; this led to circumventing (F), ‘gaming’ (A), or ignoring the CDS (M, CC). Nevertheless, adherence to guidelines and better patient care was frequently assessed as a benefit of the CDS. Where there was an additional, unmitigated administrative burden, CDS implementations were viewed negatively (N, Q, Z, HH). However, where mitigations were introduced, participants reported positive effects on knowledge and patient care (GG, UU).

There were 18% positive responses to: ‘How do individuals appraise the effects on them and their work environment?’. Effects on clinicians and clinical environment were often not reported (50%), and when it is, it is mostly negative—time burden (Z), alert fatigue (CC) and administrative tasks were identified downsides (GG).

‘How are benefits or problems identified or measured?’ was positively reported in 55% of CDS. Many of the projects either planned a qualitative element to their studies (H, Z, CC), or an iterative approach (B, F) to implementation that allowed for feedback. This meant that problems with a CDS could be identified once in practice and subsequently solved.