[ad_1]
WHAT IS ALREADY KNOWN ON THIS TOPIC
-
CDSS alerts are crucial for clinical decision-making, but frequent alerts can lead to alert fatigue, affecting physician acceptance. The TAM has been widely used to study factors influencing technology acceptance, including healthcare.
WHAT THIS STUDY ADDS
-
This study highlights that high patient volume and older age decrease PU and PEOU of CDSS alerts, while greater clinical experience positively influences these perceptions. Customisation and context-aware alert systems can improve acceptance.
HOW THIS STUDY MIGHT AFFECT RESEARCH, PRACTICE OR POLICY
-
Findings underscore the importance of context-driven customisation for CDSS alerts, suggesting artificial intelligence-based adaptive alerts to optimise clinical workflows and reduce alert fatigue, thus guiding CDSS development and policy adjustments in healthcare.
Introduction
In modern medicine, healthcare providers frequently rely on clinical decision support system (CDSS) alerts to notify physicians and prevent errors.1 2 However, these systems are often flooded with irrelevant, low-sensitivity alerts, leading physicians to become desensitised.3 This phenomenon, known as ‘alert fatigue’, can cause physicians to overlook important notifications, impacting patient care.4 To address this, researchers have been actively exploring ways to reduce alert fatigue and enhance alert effectiveness.5
Many studies have investigated factors affecting physician interaction with alerts. For example, one study suggests that setting alert sensitivity above 0.7 and making alerts more context-specific can reduce cognitive burden.6 Tailoring alerts to fit the user’s environment and preferences has also been shown to improve efficiency and acceptance.6 Drawing from fields like mobile technology, CDSS developers can optimise alert systems based on user feedback to create a more satisfying experience.3 Thus, understanding the key factors important to CDSS users is crucial for designing systems that improve both workflow efficiency and user satisfaction.
The Technology Acceptance Model (TAM) provides a strong framework to explain user acceptance of technology,7 and it has been applied in healthcare settings to study acceptance factors like interactivity, timing, and interface design.8 Although numerous studies have explored the factors influencing behavioural intention to use CDSS alerts via TAM, much of the existing research focuses primarily on medication-related alerts,9 10 while administrative alerts—which also play a crucial role in CDSS—remain underexplored. Understanding the acceptance of these types of alerts is equally important, as it helps bridge this research gap and contributes to building a structured framework for evaluating physician adoption and system effectiveness.
Thus, to address this gap, our study examines physicians’ behavioural intention to use CDSS alerts of the top 20 most frequently triggered alerts in our CDSS, the majority of which are administrative type, using the TAM. The TAM framework defines key relationships between external variables and core constructs: perceived usefulness (PU), perceived ease of use (PEOU), attitude towards usage (AT), behavioural intention to use (BI) and actual system usage (AU). In this study, external variables were selected based on the literature and discussions with clinical experts, focusing on seven critical physician-related factors: age, clinical experience, CDSS operating status, patient volume, weekly consultation frequency, gender and specialty. By applying TAM, this study seeks to inform the design of CDSS alerts to reduce alert fatigue, enhance physician engagement and ultimately improve clinical workflow efficiency.
Materials and methods
Study settings and CDSS alert context
This study was conducted at the outpatient clinic of Taipei Municipal WanFang Hospital (WFH), an academic medical centre in Taiwan, from 6 September 2021 to 29 April 2022. WFH has implemented a homegrown CDSS with a pop-up alert feature for over 25 years, generating thousands of alerts daily. These alerts include both administrative and clinical notifications, with varying levels of relevance to physicians’ decision-making. To evaluate physicians’ behavioural intention to use CDSS alerts, we applied the TAM and conducted a structured questionnaire survey among outpatient clinic physicians. The study received approval from the Taipei Medical University Research Ethics Board (protocol codes N202105104 and 20210616). The study settings, hospital background and CDSS alert examples are detailed in online supplemental materials under the section titled ‘Study settings, hospital background and CDSS alert examples’ (online supplemental table S1).11 12
Hypotheses and questionnaire design
This study adopted the TAM to investigate physicians’ perspectives on CDSS alerts, incorporating external factors such as age, clinical experience, patient volume and specialty. In addition to the original TAM constructs—PU, PEOU, AT, BI and AU—we introduced User Satisfaction (US) to better capture physicians’ experiences. A structured questionnaire, designed around these constructs, consisted of demographic information and five TAM-based scales, assessed using a five-point Likert scale. The study framework included 22 hypotheses, with 14 related to user characteristics and eight based on TAM constructs, as illustrated in figure 1. The detailed TAM model structure, hypothesis framework and questionnaire design can be found in the online supplemental materials section, titled ‘TAM model structure, hypothesis framework and questionnaire design’ (online supplemental table S2).7 10 13–16
The architecture of study hypotheses. CDSS, clinical decision support system.
Model validation and statistical analysis
Before distributing the questionnaire, 11 clinical experts reviewed its items for content validity and reliability. The content validity index (CVI) was calculated, and Cronbach’s alpha (α) was used to assess internal consistency. Statistical analyses included Pearson’s correlation to examine variable relationships and multiple regression analysis to evaluate the contribution of independent predictors while controlling for confounding factors. Variance inflation factor (VIF) was used to test for multicollinearity, ensuring predictor independence. All statistical analyses were conducted using SPSS version 24.0 (IBM Corp, Armonk, New York). The detailed descriptions of content validity, reliability assessment and statistical methods are extensively documented in the online supplemental materials, specifically in the section titled ‘Content validity, reliability assessment and statistical methods’ (online supplemental table S3).17–21
Results
Validity, reliability and final questionnaire constructs
The final questionnaire assessed five key constructs: PU, PEOU, AT, BI and US, capturing physicians’ perceptions of the CDSS alert system (table 1). Each construct included multiple items evaluating aspects such as system efficiency, alert frequency and user engagement. Content validity was confirmed through expert panel assessments, with item-level CVI (I-CVI) values consistently reaching 1.0 and scale-level CVI (S-CVI) exceeding 0.80, indicating strong agreement on relevance and clarity. Reliability analysis demonstrated high internal consistency, with Cronbach’s alpha (α) values ranging from 0.89 (PEOU) to 0.95 (US), ensuring the robustness of the questionnaire. The detailed CVI assessments and reliability coefficients are available in online supplemental materials titled ‘Results of questionnaire items, validation and reliability analysis’ (online supplemental table S4).
Questions of five constructs
Physicians’ demographic characteristics and comments on CDSS alerts
Table 2 summarises the demographic characteristics of the 72 respondents, revealing a predominantly male sample (73.6%) with an average clinical experience of 15.44 years (SD=±9.6). Most physicians fully operated the CDSS (87.5%), indicating high familiarity with the system. The respondents had diverse specialisations, with internal medicine being the most common (30.6%). Patient volume varied, with one-third managing ≤20 patients per week and weekly consultation frequencies mostly falling within 3–4 sessions.
Demographic characteristics of the respondents (n=72)
Survey results showed that most respondents selected ‘agree’ or ‘neutral’ on the questionnaire items, with 52 out of 72 physicians rating ‘AT1: it is great to have clinical alerts in the CDSS system’ above four points. However, over 42 physicians disagreed with statements suggesting the current alert system is well-designed and does not require improvement. Physicians’ feedback highlighted concerns about alert overload, as frequent pop-ups—sometimes exceeding ten per patient—disrupted workflow and contributed to alert fatigue, leading to overlooked notifications. To improve functionality, physicians suggested consolidating alerts, integrating AI for better prioritisation and reducing unnecessary administrative notifications to enhance efficiency and minimise disruptions. For detailed scoring results and comments on the CDSS alert, see the online supplementary materials, specifically in the section titled ‘Results of physician perspectives on CDSS alerts and improvement suggestions’ (online supplemental figure S2 and table S5).
Hypothesis testing
To assess the relationships between key variables, multiple regression analyses were conducted. VIF results confirmed no collinearity concerns among predictors (VIF <10).22 The findings of this research support the hypotheses outlined in figure 2, demonstrating that user characteristics, including age, clinical experience and patient volume, significantly influenced perceptions of PEOU and PU, collectively explaining 16% and 23% of their variance, respectively. Further analysis demonstrated that PEOU positively influenced PU (R²=0.67, p<0.001), while both PEOU and PU significantly shaped AT and US (R²=0.74 and 0.63, respectively, p<0.001). Moreover, PU and AT had a strong impact on BI (R²=0.83, p<0.001), whereas US did not significantly predict BI. These results emphasise that ease of use and perceived usefulness are critical factors influencing physicians’ willingness to engage with CDSS alerts. The detailed regression coefficients and statistical outputs for the six models are available in the online supplementary materials, specifically in the section titled ‘Results of regression analysis and hypothesis testing’ (online supplemental table S6).22
Result of study hypothesis analysis. CDSS, clinical decision support system.
Discussion
This study highlights several factors influencing physicians’ perspectives of CDSS alerts, especially the role of perceived ease of use in shaping perceived usefulness, user satisfaction and behavioural intention. Physicians with high patient volumes found CDSS alerts less useful and harder to use, likely due to cognitive load and alert fatigue. Older physicians also reported lower ease of use and usefulness, while those with greater clinical experience viewed alerts more positively, possibly due to familiarity with integrating them into workflows. Meanwhile, ease of use and usefulness were strongly associated with positive attitudes and satisfaction; user satisfaction alone did not significantly predict behavioural intention to use the system. Most physicians agreed that pop-up alerts were effective reminders and expressed a willingness to use them during clinical decision-making; however, they also suggested that adopting context-aware alerts, which optimise triggering rules based on specific situations, could enhance relevance. These findings provide hospitals with a holistic understanding of the factors influencing physicians’ behavioural intention to use CDSS alerts, offering guidance for CDSS alert improvements to enhance clinical workflow.
Our study aligns with previous research underscoring the importance of perceived usefulness and ease of use in the acceptance of CDSS alerts, with particular emphasis on alert frequency and context-specific features as key factors in user acceptance. For example, Van de Sijpe et al found that integrating patient-specific factors into CDSS alerts improved relevance, reduced alert fatigue and increased acceptance.23 Similarly, Hussain et al showed that tailoring alerts to specific clinical roles and refining interaction design helped minimise alert fatigue and enhance acceptance.5 Ancker et al further demonstrated that reducing the frequency of non-informative alerts can alleviate alert fatigue and increase user engagement.4 However, our findings suggested that physicians with high patient loads and older age tend to reduce acceptance. Additionally, unlike some studies that emphasise user satisfaction as a key driver of sustained system use, our research did not find a significant correlation between satisfaction and behavioural intention to use. This indicates that in high-pressure settings, practical factors like perceived usefulness and ease of use may play a more decisive role than satisfaction in influencing continued usage.
The implications of this study are substantial for clinical practice and policy-making, underscoring the importance of context-driven customisation in CDSS alerts. Our findings highlight the strong impact of perceived ease of use and usefulness on physicians’ behavioural intention to use CDSS alerts, suggesting that CDSS developers should prioritise user-friendly designs that minimise disruption and cognitive load, especially for clinicians with high patient volumes. Aligned with the ‘CDS Five Rights’ framework—delivering the right information to the right person, in the right format, through the right channel and at the right time—CDSS alerts would benefit from AI-based context-sensitive systems rather than traditional rule-based designs.24 An AI-driven approach could dynamically adjust alert timing, frequency and content based on real-time clinical context, sending notifications only at critical workflow points, such as the start of a visit, rather than repetitively.25 Policymakers can leverage these insights to develop evidence-based guidelines and visual prioritisation strategies for alert frequency and customisation, fostering efficient and clinician-centred CDSS use.26
Building on these insights, future research should examine additional factors that may influence behavioural intention to use CDSS alerts beyond perceived usefulness, ease of use and user satisfaction. As our findings suggest that context and workload play a pivotal role in how alerts are perceived, further exploration of adaptive, context-sensitive alert systems could be valuable. For hospitals aiming to optimise CDSS alerts, we recommend focusing on several practical strategies: reducing alert frequency through regular system maintenance,5 27 ensuring alerts are highly relevant to clinical processes and tailoring alert presentations based on clinical importance. Clinical alerts presented as pop-up notifications were generally preferred by physicians, whereas administrative alerts, often seen as less valuable, might benefit from alternative notification formats, such as color-coded text or soft-stop warnings.28 Engaging high-volume clinicians in feedback loops for system improvements can further enhance CDSS acceptance, ultimately contributing to improved patient safety and more efficient clinical workflows.
Limitations
This study has several limitations that should be considered. First, factors such as changes in hospital policies, the implementation of new alerts and Public Health Emergencies of International Concern may impact the findings over time, highlighting the need for regular surveys to capture longitudinal data and provide deeper insights. To address this, regular surveys should be conducted to gather longitudinal data and provide deeper insights. Second, the study focused on outpatient settings, where the context of alerts may differ significantly from inpatient environments. Therefore, our findings may not fully apply to inpatient settings, where alert needs and workflows could vary substantially. Meanwhile, this study had a relatively low response rate, which may affect the representativeness of our results and could introduce response bias. Future studies with larger sample sizes across different settings could help validate these findings. Additionally, the data for this study were collected from a single academic medical centre in Taiwan. This may not accurately reflect the clinical workflow of other hospitals, thereby limiting the generalisability of our findings. Furthermore, while behavioural intention to use can predict actual usage of alerts, it does not equate to actual alert usage. This distinction should be considered when interpreting the findings, and future studies should aim to examine real-world alert usage in addition to behavioural intentions. Lastly, the quantitative analysis method used in this study has its limitations, as the results are subjective and heavily influenced by the characteristics of the respondents. To address this, future studies should include quantitative metrics such as alert burden, alert overridden rate, and alert dwell time.
Conclusion
This study used the TAM to identify key factors influencing physicians’ behavioural intention to use CDSS alerts within an outpatient academic medical centre. Our findings demonstrated that high patient volume and older age negatively impact physicians’ perspectives of the system’s ease of use and usefulness, while greater clinical experience positively influences these perceptions. The clinical pop-up alerts were generally effective and well-received; low-value administrative alerts could benefit from alternative formats—such as color-coding, light indicators or soft stops—to reduce disruption and streamline the alert system. Physicians were also highly sensitive to alert frequency, particularly for non-clinical notifications, which may contribute to alert fatigue if not carefully managed. These insights offer valuable guidance for CDSS developers and hospital administrators aiming to design context-sensitive alerts aligned with the ‘CDS Five Rights’ framework. Implementing AI-based, adaptive alert systems that adjust to clinical context and workload can make CDSS more effective and aligned with real-world clinical needs, paving the way for improved workflows and patient safety.
Data availability statement
All data relevant to the study are included in the article or uploaded as supplementary information.
Ethics statements
Patient consent for publication
Ethics approval
This study involves human participants. The study was conducted according to the guidelines of the Declaration of Helsinki and approved by the Institutional Review Board (or Ethics Committee) of Taipei Medical University Research Ethics Board (protocol code N202105104 and 20210616 of approval). Participants gave informed consent to participate in the study before taking part.
Acknowledgments
We want to thank Ruey-Jing Wu (Information Technology Office in Taipei Municipal Wan Fang Hospital, Taipei Medical University, Taipei 110, Taiwan) for her administrative support and physicians who provided valuable insights during our study.
[ad_2]
Source link
