[ad_1]
Methods and analysis
Reporting of the review findings will be in line with the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) 2020 updated guidance and exemplars for reporting systematic reviews.8 The protocol was registered in the PROSPERO registry. One change has been made to the project title from ‘artificial intelligence-driven conversational agents’ to ‘chatbot-based interventions’ which generally consider all kinds of chatbots to ensure a more comprehensive analysis of their mental health impacts by comparing different chatbot-based interventions. However, all other processes including the eligibility criteria for study inclusion, data extraction and synthesis and reporting bias assessment remain unchanged.
Eligibility criteria
The Population, Intervention, Comparison, Outcomes, Study Design framework will be employed to guide the inclusion of eligible studies.8 The eligibility criteria are shown as follows:
-
This systematic review will only include randomised controlled trials (RCTs) among Asian people of any age whom both healthy people and people at risk of mental health conditions (eg, socially disadvantaged groups including lesbian, gay, bisexual, transgender or people having chronic conditions/substance misuse) will be considered. If the study populations consist of both Asian and non-Asian people, only data derived from the Asian participants will be extracted for data analyses. Parallel, cross-over and cluster RCTs will be considered. For cross-over RCTs, only data before the cross-over period will be used for data analyses. Only RCTs with English/Chinese full text available will be included. Studies such as conference abstracts/proceedings and case studies on one or a few subjects or other than original research including grey literature, reviews, meta-analyses, study protocols, editorials and commentaries or retracted articles will be excluded.
-
Standalone chatbot-related interventions using either mobile application-based or online web-based platforms have to be in at least one of the interventional arms within the trials. Studies involving chatbot interventions combined with other types of mental health services in one single experimental arm will be excluded because the therapeutic effects of the chatbot intervention may be caused by other interventions. Meanwhile, chatbots controlled by human operators will not be considered.
-
The effects of the chatbot-based interventions will be compared with either no interventions including ‘usual care’, ‘no treatment’ and ‘wait-list control’ or compared with other interventions.
-
The RCTs aimed at examining the effects of AI-driven chatbots on improving mental health outcomes including depression, anxiety, stress, affect, distress, sleep and subjective psychological well-being9 10.
Information sources
Studies published in the databases of PubMed, CINAHL, PsycINFO and MEDLINE will be searched from their inception through December 2024. The search queries for every electronic database are summarised in online supplemental information. The studies that were conducted in Asian countries/regions will be manually identified.
Search strategy
The initial screening (titles and abstracts) and full-text assessment of the searched studies will be conducted by the first author (WL) and subsequently verified by an independent reviewer (SCL). Since subject recruitment can be conducted through internet-based methods (eg, social media) which can target populations of different countries,11 identification of whether the studies were conducted in Asian populations will be carried out through the full-text assessment (except relevant information explicitly specified in the abstracts or titles). To avoid missing any potentially eligible studies, the reference lists of the searched relevant systematic reviews2–4 12–20 and all the included articles will also be screened for additional potential studies. Disagreements between the two reviewers (WL and SCL) will be resolved by discussion involving all review authors. The proposed PRISMA 2020 flow diagram for the study selection process is presented in online supplemental figure S1.
Data extraction
Information including publication details (authors, publication year and countries/regions), participant characteristics, intervention and control groups, training volume, recruitment and completion rates, safety and adverse effects and main findings will be extracted and summarised in an evidence table by the first author. The recruitment rate will be defined as the percentage of participants giving consent after being assessed as eligible while the completion rate will be defined as the percentage of participants remaining in the study until the last assessment visits.4 To guarantee the robustness of evidence, all authors will verify and edit every entry. The template data collection form is shown in online supplemental table S1.
Risk of bias assessment
The Cochrane Risk of Bias 2 (RoB 2) will be employed to examine the RoB of the included RCTs.21 The tool is structured into five domains covering all possible biases that may potentially affect the results of the included trials, namely bias arising from the randomisation process, bias due to deviations from intended interventions, bias due to missing outcome data, bias in the measurement of the outcome and bias in the selection of the reported result. An overall RoB judgement for every study (low risk, some concerns, high risk) will be drawn based on the results of each bias domain. The RoB assessments will be first conducted by the first author and independently verified by another review author. Disagreements between the two reviewers will be resolved through discussion.
Certainty of evidence assessment
The Grading of Recommendation Assessment, Development and Evaluation approach will be employed to examine the certainty of evidence (CoE) across included studies for clinical endpoints (eg, depression and anxiety) that were most reported. Evidence about the clinical endpoints will be downgraded one place if (1) the RoB is high (ie, most included trials have some concerns or high RoB), (2) there is evidence of imprecision (ie, 95% CI fell within the minimal clinically important difference (MCID) or the effect estimate is derived from only one or two small studies), (3) there is evidence of unexplained inconsistency or high heterogeneity (ie, I2 >50%), (4) the directness of evidence is questionable in respect of the participants studied, intervention of interest, measurements of the outcome (eg, surrogate vs intended markers) and the method of the trial of a candidate intervention (ie, depending on whether direct comparisons are available or not), (5) publication bias is detected if asymmetry of the funnel plot is observed. Overall, the CoE for the clinical endpoints will be classified as very low, low, moderate and high. The first author will assess the CoE which will hence be verified by an independent reviewer. Disagreements between the two reviewers will be resolved through discussion.
Data synthesis
Meta-analysis will be conducted for mental health outcomes (eg, depression and anxiety)9 that were reported in at least two included studies.2 4 For controlled trials, pairwise meta-analyses of postintervention data will be performed using random-effects models.22 The overall effect will be examined using either standardised mean difference or weighted mean difference depending on whether the outcomes were assessed using different scales or the same scale, respectively. Subgroup analyses will also be conducted for potential moderators (eg, age, sex, ethnicity, medical history, types of chatbots and comparison controls). Boundaries of an MCID for the outcomes will be set as half of a SD from the mean value in the control arm(s) of the included studies at baseline.2
The degree of heterogeneity across study findings will be assessed using the Higgins I2 statistics. Percentages of the I2 statistics in 0–25%, 25–50% and>50% represent low, medium and high heterogeneity, respectively. For the meta-analyses that will be identified as having high heterogeneity, sensitivity analyses will be carried out by excluding studies with a high RoB. If there is missing information, we will contact the study authors directly via email. All meta-analyses in the present proposed study will be performed using Review Manager (RevMan V.5.4) software.
Assessment of publication bias
Funnel plots will be constructed to visually examine the bias if there are 10 or more trials included in the meta-analyses or subgroup analyses.8 Asymmetry of the funnel plots indicates possible publication bias.
[ad_2]
Source link
