[ad_1]
Discussion
Existing literature supports the idea that artificial intelligence (AI) can improve the clarity and comprehensibility of complex documents. Studies have shown that AI-driven tools are able to modify information to better match the reading capabilities and health literacy levels of different populations, even improving health literacy levels themselves, which is critical in medical contexts where comprehension is essential.22 23
The aim of this pilot study was to explore the potential of LLM to improve informed consent forms for vaccinations currently used at the Prevention Department of the LHU-CT. We did this by assessing three psychometric properties (adequacy, completeness and comprehensibility) of three different vaccination consent forms: consent form A (the one already in use, constructed entirely by humans), consent form B (generated entirely by ChatGPT) and consent form C (consent form A further modified by ChatGPT).
The results highlight some important findings. First, the Gulpease index, used to support the selection of consent forms to be administered to healthcare professionals, revealed that consent form C scored the highest, being the most readable. This might suggest that the best results are achieved through the collaborative use of ChatGPT and human expertise. The subsequent score of the three consent forms by healthcare professionals showed that consent forms A and C achieved very high scores, almost the maximums for all the psychometric variables and in the overall judgement. Consent form B, generated entirely by ChatGPT, scored lower, suggesting that collaboration between AI and humans yields better results: ChatGPT alone might not be sufficient to produce the best and most accepted product, and that the best results are achieved through the collaborative use of LLMs and human expertise.
Our results are in line with findings indicating that AI-assisted modifications can produce consent forms easier to read and understand. Best practices recommend keeping consent forms at a reading level suitable for the average patient, generally at or below the eighth grade. Indeed, numerous studies have shown that consent forms are often drafted in overly technical and complex language, above the average level of comprehension.24 25 Other experiments in collaboration between humans and AI in creating more accessible informed consent forms have already been conducted. For instance, the effectiveness of using ChatGPT in simplifying surgical consent forms was investigated.22 Alongside the field of vaccines, surgical procedures represent a challenge for the average citizen, making it crucial to find the best solution that allows individuals to fully understand every aspect involved. Alternative methodologies to improve consent forms have also been tested, such as the involvement of expert groups or specific professional figures such as psychologists or ethicists, which could bring about further improvements.26 These approaches, combined with AI technology, could lead to the development of consent forms that are not only legally valid, but also more accessible and user-friendly.
This study has some limitations. First, consent form B is the result of impulses sent by humans. The process of proxying with humans in the realisation of this consent form may not have been completely effective, potentially affecting the quality of the consent form produced by ChatGPT. We used a free AI model (ChatGPT 3.5), and all responses were copied and pasted directly from the tool without further editing. This approach could limit the accuracy and contextualisation of the information provided. Second, we observe a ceiling effect in the answers. Many healthcare professionals provided very high ratings for all consent forms, making it difficult to discern the subtle differences between them. This effect indicates that although the AI-generated consent forms met a basic level of acceptability, more subtle distinctions in quality were obscured by uniformly high ratings. Thus, although our pilot study demonstrates the ability of LLMs to produce satisfactory consent forms, more precise evaluation methods may be needed to fully capture their effectiveness. Furthermore, ours was a convenience sample that may not be representative of the broader population of healthcare providers. The involvement of prompt engineering experts could refine the AI output, ensuring more accurate and contextually appropriate consent forms. This study also has a great strength in that it is a pioneering attempt to exploit LLM such as ChatGPT to improve the readability and comprehensibility of vaccination consent forms. Our aim was not to create a definitive tool, but rather to explore the potential of this technology to improve informed consent form processes. This preliminary attempt paves the way for further interactions and iterations with LLM, focusing on user involvement and refinement of the tool.
[ad_2]
Source link
