The Research problem
While we are witnessing technological progress in AI enabled healthcare applications, there is increasing concern about ethics and human-values. Generative AI relies on LLMs to provide solutions to information needs of both professional and lay users. This technology is still evolving and the accuracy and reliability of its output is highly dependent on the quality of the datasets used to train them. Errors in AI-generated information can put the user’s health at risk, making it imperative that this technology be evaluated so that healthcare providers and patients have trust in it. Another concern is bias in the algorithms due to datasets not being fully representative of a population. It is essential to address these challenges to ensure the technology’s ethical use, improves healthcare outcomes and benefits patients. We seek to achieve a better understanding of the potential and risks of generative AI in child and maternal health, regarding effective prenatal care.
Research Design
Our proposal investigates generative AI potential and risks in maternal health from the perspective of responsible AI and societal concerns regarding AI enabled technologies. These have to do with dimensions not yet explored and evaluated, such as factuality, safety, fairness, transparency, accountability, inclusiveness, and reliability. We will assess the information output of LLMs in a Question-Answer system in the context of maternal health and prenatal care. To that end, we will run a series of experiments using a dataset which is part of an app content – meu Pré-Natal. The dataset consists of a set of 98 questions about maternal health created and curated by medical specialists and made available within the app functionality “Learn More”. The content is scientifically-based and provided in Portuguese, English, and Spanish with language accessible to the general public. This set of questions will be used as a gold-standard to assess the information output of LLMs to obtain answers with different prompts. We will first automatically evaluate the responses generated by LLMs using metrics for text editing distance. Subsequently, we will ask local experts to assess LLM’s answers through questionnaires specifically developed for this purpose.
Project Objectives
We seek to evaluate the quality of information provided by generative AI in the context of maternal health and prenatal care. Through a set of experiments, we want to achieve a better understanding of the information potential and communication risks of AI in supplying accurate information on maternal health to lay users in settings with limited healthcare access. The results will subsidise discussions of the role of AI in the development of ethical and responsible solutions to improve diagnosis, prognosis, and provision of medical treatment, in addition to supporting prevention and health promotion programs and, ultimately, promoting the evolution of both professionals and the systems they use.
