Hope: a conversational agent based-model for pregnant health literacy

Abstract

The gestational period is a moment of great expectation and critical for pregnant women due to many uncertainties and doubts that affect the pregnant woman. The first months of pregnancy and motherhood, known as the baby’s thousand days period, include the prenatal and postnatal stages with pregnant women investigating many topics, including risk management, physical activity, nutrition, and other issues that cause uncertainty and anxiety. Conversational agents have been played a role over the years as engagement, support, and information tools in different areas in the health field for collaborative action with patients and doctors. We propose in this thesis the development, implementation, and evaluation of conversational agents based on the HoPE (Help in Obstetrician for PrEgant) architecture model, which aims to promote literacy in pregnant women through reliable information. We evaluated this model through clinical trials and experiments involving information retrieval. Studies involving clinical trials with health professionals and pregnant women are still remote and need further investigation. The strategies we have listed for managing dialogue and retrieving information are unprecedented in the scientific context, and we have not found any proposal that promotes models with similar concepts. The architecture developed has as main pillars the ability to recover and disambiguate information using ontologies and architectures based on Transformers as a center. We carried out five assessments that provided numerous insights for studies in this field. Initially, we applied a survey to get a general picture of the subject in question. In a quantitative study using semi-structured questionnaires, pregnant women and health professionals interacted with conversational agents trained in nutritional data. The results showed that both groups have positive perceptions about the experience with the conversational agent and statistically the null hypothesis was accepted (P-value = 0.713). A second evaluation with a sample formed by different pregnant women and doctors verifies through a mixed analysis that the perceptions of these groups are complementary and positive, regarding the use of conversational health agents trained in general data of the content of a thousand days in pregnancy. The new sample of pregnant women again showed a positive perception in general about the new constructs evaluated (Overall Mean = 4.0 Mean Deviation = 1.1). Also, insights generated by doctors through qualitative analysis indicated some improvements as the inclusion of COVID-19 content and family behavior, as well as adjustments in the approach and language of the conversational agent. We evaluated the pre-trained Sentence-BERT models in Portuguese, adjusted to health protocol data that we extracted from official protocols of the Brazilian Government. The BERTimbau model, trained in data augmentation strategies, obtained the highest correlation with embeddings generated by the health data corpus (Spearman:95.55) and was selected as the winning model in our experiments. Using this model, we performed the second study that evaluated the performance of the HoPE architecture for conversational agents. Three main metrics were evaluated in this study: information retrieval efficacy, architecture’s ability to identify composite intents, and architecture inference speed. For the information retrieval task, the HoPE architecture obtained an F1-Score of (0.89) under the test data, a hit score of (90%) in the identification of composite/unique intents under a set of 10 sentences, and a performance regular in the information retrieval speed (CPU=2.223, GPU=0.222). Future studies will evaluate through clinical studies the hybrid HoPE architecture for information retrieval, validation for groups of pregnant women from different demographic strata, and deepen the study on mechanisms for identifying multiple intentions in dialogues.