Variants of knowledge-based chatbots in family caregiving – Human evaluation and comparison
I. Nowak, D. Zühlke, N. Nikzad-Khasmakhi, D. A. Kring, T. Grüb-Okkan, J. Steinborn, I. Zorn
Full text PDF 
( Download count: 65)
AbstractIn Germany, 82% of people in need of care are cared for at home by informal carers. 80% of informal carers have no previous care-related training and 57% are over 60 years old (BARMER 2018). People in need of care as well as their caregivers need reliable information regarding their rights, support structures, or potential ressources for enhancing life with care. Limited accessibility of professional care advice services further exacerbates the need for reliable information resources. Family care is characterized by its complexity and the variety of conditions and specific requirements. Informal carers are ever more in need of easy to access information about coping in life with care for the elderly and are not content with existing care counseling (Bohnet-Joschko/Bidenko 2019). To deliver such information existing knowledge from a variety of sources must be classified in terms of its reliability, credibility and seriousness (Hoffmann et al. 2021). Older people in particular show deficits in digital research skills when it comes to locating, evaluating and using online information (Initiative D21 2021). Against this background, a knowledge-integrated chatbot based on a Large Language Model (LLM) is being developed to provide reliable information on legal entitlements, benefit entitlements and their application procedures. Given the increasing prevalence of LLMs and their application in sensitive areas such as family care, we present results of a comprehensive human evaluation of the effectiveness of different methods of knowledge integration into chatbot technologies. The aim is to increase the accuracy of the information provided and minimize so-called hallucinations, i.e. factual inaccuracies.Keywords: large language models, hallucination, family caregiving, chatbot, response accuracy
I. Nowak, D. Zühlke, N. Nikzad-Khasmakhi, D. A. Kring, T. Grüb-Okkan, J. Steinborn, I. Zorn (2024). Variants of knowledge-based chatbots in family caregiving – Human evaluation and comparison. Gerontechnology, 23(2), 1-1
https://doi.org/10.4017/gt.2024.23.s.1080.opp