Dataset of analysis of the new Bing chatbot’s advice on help in heart attack
This dataset contains results of evaluation of quality of the new Bing (Microsoft Corporation, Redmond, Washington, USA) chatbot advice on how to give help in heart attack. On 10-12th May 2023, the chatbot was repeatedly queried in English “heart attack what to do”. Search locations included three capital cities: Banjul (the Gambia), Delhi (India), and Washington DC (the USA). For each location, the chatbot was prompted 20 times. Chatbot’s conversation style was set to “More Precise” mode. The chatbot responses were tabulated and manually evaluated by the authors for conformity with the International First Aid, Resuscitation, and Education Guidelines 2020 [IFRC, 2020] using a checklist predeveloped by the authors. The responses’ content was judged in terms of satisfaction of each checklist item wording as True (completely satisfied), Partially True (satisfied in part), or Not True (not satisfied/absent). Additionally, the chatbot responses were evaluated for length (number of sentences), and readability based on the Flesch-Kincaid Grade Level [Kincaid et al., 1975]. Webpages referenced by the chatbot as a source for generating its responses were classified as Academic, Charity/Non-profit entity, Commercial, Government, and Social media, and content of the source webpages was evaluated for quality using the same checklist with a focus on source article text. References 1. International Federation of Red Cross and Red Crescent Societies. International first aid, resuscitation, and education guidelines 2020. https://www.ifrc.org/sites/default/files/2022-02/EN_GFARC_GUIDELINES_2020.pdf. Accessed August 02, 2023. 2. Kincaid JP, Fishburne Jr RP, Rogers RL, Chissom BS. Derivation of new readability formulas (automated readability index, fog count and flesch reading ease formula) for navy enlisted personnel. Naval Technical Training Command Millington TN Research Branch; 1975.