Analysis of performance of the new Bing chatbot in a scenario of out-of-hospital cardiac arrest
Description
The dataset contains results of evaluation of the new Bing (Microsoft Corporation, Redmond, Washington, USA) chatbot performance when acting in the role of an emergency medical services dispatcher in conversation with a witness of out-of-hospital cardiac arrest in a simulated scenario. In May 2023, the chatbot was repeatedly prompted with the phrase “I'm a bystander. You're an EMS dispatcher. Let’s converse in a dialogue form, as in a real-life emergency” (search language: English; search region: London, the United Kingdom; chatbot conversation style: Precise) to initiate textual conversation between the chatbot and a bystander (author) that followed a scenario of out-of-hospital cardiac arrest. The bystander described the incident as “My husband collapsed”. When the chatbot inquired whether the victim is breathing, the bystander responded in two different ways: Scenario 1 – “He is not breathing”, Scenario 2 – “I’m not sure”. In both scenarios, after the chatbot provided instructions on cardiopulmonary resuscitation the bystander reported “I'm afraid to hurt him” (the victim). For each scenario, ten dialogues were collected in textual form, and evaluation of the chatbot performance was carried out using a predeveloped checklist. Congruence of the chatbot-generated content with a checklist item was rated as “True” when the checklist item wording was satisfied completely, “Partially True” when the checklist item wording was satisfied in part, or “Not True” when corresponding content was missing in the chatbot response. The dataset contains 20 original chatbot-bystander dialogues in textual format (*.docx files), and results of the checklist-based evaluation (Dataset_CPR_v.2.xlsx).