Automated Misinformation on the GPT-3 Platform: A Threat to Evidence-Based Medicine
Published: 31 January 2023| Version 1 | DOI: 10.17632/6bvr43zbp6.1
Contributor:
Alex Glynn
Description
Having anecdotally observed the tendency of the OpenAI API to cite non-existent articles as sources, it was hypothesized that if the API were queried repeatedly with the same prompt (specifying that all sources should be cited), a high proportion of references would be fabricated. The API was queried 50 times as to the proportion of community-acquired pneumonia cases attributable to Rhodococcus equi infection. The 50 responses cited 72 references from 32 different journals; 100% of these references were fabricated.
Files
Steps to reproduce
An R markdown document `open-ai.Rmd` is provided.
Institutions
University of Louisville School of Medicine
Categories
Medicine, Informatics