Automated Misinformation on the GPT-3 Platform: A Threat to Evidence-Based Medicine

Published: 31 January 2023| Version 1 | DOI: 10.17632/6bvr43zbp6.1
Alex Glynn


Having anecdotally observed the tendency of the OpenAI API to cite non-existent articles as sources, it was hypothesized that if the API were queried repeatedly with the same prompt (specifying that all sources should be cited), a high proportion of references would be fabricated. The API was queried 50 times as to the proportion of community-acquired pneumonia cases attributable to Rhodococcus equi infection. The 50 responses cited 72 references from 32 different journals; 100% of these references were fabricated.


Steps to reproduce

An R markdown document `open-ai.Rmd` is provided.


University of Louisville School of Medicine


Medicine, Informatics