Comparing the performance of GPT-4 Omni against specialised neural networks in identifying malignant dermatological lesions from smartphone images and structured clinical data

Description

The original structured clinical data for the 1000 test cases are stored as clinical_data_original.csv. The imputed version of the clinical dataset is stored as clinical_data_imputed.csv. The image data for all 1000 test cases are stored in the Images folder in .png formats. The JSON files uploaded to the OpenAI API endpoint are in Prompt JSON, separated by modality (clinical, img, multimodal) under the corresponding subfolders. These prompts contain the system prompt, the text prompt, and the image in Base64 format (if applicable). Because the API endpoint had a tendency to error out with larger JSON uploads, the 1000 cases are separated into batches: 10 batches for clinical data inputs (100 cases per batch), 50 batches for image data inputs (20 cases per batch), 100 batches for multimodal inputs (10 cases per batch). The raw 150000 GPT responses are available in GPT Responses, separated by modality under the corresponding subfolders. Each csv file contains responses for all 1000 cases. Each modality subfolder has 50 csv files corresponding to 50 trials each. The cleaned binary labels are available in Parsed Binary Labels, separated by modality under the corresponding subfolders. Each csv file contains a gpt_pred column flagged as either 0 (benign) or 1 (malignant), and this represents prediction by GPT-4o. Each file also contains a true_pred column which contains the ground truth from the PAD-UFES-20 dataset. Each csv file contains responses for all 1000 cases. Each modality subfolder has 50 csv files corresponding to 50 trials each.

Institutions

Categories

Funding

Licence