The number of nail fold capillaries and nail fold bleedings reflects the clinical manifestations of systemic sclerosis
Description
Participants and imaging methods Capillaroscopy videos and clinical images were obtained from SSc patients who had given consent between April 2016 and December 2018.The number of SSc patients was 71 and 550 nails were included. All patients were taking oral vasodilators. Capillaroscopy movies were taken with a Toku Capillaro-01 capillaroscope. The capillaroscope itself was fixed on a stand attached to the capillaroscope. After applying a drop of oil to the base of the patient's nail, the patient's finger was placed on the stand with the capillaroscope attached, and the capillaroscope was moved by using a lever on the stand to move the finger. The video images were saved as MPG files using the supplied software. Fingers with ulcers were not captured. From the video images, a set of images was extracted using the preview function of the Macintosh. A folder was created for each nail and the cropped images were saved in each folder. The study was approved by the Ethics Committee of the Graduate School of Medicine of the University of Tokyo (2019093NI). Written informed consent was obtained from all participants. Overlaying nail images using deep learning Since NFCs may be unevenly distributed in one nail, NFB and NFC count were evaluated on a nail-by-nail basis to eliminate errors due to the measurement site. On the other hand, images taken by capillaroscopy are at high magnification (300x), and multiple images must be connected to generate a single nail image. In order to count the number of NFCs and NFBs, we first decided to overlay the nail images. In existing reports, characteristic blood vessels were used as a marker for overlapping images. However, in this study, some images were found to have few blood vessels, so we decided to use the nail border as a marker for overlapping images. The images of 50 nails in 10 cases were annotated by a SSc specialist with supervised data on nail boundaries, NFBs, and NFCs. LabelImg was used for annotation. We used Tensorflow 1.13.2 and Keras 2.2.4 as a deep learning framework. First, we trained U-net with images of nails to enable the recognition of nail borders. Second, we draw the nail borders in images using the trained U-net and arranged the images using stitching technique so that the boundary line between the nail and the cuticle was on the arc. Third, we trained yolo-v3, which was the object detection algorithm , using images with NFC and NFB annotations, and detected NFCs and NFBs using the trained yolo-v3. The system was programmed to detect the NFBs and NFCs specifically. (NFC Count Tool and NFB Count Tool). For the remaining 61 patients and 500 nails, the NFC and NFB counts were measured using the NFC count tool and the NFB count tool after clipping from the video, and the correlation with the clinical image was examined. We took all remaining nails per patient and averaged them.