01 Dec Machine Learning Program Superior to Humans in Non-Pigmented Skin Lesions
MedicalResearch.com Interview with:
Philipp Tschandl, MD PhD, Priv.Doz.
Assistant Professor
Department of Dermatology
Medical University of Vienna
MedicalResearch.com: What is the background for this study? What are the main findings?
Response: Dermatoscopy is a non-invasive imaging technique, where the surface of the skin is rendered translucent and additional important morphologic features become visible from deeper layers. This is achieved through use of immersion fluid or cross-polarised light – equivalent to the effect when using a pair of goggles to look underwater, or polarised sunglasses to reduce glare on glass surfaces. After the first description of “Dermatoskopie” almost 100 years ago by a German dermatologist (Johann Saphier), this technique has evolved to a successful, low-cost, state-of-the-art technique for clinical skin cancer detection in the last decades.
Convolutional neural networks (“CNN”) are powerful machine learning methods, and frequently applied to medical image data in the recent scientific literature. They are highly accurate for basic image classification tasks in experimental settings, and found to be as good as dermatologists in melanoma recognition on clinical or dermatoscopic images. In this study we trained a CNN to diagnose non-pigmented skin lesions (where melanomas are only a minority) through analysis of digital images, and compared the accuracy to >90 human readers including 62 board-certified dermatologists. This study expands knowledge in the following ways compared to previous work:
– We applied the network for the detection of non-pigmented skin cancer, which is far more common in the (caucasian) population than melanoma.
– We created a prediction model that combines analysis of a dermatoscopic and clinical image (“cCNN”) which is able to further increase diagnostic accuracy.
– We compared accuracy not only to experts, but users with different level of experience
MedicalResearch.com: What should readers take away from your report? How difficult/expensive with the new technology be to implement?
Response: We now measured superiority of CNNs to average human raters for classification of skin cancer, so the question arises whether we not only can, but have the duty to try to implement such methods in clinical practice. With todays’ infrastructure it is technically easy to provide CNN predictions on smartphones, computers, or over simple websites – at low infrastructure cost. As an example, we have set up a free website (http://ypsono.com) where one can use a neural network to search for similar images in public datasets – at a negligible total running cost for us of ~10$ per month (this does of course not include costs of creating the dataset and networks, as well as image hosting – and therefore is not comparable to estimations of a commercial product).
One should not be lured by technical feasibility though, as there are numerous problems with actual implementation of an “artificial intelligence classifier” (the ones listed below are not exhaustive and focus on the recent study):
– Included cases have a selection bias, i.e. they were deemed suspicious enough by a physician at one point to make a photograph, and commonly to make a biopsy. Therefore, more common inconspicuous lesions are most probably missing. This problem is even more striking keeping in mind that the cCNN performed much worse on benign disease classes, and for the ones not present in the training set it – naturally – wasn’t able to make such diagnoses at all. Therefore, such a system may perform much worse in a low-prevalance setting and cause a myriad of false positives.
– Image quality may be different in practice, where as a result performance could be decreased. Images from the study came from skin cancer practices who have developed standardised imaging methods, whereas practitioners may have different equipment.
– Superior diagnostic accuracy in regard to “malignancy” detection does not necessarily mean better specific diagnoses or management decisions which are more important in clinical practice (https://jamanetwork.com/journals/jama/fullarticle/2681495).
MedicalResearch.com: What recommendations do you have for future research as a result of this work?
Response: Like previous studies, we have limited comparison with human readers to a single “best-performing” network. What is missing is a large-scale analysis of available machine-learning models, ideally using public datasets, which would cover general state of the field. This is something we try to tackle with the “HAM10000”-dataset (https://www.nature.com/articles/sdata2018161) and -study including a multitude of CNN models from the ISIC 2018 challenge (https://challenge2018.isic-archive.com/).
Recent, and older (https://doi.org/10.1097/CMR.0b013e32832a1e41), work shows decreased CNN classification accuracy in an actual “real-life” setting compared to experimental counterparts, so we need to find ways to measure – and maybe simulate – these settings to provide more meaningful metrics before commencing more expensive clinical studies. But of course, also research on actual clinical application of CNN classifiers is needed, and almost entirely missing in the dermatologic field. What output of CNNs do we show, or not show? How are the right physicians for the right lesions influenced in the right direction? Would classifiers do harm on a population scale? There are many open questions, and for sure surprising answers: As an example, in an early, and to my knowledge the only, prospective clinical trial on neural-network based automated melanoma detection (https://doi.org/10.1097/CMR.0b013e32832a1e41) three melanomas were missed by the system – because they were never photographed by the study physician!
There are no additional disclosures to what is present in the article.
Citation:
[wysija_form id=”3″]
[last-modified]
The information on MedicalResearch.com is provided for educational purposes only, and is in no way intended to diagnose, cure, or treat any medical or other condition. Always seek the advice of your physician or other qualified health and ask your doctor any questions you may have regarding a medical condition. In addition to all other limitations and disclaimers in this agreement, service provider and its third party providers disclaim any liability or loss in connection with the content provided on this website.
Last Updated on October 10, 2020 by Marie Benz MD FAAD