Research Abstract

Analysing The Accuracy of Medical Artificial Intelligence : For The Purpose Of Diagnosis, Triaging and Management Application.

NMRR ID-22-02529-WLP

Kien How Wong, Amanda Qiao Ying Yap, Nai Ming Lai, Khairul Nizam bin Hassan, Siti Alia binti Elliza, Ze Shun Lim, Nursaleha binti Mohammad Pala, Rashidah binti Bahari, Min Thein Win.

Introduction:

The use of Artificial intelligence (AI) in health care practice can be seamlessly incorporated into various stages of health care delivery. The Mediverse AI symptom checker will be made available as an advocated tool to deliver health care more efficiently. This study aimed to analyse the accuracy of the Mediverse AI symptom checker for the purpose of diagnosis, triaging, and management education. The study also aimed to assess the acceptance level of treating doctors and patients and patient-friendly use of the system. 

Method:

This study is a prospective validation study which included two components: a comparative study to evaluate human versus Mediverse AI symptom checker in the diagnosis of the medical condition, and a cross-sectional survey of doctors’ satisfaction of usage of Mediverse AI symptoms in terms of differential diagnosis, triage, and management education. Patient presenting to healthcare facilities in Hospital Putrajaya, was first assessed and treated by doctors, and subsequently was subject to the use of Mediverse AI symptom checker. Diagnosis accuracy was evaluated by comparing the AI generated diagnosis with the actual diagnosis of the treating doctor. Both patients and doctors proceed with the survey respectively.

Result:

The study involved the participation of 75 doctors and 325 patients from Hospital Putrajaya. It is found that 75.1% of the patient diagnoses generated matched those given by the doctor, with another 15.4% of the diagnoses falling within the top three diagnoses. This resulted in a cumulative match rate of 90.5%. The concordance analysis by the independent assessor showed that 78.2% of the diagnoses provided by the AI symptom checker were a complete match with another 8.9% partial match with the diagnoses provided by the physicians, resulting in an overall match rate of 87.3%. The physician analysis showed that 97.6% of the physicians found the AI symptom checker user-friendly or very user-friendly, 98.2% rated the system as acceptable or very acceptable. The patient analysis showed that 92.3% of the patients found the AI symptom checker user-friendly or very user-friendly, with 93.2% of the patients found it acceptable or very acceptable.

Conclusion:

In conclusion, Mediverse AI symptom checker showed a high level of accuracy in arriving at the diagnosis. Both doctors and patients had a high level of satisfaction towards the AI in terms of user friendliness and acceptance level, suggesting that the Mediverse AI system can be used to benefit the medical community by instituting trust towards the AI system among both medical practitioners and laymen alike.