Visual Attention Distribution Based Assessment of User's Skill in Electronic Medical Record Navigation

Currently, the most precise way of reflecting the skills level is an expert's subjective assessment. In this paper we investigate the possibility of the use of eye tracking data for scalar quantitative and objective assessment of medical staff competency in EMR system navigation. According to the experiment conducted by Yarbus the observation process of particular features is associated with thinking. Moreover, eye tracking is proved to clarify the relationships between eye movements, visual attention and insight to the complex task performance. Using the similarities between the reference attention map and the one recorded for the assessed person, as well as the total time of the performance, we have proposed the Visual Attention Time Product (VATP) to distinguish the level of knowledge and familiarity with EMR systems. We used Spearman Rank to compare the obtained VATP scores with the 5-point scale expert's assessment about the test participants' performance. The VATP was also compared with the other methods used for classification of the competency test results (attention, similarities and time). Twenty-five people with completed basic training in the EMR usage participated in the proposed experiment. The proposed VATP, calculated with regards to the reference performance of the exemplary expert, shows very strong correlation (0, 92) with the expert's assessment of the observed performance in solving EMR related tasks. The VATP was also calculated regarding the average performance of all experts who participated in the experiment. Again the results show strong correlation (0, 90). Moreover, the results show that the eye tracking interface does not influence on the EMR navigation. We demonstrate that the eye tracking data and proposed VATP can be successfully used to differentiate the level of performance in EMR system related tasks. It also appeared that the parameter VATP enables the assessment of the examinee comparable to that performed by expert. The level of complexity of the computation allows the gathering of results directly after the conducted test. Thus, the VATP can be successfully used for EMR end user skills differentiation and competency testing.