A citizen science approach to optimising computer aided detection (CAD) in mammography

Computer aided detection (CAD) systems assist medical experts during image interpretation. In mammography, CAD systems prompt suspicious regions which help medical experts to detect early signs of cancer. This is a challenging task and prompts may appear in regions that are actually normal, whilst genuine cancers may be missed. The effect prompting has on readers performance is not fully known. In order to explore the effects of prompting errors, we have created an online game (Bat Hunt), designed for non-experts, that mirrors mammographic CAD. This allows us to explore a wider parameter space. Users are required to detect bats in images of flocks of birds, with image difficulty matched to the proportions of screening mammograms in different BI-RADS density categories. Twelve prompted conditions were investigated, along with unprompted detection. On average, players achieved a sensitivity of 0.33 for unprompted detection, and sensitivities of 0.75, 0.83, and 0.92 respectively for 70%, 80%, and 90% of targets prompted, regardless of CAD specificity. False prompts distract players from finding unprompted targets if they appear in the same image. Player performance decreases when the number of false prompts increases, and increases proportionally with prompting sensitivity. Median lowest d' was for unprompted condition (1.08) and the highest for sensitivity 90% and 0.5 false prompts per image (d'=4.48).