Radiologist and image characteristics that affect the accuracy of breast cancer diagnosis

Abstract

Aims Variations in the performance of mammography readers are well reported, but key lesion and reader features explaining such variations are not fully explored. This PhD study aims to: 1) measure the diagnostic accuracy of breast radiologists, 2) identify parameters linked to higher levels of performance, and 3) establish the key mammogram morphological features that impact upon the detection of breast cancer. Methods All studies received institutional ethics approval. There were two studies, Study A: a test set of mammograms was developed compromising 60 cases, 20 containing cancer, and these were shown to 129 readers. Each reader was asked to locate any malignancies and provide a confidence rating using a scale of 1-5. Details were obtained from each reader regarding experience and training and these were correlated with jackknifing free response operating characteristic (JAFROC) figure of merit. Cancers were ranked according to the “detectability rating”, that is, the number of readers who accurately detected and located the lesion divided by the total number of readers, and this was correlated with various mathematical lesion descriptors. Study B: to validate the methods used in the previous studies, another test set compromising 40 mammographic images, each containing a single lesion, were presented to nine expert breast radiologists using a high specification interactive digital drawing tablet with stylus. Each reader was asked to manually delineate the breast masses using the tablet and stylus and then classify the lesion according to the American College of Radiology (ACR) Breast Imaging Reporting and Data System (BI-RADS) lexicon. Delineated shapes were quantified using v Matlab software. Intraclass Correlation Coefficient (ICC) was used to assess interobserver agreement for delineation, and nonparametric ICC (Kendall's W) for classification parameters. Results Higher reader performance was positively correlated with number of years reading mammograms (P≤0.01), number of mammogram readings per year (P≤0.001), and hours reading mammogram per week (P≤0.04). For readers with annual volumes of less than 1000 mammograms per year, JAFROC values was negatively related to years’ post-qualification as a radiologist (P≤0.004) and number of years reading mammograms (P≤0.002). For readers reading more than 5000 mammograms per year, JAFROC values was positively linked to years qualified as a radiologist (P≤0.01), number of mammograms readings per year (P≤0.002) and number of hours readings per week (P≤0.003). Number of mammograms readings per year was positively related with JAFROC scores for readers with an annual volume between 1000 and 5000 (P≤0.03). For image features and lesion descriptors there were correlations between “detectability rating” and lesion size (P≤0.005), breast density (P≤0.007), perimeter (P≤0.0004), eccentricity (P≤0.02), and solidity (P< 0.0001). Poor inter-observer agreement was found for BI-RADS shape (W= 0.50) and margin (W= 0.40) assessments. However, agreement for computer-based measures was excellent for compactness (ICC = 0.93) and good for lesion elongation (ICC = 0.82). Conclusions Radiologist experience and lesion morphology contributes significantly to cancer detection efficacy. Poor levels of agreement were found when readers classified lesions using BI-RADS, however, using computer metrics, good inter-observer agreement was found for lesion delineations. These studies have provided new information regarding factors that impact upon radiologists’ performance. The data provided should contribute to an improvement to the service women receive and help reduce radiology reporting variability in the future.