1 AAPM meeting 2001, Salt Lake City, Utah CE: Mammography 4: Update on Computer-Aided Diagnosis in Mammography Handout* Maryellen L. Giger, Ph.D. Professor of Radiology Department of Radiology The University of Chicago MC2026 5841 S. Maryland Ave. Chicago, Illinois 60637 Phone: 773-702-6778 Fax: 773-702-0371 m-giger@uchicago.edu * Excerpted and modified from: Giger ML: "Computer-aided diagnosis". In: AAPM/RSNA Categorical Course in Diagnostic Radiology Physics: Physical Aspects of Breast Imaging -- Current and Future Considerations, (Haus A. and Yaffe M., eds.) pp. 249-272, 1999. I. INTRODUCTION Computer-aided diagnosis (CAD) can be defined as a diagnosis made by a radiologist who uses the output from a computerized analysis of radiographic images as a "second opinion" in detecting lesions and in making diagnostic decisions. The final diagnosis is made by the radiologist. Although mammography is currently the best method for the detection of breast cancer, between 10-30% of women who have breast cancer and undergo mammography have negative 2 mammograms (1-6). In approximately two-thirds of these false-negative mammograms, the radiologist failed to detect the cancer that was evident retrospectively (4,5,7,8). The missed detections may be due to the subtle nature of the radiographic findings (i.e., low conspicuity of the lesion), poor image quality, eye fatigue or oversight by the radiologists. In addition, it has been suggested that double reading (by two radiologists) may increase sensitivity (9-13). Thus, one aim of CAD is to increase the efficiency and effectiveness of screening procedures by using a computer system, as a "second reader" (like a “spell checker”), to aid the radiologist by indicating locations of suspicious abnormalities in mammograms, leaving the final decision regarding the likelihood of the presence of a cancer and patient management to the radiologist (14). Since mammography is becoming a high volume x-ray procedure routinely interpreted by radiologists and since radiologists do not detect all cancers that are visible on images in retrospect, it is expected that the efficiency and effectiveness of screening could be increased by CAD. The interpretation of screening mammograms lends itself to CAD since it is a repetitive task involving mostly normal images. If a suspicious region is detected by a radiologist, he or she must then visually extract various radiographic characteristics. Using these features, the radiologist then decides if the abnormality is likely to be malignant or benign, and what course of action should be recommended (i.e., return to screening, return for follow-up or return for biopsy). Many patients are referred for surgical biopsy on the basis of a radiographically detected mass lesion or cluster of microcalcifications. Although general rules for the differentiation between benign and malignant breast lesions exist (1,15), considerable variabillity occurs in the interpretation of lesions by radiologists with current radiographic techniques (6,16). On average, only 10-20% of masses referred for surgical breast biopsy are actually malignant (1,17-20). Thus, another aim of CAD is to extract and analyze the characteristics of benign and malignant lesions seen on mammography in an objective manner in order to aid the radiologist by increasing diagnostic 3 accuracy and reducing the numbers of false-positive diagnoses of malignancies; thereby decreasing patient morbidity as well as the number of surgical biopsies performed and their associated complications. It is hoped that CAD will improve the reproducibility of mammographic interpretation. Computerized classification methods are also being extended to ultrasound and magnetic resonance images of the breast. The presentations of the various detection and classification methods vary depending on the details of the specific methodology that are available and on the database employed with which to evaluate the computerized approach. It is important to note that a comparison of different computerized methods is not possible due to the use of different databases. That is, it cannot be assumed that a computerized scheme that achieves a high sensitivity with one database of mammograms will achieve a similar performance level with another database or with an actual patient population. In addition to the database, the method of evaluation will influence performance and expectations of a specific computerized scheme. Some of the computerized schemes have been tested, or merely demonstrated, only with a few images; whereas others have used moderate-size databases and statistical testing methods. Performance levels, in the latter situations, are usually given for the detection schemes in terms of the sensitivity (true-positive rate) for detection and the number of false-positive detections per image; and for the classification schemes in terms of the sensitivity for classification and the specificity (i.e., one minus the false-positive fraction). Such issues will be discussed later. II. COMPUTER VISION AND ARTIFICIAL INTELLIGENCE Computer vision involves having a computer extract from a digital image features that may or may not be otherwise perceived by a human. The development of computer vision schemes requires a priori information about the medical image (e.g., the mammogram) and knowledge of various computer processing techniques and decision analysis methods. The 4 required a priori knowledge includes the physical imaging properties of the digital image acquisition system and morphological information concerning the abnormality (e.g., mass lesion or cluster of microcalcifications) along with its associated anatomic background. That is, a sufficient database is needed in order to cover the entire range of abnormals and normals. Computer vision techniques include, in general, image processing, image segmentation and feature extraction (21,22). Once the mammographic image is in digital format, the data are accessible for computer manipulations. Image processing techniques can be employed to enhance features for ease of extraction later or to de-emphasize unwanted features such as background structures. Segmentation of an image involves the separation of the image into regions of similar attributes. Basically, a segmentor only subdivides an image and does not try to recognize the individual segments. Feature extraction entails the recognition of specific characteristics of the individual segments. Once features can be extracted, relevant ones need to be selected and optimally merged by a classifier. Artificial intelligence techniques include feature selection methods such as stepwise selection and genetic algorithms, and classifiers such as discriminant analysis techniques, rule-based expert systems, and artificial neural networks. Such techniques are used to merge computer-extracted features and/or radiologist-extracted features into diagnostic decision aids. Currently, screen/film combinations are used as the image recording system for mammographic imaging with investigators in CAD using digitized mammograms almost exclusively. Developments in full-field digital mammography (FFDM) (29), however, have approached clinical usage and thus, yielding the opportunity for CAD on FFDM. Digital image acquisition (film digitization or FFDM) can vary in terms of pixel size and number of quantization levels. In general, when developing a detection scheme, it is important to consider the size of the lesion of interest when choosing pixel size. For example, the detection of 5 microcalcifications requires a much smaller pixel size (probably around 0.1 mm) than does the task of detecting mass lesions. In addition, the classification of a lesion in question (to be discussed later) may require higher spatial resolution (finer digitization) than that needed for detection. Chan et al. (40,41) have examined the effect of various system parameters, such as pixel size, quantization and data compression, on the detection performance of computerized methods. They demonstrated the trade-off between detection accuracy and data compression rate or digitization resolution. The availability of high-speed computers and high-resolution film digitizers has been instrumental in the development of advanced computerized detection schemes. It is assumed that over 100 research groups in either academia or industry are developing computerized schemes for the detection of masses and clustered microcalcifications in digital mammograms. Many involve the use of classifiers to distinguish between actual lesions and false-positive detections, or between malignant and benign lesions. Examples of these classifiers in CAD schemes include rule-based methods (48,90), discriminant analysis (69), Baysian methods (68, 85), artificial neural networks (55,70,71), and fuzzy logic (76). The features for input to these classifiers are selected using a variety of techniques including step-wise methods (130) and genetic algorithms (56, 84, 130). Researchers are investigating two types of computer-extracted features: radiologist-used features (e.g., see 124, 126) and higher-order features that may not be as intuitive to radiologists (e.g., see 129). III. COMPUTER-AIDED DETECTION The computerized detection of lesions involves having the computer locate suspicious regions, leaving the classification of the lesion totally to the radiologist. It has been reported that between 30 to 50% of breast carcinomas detected mammographically demonstrate clustered microcalcifications (23-25), although about 80% of breast carcinomas reveal microcalcifications 6 upon microscopic examination (26-29). In addition, studies indicate that 26% of nonpalpable cancers present mammographically as a mass while 18% present both with a mass and microcalcifications (25). Thus, most computerized schemes in mammography are being developed for the detection of mass lesions (77-100) or clustered microcalcifications (42-76, 164). In order to limit the region of search for lesion detection, the breast region may to be initially segmented from the image. Bick, et. al (30) and Mendez et al. (31), have described methods using computer-defined unexposed and direct-exposure image regions to generate a border around the breast region. Once the breast region is known, preprocessing and search can be contained to within the breast border to yield potential lesion sites. Some methods incorporate an equalization of the gray levels near the periphery of the breast in order to correct for the increased density due to the reduced breast thickness (32,33) Most computer detection methods can be described by a preprocessing stage during which a threshold is applied on each pixel based on a feature such as gray level (34), gradient (90), line orientation (35) or a combination of features (36,37) (e.g., from a feature image). Remaining pixels are then grouped yielding lesion candidates from which various features can be extracted and merged in an attempt to reduce false-positive detections. Various details of developing computer detection methods can be found in proceedings of the International Workshops on Digital Mammography (101-103). IV. COMPUTER-AIDED DIAGNOSIS Once a lesion is detected, a radiologist must make a decision concerning patient management -- that is, should the patient return to screening, have a follow-up or be sent for biopsy? This differs from a purely benign versus malignant determination. 7 Computerized analysis schemes are being developed to aid in distinguishing between malignant and benign lesions in order to improve both sensitivity and specificity. Such methods may use features extracted either by computer or by radiologists. The benefit of computerextracted features is the objectivity and reproducibility of the measure of the specific feature. However, radiologists employ many radiographic image features, which they seem to extract and interpret simultaneously and instantaneously. Thus, the development of methods using computer-extracted features requires, besides the determination of which individual features are clinically significant, the computerized means for the extraction of each such feature. It is important to restate that one of the aims of computerized classification is to reduce the number of benign cases sent for biopsy. Such a reduction will be clinically acceptable only if it does not result in unbiopsied malignant cases, however, since the "cost" of a missed cancer is much greater than misclassification of a benign case. Thus, computer classification schemes should be developed to improve specificity but not at the loss of sensitivity. Various computerized classification systems are being developed using human-extracted features (104-116) and computer-extracted features (117-130). The benefit of using computerextracted features is the objectivity and reproducibility of the result. However, radiologists employ many radiographic image features, which they seem to extract and interpret simultaneously and instantaneously. The development of methods using computer-extracted features requires initial determination of which individual features are clinically significant, prior to devising means for the extraction of each such feature. During the past ten years, large growth has been seen in the development of computerized classification methods for lesions on breast images. Most methods are evaluated in terms of their ability to distinguish between malignant and benign lesions, however, a few have been evaluated in terms of patient management (i.e., return to screening vs. biopsy). 8 Ultrasound and magnetic resonance images of the breasts are also being subjected to computerized analysis to help distinguish between cancerous and non-cancerous lesions (131136 and 138-140, espectively). Breast sonography is used as an important adjunct to diagnostic mammography and is typically performed to evaluate palpable and mammographically identified masses in order to determine their cystic vs. solid nature. The accuracy of ultrasound has been reported to be 96-100% in the diagnosis of simple benign cysts. Masses so characterized do not require further evaluation. Breast sonography, however, has not been routinely used to distinguish benign from malignant solid masses because of the considerable overlap in their sonographic appearances. Magnetic resonance imaging (MRI) may aid in the diagnosis of breast cancer as a complementary technique to mammography. MRI has the benefit of yielding threedimensional information of the breast. Contrast-enhanced MRI of the breast is used to indicate differences between lesions and normal tissue due to the increased vascularity and capillary permeability of tumors. However, some benign lesions are also enhanced with contrast-enhanced MRI, and thus, the specificity of the technique is limited. Mussurakis et al. (137) reported that significant variability exists in the assessment of lesions by humans using MR images. With MRI, both temporal and spatial features of lesions may be useful in the discrimination task. V. COMPUTERIZED BREAST CANCER RISK ASSESSMENT Breast cancer risk assessment is expected to aid in appropriate surveillance plans that may include enhanced screening for women at increased risk of breast cancer. Computerized methods are being developed to relate characteristics of mammographic parenchymal patterns to breast cancer risk. Mammographic density has benn shown to be strongly related to risk of breast cancer (93). Computerized analysis of mammographic parenchymal patterns may provide an objective and quantitative characterization and classification of these patterns (141-148). 9 VI. CLINICAL EXPERIENCES AND COMMERCIAL SYSTEMS The ultimate evaluation of CAD is not how well the computerized method performs in its task, but rather how well the human performs when the computer output is used as an aid. Observer studies have shown that radiologists' performance increased when using a computer output as an aid -- in detection (36,45) and in diagnosis (127). It is important to note though that the type of observer may affect the reported effect of CAD. One needs to determine and report the amount of experience of the observers and their current mammography reading load. In addition, it is important to consider the effect of disease prevalence on the observer study. For example, in screening mammography, typically less than 1% of the cases will result in cancer. One method of evaluating a CAD method is to determine its performance on a database of missed lesions. Schmidt et al. (158) conducted a retrospective analysis using computer analyses of a series of mammographic lesions missed in routine clinical practice. With a database of 69 missed cases, the computer found 50% of the lesions with a false positive detection rate of approximately 1.3 per image total for both microcalcification cluster and mass detection algorithms (with a range of 0 to 11 false detections per image ). Similar.results from a missed lesion study were found by te Brake et al. (159). In their study, the routine clinical examination had included double reading. Twenty-two of 65 cases were detected at one false positive finding per image. It should be noted that the use of CAD does not require a "digital" radiology department. Most likely, primary diagnosis will still be made from original mammographic films for many years. The design of a dedicated CAD workstation would incorporate detection schemes for both clustered microcalcifications and masses. Appropriate man-machine interfaces must be developed for the effective and efficient implementation of these CAD schemes. The basic hardware of the workstation could include a laser film digitizer (or an interface to a digital mammographic system), a high-speed computer, a mechanism for CAD implementation and an 10 optional link to PACS (picture and archival communications systems). Possible mechanisms for CAD reporting include (1) low-resolution image paper printer or monitor with primary diagnosis from original films, (2) high-resolution film printer with primary diagnosis from original films (or possibly the printed film), or (3) multiple high-resolution monitors with primary diagnosis from the CRT screen. The computer output may be presented to the radiologists either by symbols (arrows, circles, etc.) that may be toggled on and off, or by simple text indicating the location of possible lesions in the mammogram. Radiologists would use the computer-reported results as aids in making their final diagnostic decision. In the clinical setting, CAD methods might be used as a test invoked by the radiologist upon viewing a particular mammographic case or as a routine screening procedure performed on all examinations. The workstation could be configured to allow the radiologist to control the sensitivity and specificity of the computer output. Obviously, a choice of fewer false-positive lesions would be achieved at the cost of a lower number of true-positive lesions, and vice-versa. This trade-off could be adjusted by the radiologist, depending on the nature of the case material and personal preference. For example, a radiologist might choose an output with high sensitivity for examining high-risk patients being screened for cancer, whereas a lower sensitivity and correspondingly lower false-positive rate might be desired for patients at low risk for cancer. Since November 1994, researchers at The University of Chicago have been analyzing screening mammograms on an "intelligent" workstation for use as a "second opinion" (160,161). This large-scale evaluation of computer-aided diagnosis in mammography in a clinical environment is being conducted with the CAD code “frozen” since 1994, and to date, over 19,000 mammographic screening cases have been analyzed by the computer system. The four standard screening mammograms for a given case are digitized and analyzed by the system. In a prospective study of 10,000 consecutive screening mammograms (using the 1994 version of the detection methods), the computer indicated cancer about one year before it was clinically 11 diagnosed in approximately 15% of all cancer cases and in 56% of cases where the cancer was visible in retrospect on a clinically negative-read screening mammogram (161). The computer had a false-positive rate of approximately 1.3 false clusters per image and 2.1 false masses per image. It should be noted that on a subset of cases in which the radiologist used the computer output , no change in call back rate occurred. Hendricks (162) of the University of Nijmegen compared use of the ImageChecker system (R2 Technology, Inc., Los Altos, California) with double reading with two radiologists. They reported no difference between double reading and CAD. Sittek and Reiser (38) reported on their initial experience with the ImageChecker system at the University of Munich in which 82% of the malignant lesions were detected by the computer. They found that CAD analysis can be implemented into an existing mammography section without disturbing patient handling. They also noted that use of the system prolonged the patient waiting time by approximately 15 minutes, but felt the extra time was “well worth it”. Currently, due to the false positive rate of the ImageChecker, the investigators reported that the computer marks need to be rechecked by an experienced radiologist. They do expect, however, that use of such a CAD system will increase detection for lesser-experienced radiologists. A large study (163, 165) involving a retrospective analysis of 1083 consecutive cancer cases with 13 institutions and more than 24 radiologists was performed as part of a FDA approval process for the ImageChecker (R2 Technology, Inc., Los Altos, California). The sensitivity for microcalcification cluster detection was 98.3% and that for mass detection was 72% with an average false positive rate (cluster and mass combined) of one per image. A prospective component of their study included an analysis of 14,817 cases with the CAD system. No statistically significant change was observed in the radiologists’ workup rate when the system was used as a screening tool. 12 Results from these various “clinical” studies are quite encouraging, thus leading to continued research in CAD and the arrival of more commercial systems (e.g., SecondLook from CADx and Scanis System from Trex Medical), which may seek FDA approval. VI. DISCUSSION and SUMMARY Overall CAD systems have been shown to 1) detect lesions missed by radiologists, 2) not affect the recall rate in screening mammography, 3) exhibit sufficient robustness to changes in image acquisiton and digitization, 4) integrate well into pre-existing mammography sections, and 5) classify lesions similar to expert mammographers. As further improvements in the performance of computer methods are obtained, further testing of systems in the clinical environment needs to be done. Due to the success of computerized classification in observer performance studies, plans should include the integration of detection and classification methods into more clinical prototypes. In addition, with the advent of direct digital mammography systems, CAD will be directly incorporated into systems and tested. Investigators should also take advantage of the digital image data from mammographic, sonographic, and MR images of the breast, thus, allowing for the integration of image information from multimodalities. For the present, it is important to appropriately report CAD results so that they are useful to the research community. When presenting performance results of a computerized image analysis method, the following should be clearly described: (a) acquisition of the digital image and image quality, (b) characteristics of the database including method of verifying diagnostic truth, (c) use of the database(s) with respect to development and testing, (d) method of scoring, (e) type of performance measures, and (f) performance as an aid to observers. The ultimate test of any computerized analysis scheme will be its ability to actually improve radiologists' accuracy when used as an aid. Since, in CAD, the computer is used as a 13 second opinion and not alone, it may not need to be perfect. It should be noted that a CAD scheme will be useful even at a less-than-perfect sensitivity, especially if the mammographic lesions detected by the computer do not overlap completely with those detected by a radiologist. However, a high sensitivity, at the likely cost of some acceptable number of false positives, is preferred since false negatives in the screening for breast cancer are significantly worse in terms of clinical outcome than are false-positive diagnoses. It is important to note that currently computer analysis schemes are being used as “second readers” and not as primary readers. Over thirty years after Winsberg et al. (77) first reported on a computerized method for mammograms, CAD has become a recognized and accepted component of future breast imaging as evidenced by its clear presence at the annual meetings of AAPM and RSNA, and by the fact that a commercial CAD system for mammography has had FDA approval for three years and is in routine clinical use. A systematic and gradual introduction of CAD into radiology departments is necessary so that minimum disruptions occur in current reading habits, thereby insuring the acceptability of CAD and optimal diagnostic performance by the radiologist. The information technology revolution is now changing methods of interpretation of mammographic images and these changes are being enthusiastically embraced by both the medical profession and the public (patient). This progress can be attributed to a variety of reasons including the following: • Digital image quality has improved considerably. • Computers are faster, have sufficient storage capabilities, and are able to efficiently accommodate large images over networks. • Investigators have realized that the development of robust methods requires large databases of clinical images, which have become more feasible to collect. • There exists more focussed attention to current and new computer vision techniques. 14 • Radiologists/clinicians have recognized the medical necessity of computerized assistance especially in settings such as screening for cancer. • Investigators are sensitive to the constraints imposed by the end user, e.g., the radiologist. • Public awareness of computer applications in medical imaging has increased. This is just the beginning. In the future, potentially all medical images will have some form of computer analysis performed on them in order to benefit the diagnosis. ACKNOWLEDGEMENTS The author is grateful to various members of the Kurt Rossmann Laboratories for Radiologic Image Research for useful discussions. This research was supported in parts by USPHS grants CA60187, T32 CA09649, and RR11459, and U.S. Army Medical Research and Materiel Command (DAMD 17-96-6058, 17-97-1-7202, 17-98-1-8194). M. Giger is a shareholder in R2 Technoloy, Inc. (Los Altos, CA). It is the University of Chicago Conflict of Interest Policy that investigators disclose publicly actual or potential significant financial interests which would reasonably appear to be affected by the research activities. REFERENCES 1. Bassett LW, Gold RH: Breast cancer detection: Mammography and other methods in breast imaging, Grune and Stratton (New York), 1987. 2. Baines CJ, Miller AB, Wall C, McFarlane DV, et al.: Sensitivity and specificity of first screen mammography in the Canadian National Breast Screening Study: A preliminary report from five centers. Radiology 160: 295-298, 1986. 3. Pollei SR, Mettler FA, Bartow SA, Moradian G, Moskowitz M: Occult breast cancer: Prevalence and radiographic detectability. Radiology 163: 459-462, 1987. 15 4. Andersson I: What can we learn from interval carcinomas? Recent Results in Cancer Research 90: 161-163, 1984. 5. Martin JE, Moskowitz M, Milbrath JR: Breast cancers missed by mammography. AJR 132: 737, 1979. 6. Elmore JG, Wells CK, Lee CH, Howard DH, Feinstein AR: Variability in radiologists’ interpretations of mammograms. New England Journal of Medicine 331: 1493-1499, 1994. 7. Buchanann JR, Spratt JS, Heuser LS: Tumor growth, doubling times, and the inability of the radiologist to diagnose certain cancers. Radiologic Clinics of North America 21: 115, 1983. 8. Holland T, Mrvunac M, Hendriks JHCL, et al.: So-called interval cancers of the breast. Pathologic and radiographic analysis. Cancer 49:2527, 1982. 9. Murphy WA Jr, Destouet JM, Monsees BS: Professional quality assurance for mammography screening programs. Radiology 175: 319-320, 1990. 10. Bird RE: Professional quality assurance for mammography screening programs. Radiology 177: 587, 1990. 11. Brenner RJ: Medicolegal aspects of breast imaging: variable standards of care relating to different types of practice. AJR 156: 719-723, 1991. 12. Tabar L, Fagerberg G, Duffy SW, Day NE, Gad A, Grontoft O: Update of the Swedish two-country program of mammographic screening for breast cancer. Radiol Clin North Am 30: 187-210, 1992. 13. Thurfjell EL, Lernevall KA, Taube AA: Benefit of independent double reading in a population-based mammography screening program. Radiology 191: 241-244, 1994. 14. Vyborny CJ, Giger ML: Computer vision and artificial intelligence in mammography. AJR 162: 699-708, 1994. 15. Tabar L, Dean PB: Teaching Atlas of Mammography, George Thieme Verlag (Stuttgart, New York), 1983. 16. Ciccone G, Vineis P, Frigerio A, Segnan N: Inter-observer and intra-observer variability of mammogram interpretation: a field study. Eur J Cancer 28: 1054-1058, 1992. 17. Moskowitz M: Screening for breast cancer: How effective are our tests? A critical review. Ca-A Cancer Journal for Clinicians 33: 26-39, 1983. 18. Sickles EA: Mammographic features of 300 consecutive nonpalpable breast cancers. AJR 146: 661-6632, 1986. 19. Kopans DB: The positive predictive value of mammography. AJR 158: 521-526, 1992. 20. Knutzen AM, Gisvold JJ: Likelihood of malignant disease for various categories of mammographically detected, nonpalpable breast lesions. Mayo Clin Proc 68: 454-460, 1993. 16 21. Ballard DH, Brown CM: Computer Vision, Prentice-Hall, Inc. 1982. 22. Sonka M, Hlavac V, Boyle R: Image Processing, Analysis, and Machine Vision, Brooks/Cole Pub., Pacific Grove, CA , 1999. 23. Wolfe JN: Analysis of 462 breast carcinomas. AJR 121: 846-853, 1974. 24. Sickles EA: Mammographic detectability of breast microcalcifications. AJR 139: 913-918, 1982. 25. Sickles EA: Mammographic features of 300 consecutive nonpalpable breast cancers. Am J Rad 146: 662-663, 1986. 26. Fisher ER, Gregorio RM, Fisher B, et al.: The pathology of invasive breast cancer. Cancer 36: 1-84, 1975. 27. Millis RR, Davis R, Stacey AJ: The detection and significance of calcifications in the breast: A radiological and pathological study. Br. J. Radiol. 49: 12-26, 1976. 28. Murphy WA, DeSchryver-Kecskemeti K: Isolated clustered microcalcifications in the breast: Radiologic-pathologic correlation. Radiology 127: 335-341, 1978. 29. Muir BB, Lamb J, Anderson TJ: Microcalcification and its relationship to cancer of the breast: Experience in a screening clinic. Clin. Radiol. 149: 193-200, 1983. 30. Bick U, Giger ML, Schmidt RA, Nishikawa RM, Wolverton DE, Doi K: Automated segmentation of digitized mammograms. Acad Radiol 2: 1-9, 1995. 31. Mendez AJ, Tahoces PG, Lado MJ, Souto M, Correa JL, Vidal JJ: Automatic detection of breast border and nipple in digital mammograms. Comput Methods Programs Biomed 49: 253-262, 1996. 32. Bick U, Giger ML, Schmidt RA, Nishikawa RM, Doi K: Density correction of peripheral breast tissue on digital mammograms. RadioGraphics 16: 1403-1411, 1996. 33. Byng JW, Critten JP, Yaffe MJ: Thickness-equalization processing for mammographic images. Radiology 203: 564-568, 1997. 34. Yin F-F, Giger ML, Doi K, Metz CE, Vyborny CJ, Schmidt RA: Computerized detection of masses in digital mammograms: Analysis of bilateral-subtraction images. Medical Physics 18: 955-963, 1991. 35. Karssemeijer N: Recognition of stellate lesions in digital mammograms. In: Digital Mammography (Gale AC, Astley SM, Dance DR, Cairns AY eds), (Elsevier Science B.V.), 1994. 36. Kegelmeyer WP Jr, Pruneda JM, Bourland PD, Hillis A, Riggs MW, Nipper ML: Computer-aided mammographic screening for spiculated lesions. Radiology 191: 331-337, 1994. 17 37. te Brake GM, Karssemeijer K: Detection of stellate breast abnormalities. In: Digital Mammography 96 (eds. K. Doi, M. L. Giger, R. M. Nishikawa, R. A. Schmidt). Elsevier Scientific Publishers, pgs. 341-346, 1996. 39. Yaffe M: Digital Mammography. In: RSNA Categorical Course in Physics, Technical Aspects of Breast Imaging. Pp. 229-238, 1999. 40. Chan H-P, Cheng SN, Ikeda DM, Niklason L, Adler D: Computerized detection of microcalcifications on mammograms: effects of data compression. Radiology 177(p): 311, 1991. 41. Chan HP, Niklason LT, Ikeda DM, Lam KL, Adler DD: Digitization requirements in mammography: effects on computer-aided detection of microcalcifications. Med Phys 21: 1203-1211, 1994. 42. Spiesberger W: Mammogram inspection by computer. IEEE Transactions on Biomedical Engineering 26: 213-219, 1979. 43. Chan HP, Doi K, Galhotra S, Vyborny CJ, MacMahon H, Jokich PM: Image feature analysis and computer-aided diagnosis in digital radiography. 1. Automated detection of microcalcifications in mammography. Med Phys 14: 538-548, 1987. 44. Chan HP, Doi K, Vyborny CJ, Lam KL, Schmidt RA: Computer-aided detection of microcalcifications in mammograms: Methodology and preliminary clinical study. Invest Radiol 23: 664-671, 1988. 45. Chan HP, Doi K, Vyborny CJ, Schmidt RA, Metz CE, Lam KL, Ogura T, Wu Y, MacMahon H: Improvement in radiologists' detection of clustered microcalcifications on mammograms: The Potential of computer-aided diagnosis. Invest Radiol 25: 1102-1110, 1990. 46. Nishikawa RM, Doi K, Giger ML, Yoshimura H, Wu Y, Vyborny CJ, Schmidt RA, Chan HP: Use of morphological filters in the computerized detection of microcalcifications in digital mammograms. Medical Physics 17: 524, 1990. 47. Nishikawa RM, Giger ML, Doi K, Vyborny CJ, Schmidt RA: Computer-aided detection of clustered microcalcifications: An improved method for grouping detected signals. Med Phys 20: 1661-1666, 1993. 48. Nishikawa RM, Giger ML, Doi K, Vyborny CJ, Schmidt RA: Computer-aided detection of clustered microcalcifications on digital mammograms. Medical and Biological Engineering and Computing 33:174-178, 1995. 49. Wu Y, Doi K, Giger ML, Nishikawa RM: Computerized detection of clustered microcalcifications in digital mammograms: applications of artificial neural networks. Medical Physics 19: 555-560, 1992. 50. Yoshida H, Doi K, Nishikawa RM, Giger ML, Schmidt RA: An improved computerassisted diagnostic scheme using wavelet transform for detecting clustered microcalcifications in digital mammograms. Acad Radiol 3: 621-627, 1996. 18 51. Zhang W, Doi K, Giger ML, Wu Y, Nishikawa RM, Schmidt RA: Computerized detection of clustered microcalcifications in digital mammograms using a shift-invariant artificial neural network. Med Phys 21: 517-524, 1994. 52. Zhang W, Doi K, Giger ML, Nishikawa RM, Schmidt RA: An improved shift-invariant artificial neural network for computerized detection of clustered microcalcifications in digital mammograms. Med Phys 23: 595-601, 1996. 53. Ema T, Doi K, Nishikawa RM, Jiang Y, Papaioannou J: Image feature analysis and computer-aided diagnosis in mammography: reduction of false-positive clustered microcalcifications using local edge-gradient analysis. Med Phys 22: 161-169, 1995. 54. Zhang W, Yoshida H, Nishikawa RM, Doi K: Optimally weighted wavelet transform based on supervised training for detection of microcalcifications in digital mammograms. Med Phys 25: 949-956, 1998. 55. Nagel RH, Nishikawa RM, Doi K: Analysis of methods for reducing false positives in the automated detection of clustered microcalcifications in mammograms. Med Phys 25: 1502-1506, 1998. 56. Anastasio MA, Yoshida H, Nagel R, Nishikawa RM, Doi K: A genetic algorithm-based method for optimizing the performance of a computer-aided diagnosis scheme for detection of clustered microcalcifications in mammograms. Med Phys 25: 1613-1620, 1998. 57. Metz CE: ROC methodology in radiologic imaging. Invest Radiol 21: 720-733, 1986. 58. Astley S, Taylor C, Boggis C, Wilson M, Ellison T: Automated detection of abnormalities on screening mammograms. Radiology 177(P):288, 1990. 59. Fam BW, Olson SL, Winter PF, Scholz FJ: Algorithm for the detection of fine clustered calcifications on film mammograms. Radiology 169: 333-337, 1988. 60. Davies DH, Dance DR: Automatic computer detection of clustered calcifications in digital mammograms. Phys. Med. Biol. 35: 111-118, 1990. 61. Davies DH, Dance DR. The automatic computer detection of subtle calcifications in radiographically dense breasts. Phys Med Biol, 1992; 37: 1385-1390. 62. Karssemeijer N: A stochastic method for automated detection of microcalcifications in digital mammograms. Information Processing in Medical Imaging, Springer-Verlag (New York), pp.227-238, 1991. 63. Karssemeijer N, van Earning L: Iso-precision scaling of digitized mammograms to facilitate image analysis. Proc. SPIE 1444, 1991. 64. Woods KS, Solka JL, Priebe CE, Doss CC, Bowyer KW, Clarke LP. Comparative evaluation of pattern recognition techniques for detection of microcalcifications. Proc SPIE 1905, 1993 (in press). 19 65. Qian W, Kallergi M, Clarke LP, Li HD, Venugopal P, Song D, Clark RA: Tree structured wavelet transform segmentation of microcalcifications in digital mammography. Med Phys 22: 1247-1254, 1995. 66. Qian W, Clarke LP, Song D, Clark RA: Digital mammography: hybrid four-channel wavelet transform for microcalcification segmentation. Acad Radiol 5: 354-364, 1998. 67. Heine JJ, Deans SR, Cullers DK, Stauduhar R, Clarke LP: Multiresolution statistical analysis of high-resolution digital mammograms. IEEE Trans Med Imaging 16: 503-515, 1997. 68. Kalman BL, Reinus WR, Kwasny SC, Laine A, Kotner L: Prescreening entire mammograms for masses with artificial neural networks: preliminary results. Acad Radiol 4: 405-414, 1997. 69. Bankman IN, Christens-Barry WA, Kim DW, Weinberg IN, Gatewood OB, Brody WR. Automated recognition of microcalcification clusters in mammograms. Proc SPIE 1905, 1993 (in press). 70. Mascio LN, Hernandez JM, Logan CM. Automated analysis for microcalcifications in high resolution digital mammograms. Proc SPIE 1905, 1993 (in press). 71. Shen L, Rangayyan RM, Desautels JEL. An automatic detection and classification system for calcifications in mammograms. Proc SPIE 1905, 1993 (in press). 72. Chan HP, Lo SC, Sahiner B, Lam KL, Helvie MA: Computer-aided detection of mammographic microcalcifications: pattern recognition with an artificial neural network. Med Phys 22: 1555-1567, 1995. 73. Ibrahim N, Fujita H, Hara T, Endo T: Automated detection of clustered microcalcifications on mammograms: CAD system application to MIAS database. Phys Med Biol 42: 25772589, 1997. 74. Chang YH, Zheng B, Good WF, Gur D: Identification of clustered microcalcifications on digitized mammograms using morphology and topography-based computer-aided detection schemes: A preliminary experiment. Invest Radiol 33: 746-751, 1998. 75. Zheng B, Chang YH, Staiger M, Good W, Gur D: Computer-aided detection of clustered microcalcifications in digitized mammograms. Acad Radiol 2: 655-662, 1995. 76. Cheng HD, Lui YM, Freimanis RI: A novel approach to microcalcification detection using fuzzy logic technique. IEEE Trans Med Imaging 17: 442-450, 1998. 77. Winsberg F, Elkin M, Macy J, Bordaz V, Weymouth W: Detection of radiographic abnormalities in mammograms by means of optical scanning and computer analysis. Radiology 89: 211-215, 1967. 78. Kimme C, O'Loughlin BJ, Sklansky J: Automatic detection of suspicious abnormalities in breast radiographs. In: Data Structures, Computer Graphics, and Pattern Recognition, edited by A. Klinger, K. S. Fu, T. L. Kunii (Academic Press, New York), 1975, pp. 427-447. 20 79. Hand W, Semmlow JL, Ackerman LV, Alcorn FS: Computer screening of xeromammograms: A technique for defining suspicious areas of the breast. Computers and Biomedical Research 12: 445-460, 1979. 80. Giger ML, Yin F-F, Doi K, Metz CE, Schmidt RA, Vyborny CJ: Investigation of methods for the computerized detection and analysis of mammographic masses. Proc. SPIE 1233: 183-184, 1990. 81. Yin FF, Giger ML, Doi K, Metz CE, Vyborny CJ, Schmidt RA: Computerized detection of masses in digital mammograms: Analysis of bilateral subtraction images. Med Phys 18: 955-963, 1991. 82. Yin FF, Giger ML, Doi K, Vyborny CJ, Schmidt RA: Computerized detection of masses in digital mammograms: Automated alignment of breast images and its effect on bilateral-subtraction technique. Med Phys 21: 445-452, 1994. 83. Kupinski MA, Giger ML, Lu P, Huo Z: Computerized detection of mammographic lesions: performance of an artificial neural network with enhanced feature extraction. Proc SPIE, pp. 598-60, 1995. 84. Kupinsk MA, Giger ML, Doi K: Optimization of neural network inputs with genetic algorithms. In: Digital Mammography 96 (eds. K. Doi, M. L. Giger, R. M. Nishikawa, R. A. Schmidt). Elsevier Scientific Publishers, pgs. 401-404, 1996. 85. Kupinski MA, Giger ML: Investigation of regularized neural networks for the computerized detection of mass lesions in digital mammograms. Proc IEEE Eng Medicine & Biology Conference, pp. 1336-1339, 1997. 86. Kupinski MA, Giger ML: Feature selection and classifiers for the computerized detection of mass lesions in digital mammography. IEEE Int Congr Neural Networks, pp. 24602463, 1997. 87. Kupinski MA, Giger ML: Automated seeded lesion segmentation on digital mammograms. IEEE Trans Med Imaging 17: 510-517, 1999. 88. Lai SM, Li X, Bischof WF: On techniques for detecting circumscribed masses in mammograms. IEEE Transactions on Medical Imaging 8: 377-386, 1989. 89. Brzakovic D, Luo XM, Brzakovic P: An approach to automated detection of tumors in mammograms. IEEE Transactions on Medical Imaging 9: 233-241, 1990. 90. Polakowski WE, Cournoyer DA, Rogers SK, DeSimio MP, Ruck DW, Hoffmeister JW, Raines RA: Computer-aided breast cancer detection and diagnosis of masses using difference of Gaussians and derivative-based feature saliency. IEEE Trans Med Imaging 16: 811-819, 1997. 91. Bick U, Giger ML, Schmidt RA, Doi K: A new single-image method for computer-aided detection of small mammographic masses. Proc CAR -- Computer Assisted Radiology, Lemke HU, Inamura K, Jaffe CC, Vannier MW, eds., pgs. 357-363, 1995. 92. Karssemeijer N, te Brake G: Combining single view features and asymmetry for detection of mass lesions. In: Digital Mammography, Nijmegen 1998 (eds. N. 21 Karssemeijer, M. Thijssen, J. Hendriks, L. van Erning). Dordrecht (Kluwer Academic Publishers), pp. 95-102, 1998. 93. Boyd NF, Lockwood GA, Byng JW, Tritchler DL, Yaffe MJ: Mammographic densities and breast cancer risk. Cancer Epidemiol Biomarkers Prev 7: 1133-1144, 1998. 94. Chang YH, Zheng B, Gur D: Computerized identification of suspicious regions for masses in digitized mammograms. Invest Radiol 31: 146-153, 1996. 95. Chang YH, Zheng B, Gur D: Robustness of computerized identification of masses in digitized mammograms: A preliminary assessment. Invest Radiol 31: 563-568, 1996. 96. Zheng B, Chang YH, Gur D: Computerized detection of masses in digitized mammograms using single-image segmentation and a multilayer topographic feature analysis. Acad Radiol 2: 959-966, 1995. 97. Zheng B, Chang YH, Gur D: Computerized detection of masses in digitized mammograms: comparison of single-image segmentation and a bilateral-image subtraction. Acad Radiol 2: 1056-1061, 1995. 98. Rymon R, Zheng B, Chang YH, Gur D: Incorporation of a set enumeration trees-based classifier into a hybrid computer-assisted diagnosis scheme for mass detection. Acad Radiol 3: 181-187, 1998. 99. Mendez AJ, Tahoces PG, Lado MJ, Souto M, Vidal JJ: Computer-aided diagnosis: automatic detection of malignant masses in digitized mammograms. Med Phys 6: 957-961, 1998. 100. Zouras WK, Giger ML, Lu P, Wolverton DE, Vyborny CJ, Doi K: Investigation of a temporal subtraction scheme for computerized detection of breast masses. In: Digital Mammography ‘96 (eds. K. Doi, M. L. Giger, R. M. Nishikawa, R. A. Schmidt). Amsterdam (Elsevier Science (Excerpta Medica International Congress series), pp. 411415, 1996. 101. Gale AG, Astley SM, Dance DR, Cairns AY: Digital Mammography (Elsevier Science, Amsterdam), 1994. 102. Doi K, Giger ML, Nishikawa RM, Schmidt RA: Digital Mammography ‘96 (Elsevier Science, Amsterdam), 1996. 103. Karssemeijer N, Thijssen M, Hendriks J, van Erning L: Digital Mammography (Kluwer Academic Publishers), 1998. 104. Gale AG, Roebuck EJ, Riley P, Worthington BS, et al.: Computer aids to mammographic diagnosis. British Journal of Radiology 60: 887-891, 1987. 105. Getty DJ, Pickett RM, D'Orsi CJ, Swets JA: Enhanced interpretation of diagnostic images. Invest. Radiol. 23: 240-252, 1988. 22 106. Swets JA, Getty DJ, Pickett RM, D'Orsi CJ, Seltzer SE, McNeil BJ: Enhancing and evaluating diagnostic accuracy. Med Decis Making 11:9-18, 1991. 107. Cook HM, Fox MD: Application of expert systems to mammographic image analysis. American Journal of Physiologic Imaging 4: 16-22, 1989. 108. Swett HA, Miller PA: ICON: A computer-based approach to differential diagnosis in radiology. Radiology 163: 555-558, 1987. 109. Swett HA, Fisher PR, Cohn AI, Miller PI, Mutalik PG: Expert system controlled image display. Radiology 172: 487-493, 1989. 110. Wu Y, Giger ML, Doi K, Vyborny CJ, Schmidt RA, Metz CE: Application of neural networks in mammography: Applications in decision making in the diagnosis of breast cancer. Radiology 187: 81-87, 1993. 111. D’Orsi CJ, Bassett LW, Feig SA, Jackson VP, Kopans DB, Linver MN, Sickles EA, Stelling CB: Breast Imaging Reporting and Data System (BI-RADS). Reston, VA (American College of Radiology), 1998. 112. Baker JA, Kornguth PJ, Lo JY, Floyd CE Jr: Artificial neural network: improving the quality of breast biopsy recommendations. Radiology 1: 131-135, 1996. 113. Floyd CE Jr, Lo JY, Yun AJ, Sullivan DC, Kornguth PJ: Prediction of breast cancer malignancy using an artificial neural network. Cancer 74: 2944-1948, 1994. 114. Baker JA, Kornguth PJ, Lo JY, Williford ME, Floyd CE Jr: Breast cancer: prediction with artificial neural network based on BI-RADS standardized lesions. Radiology 196: 817-822, 1995. 115. Kovalerchuk B, Triantaphyllou E, Ruiz JF, Clayton J: Fuzzy logic in computer-aided breast cancer diagnosis: analysis of lobulation. Artif Intell Med 11: 75-85, 1997. 116. Baker JA, Kornguth PJ, Floyd CE Jr: Breast imaging reporting and data system standardized mammography lexicon: observer variability in lesion description. AJR 166: 773-778, 1996.117. Ackerman LV, Gose EE: Breast lesion classification by computer and xeroradiography. Cancer 30: 1025-1035, 1972. 117. Ackerman LV, Gose EE: Breast lesion classification by computer and xeroradiography. Breast Cancer 30:1025-1035, 1972. 118. Wee WG, Moskowitz M, Chang N-C, Ting Y-C, Pemmeraju S: Evaluation of mammographic calcifications using a computer program. Radiology 116:717-720, 1975. 119. Semmlow JL, Shadagoppan A, Ackerman LV, Hand W, Alcorn FS: A fully automated system for screening xeromammograms. Computers and Biomedical Research 13: 350-362, 1980. 120. Fox SH, Pujare UM, Wee WG, Moskowitz M, Hutter RVP: A computer analysis of mammographic microcalcifications: Global approach. Proc. IEEE 5th International Conf. on Pattern Recognition: 624-631, 1980. 23 121. Magnin IE, ElAlaoui M, Bremond A: Automatic microcalcifications pattern recognition from x-ray mammographies. Proc. SPIE 1137: 170-175, 1989. 122. Giger ML, Vyborny CJ, Schmidt RA: Computerized characterization of mammographic masses: Analysis of spiculation. Cancer Letters 77: 201-211, 1994. 123. Huo Z, Giger ML, Vyborny CJ, Bick U, Lu P, Wolverton DE, Schmidt RA: Analysis of spiculation in the computerized classification of mammographic masses" Medical Physics 22:1569-1579, 1995. 124. Huo Z, Giger ML, Vyborny CJ, Wolverton DE, Schmidt RA, Doi K: Automated computerized classification of malignant and benign mass lesions on digitized mammograms. Academic Radiology 5: 155-168, 1998. 125. Patrick EA, Moskowitz M, Mansukhani VT, Gruenstein EI: Expert learning system network for diagnosis of breast calcifications. Invest Radiol 16: 534-539, 1991. 126. Jiang Y, Nishikawa RM, Wolverton DE, Metz CE, Giger ML, Schmidt RA, Vyborny CJ, Doi K: Automated feature analysis and classification of malignant and benign clustered microcalcifications. Radiology 198:671-678, 1996. 127. Jiang Y, Nishikawa RM, Schmidt RA, Metz CE, Giger ML, Doi K: Improving breast cancer diagnosis with computer-aided diagnosis. Academic Radiology 6: 22-33, 1999. 128. Chan HP, Wei D, Helvie MA, Sahiner B, Adler DD, Goodsitt MM, Petrick N: Computeraided classification of mammographic masses and normal tissue: Linear discriminant analysis in texture feature space. Phys Med Biol 40: 857-876, 1995. 129. Chan HP, Sahiner B, Lam KL, Petrick N, Helvie MA, Goodsitt MM, Adler DD: Computerized analysis of mammographic microcalcifications in morphological and texture feature spaces. Med Phys 25: 2007-2019, 1998. 130. Sahiner B, Chan HP, Petrick N, Helvie MA, Goodsitt MM: Design of a high-sensitivity classifier based on a genetic algorithm: application to computer-aided diagnosis. Phys Med Biol 43: 2853-2871, 1998. 131. Stavros AT, Thickman D, Rapp CL, Dennis MA, Parker SH, Sisney GA: Solid breast nodules: use of sonography to distinguish between benign and malignant lesions. Radiology 196:123-134, 1995. 132. Huber S, Delorme S, Knopp MV, Junkermann H, Zuna I, von Fournier D, van Kaick G: Breast tumors: computer-assisted quantitative assessment with color Doppler US. Radiology 192:797-801, 1994. 133. Birdwell RL, Ikeda DM, Jeffrey SS, Jeffrey RB, Jr.: Preliminary experience with power Doppler imaging of solid breat masses. AJR 169: 703-707, 1997. 134. Giger ML, Huo Z, Wolverton DE, Vyborny C, Moran C, Schmidt RA, Al-Hallaq H, Nishikawa R, Doi K: Computer-aided diagnosis of digital mammographic and ultrasound images of breast mass lesions. Digital Mammography 1998, Eds.: Karssemeijer N, 24 Thijssen M, Hendriks J, van Erning L. Kluwer Academic Publishers, Dordrecht, The Netherlands, pp. 143-147, 1998. 135. Garra BS, Krasner BH, Horii SC, Ascher S, Mun SK, Zeman RK: Improving the distinction between benign and malignant breast lesions: the value of sonographic texture analysis. Ultrason Imaging 15: 267-285, 1993. 136. Giger ML, Al-Hallaq H, Huo Z, Moran C, Wolverton DE, Chan CW, Zhong W: Computerized analysis of lesions in ultrasound images of the breast. Academic 6: 665-674, 1999. 137. Mussurakis S, Buckley DL, Coady AM, Turnbull LW, Horseman A: Observer variability in the interpretation of contrast enhanced MRI of the breast. Br J Radiol 69: 1009-1016, 1996. 138. Nunes LW, Schnall MD, Orel SG, et al.: Breast MR imaging: interpretation model. Radiology 202: 833-841, 1997. 139. Penn AI, Bolinger L, Schnall MD, Loew MH: Discrimination of MR images of breast masses with fractal-interpolation function models. Acad Radiol 6: 156-163, 1999. 140. Gilhuijs KGA, Giger ML, Bick U: Automated analysis of breast lesions in three dimensions using dynamic magnetic resonance imaging. Medical Physics 25:1647-1654, 1998. 141. Magnin IE, Cluzeau F, Odet CL, Bremond A: Mammographic texture analysis: an evaluation of risk for developing breast cancer. Optical Engineering 25: 780-784, 1986. 142. Wolfe JN: Breast patterns as an index of riskfor developing breast cancer. AJR 126: 11301139, 1976. 143 Caldwell CB, Stapleton SJ, Holdsworth DW, Jong RA, Weiser WJ, Cooke G, Yaffe MJ: Characterization of mammographic parenchymal pattern by fractal dimensions. Phys. Med. Biol. 35: 235-247, 1990. 144. Byng JW, Yaffe MJ, Jong RA, Shumak RS, Lockwood GA, Tritchler DL, Boyd NF: Analysis of mammographic density and breast cancer risk from digitized mammograms. RadioGraphics 18: 1587-1598, 1998. 145. Byng JW, Boyd NF, Fishell E, Jong RA, Yaffe MJ : Automated analysis of mammographic densities. Phys Med Biol 41: 909-923, 1996. 146. Byng JW, Yaffe MJ, Lockwood GA, Little LE, Tritchler DL, Boyd NF: Automated analysis of mammographic densities and breast carcinoma risk. Cancer 80: 66-74, 1997. 147. Huo Z, Giger ML, Olopade OI, Wolverton DE, Weber B, et al: Computer-aided diagnosis: Breast cancer risk assessment from mammographic parenchymal patterns in digital mammograms. Digital Mammography '96. Proceedings of the 3rd International Workshop of Digital Mammography, Elsevier, New York, pp. 191-194, 1996. 25 148. Huo Z, Giger ML, Olopade FI, Wolverton DE, Cummings S, Zhong W, Doi K: Computerized analysis of digitized mammograms of women with low breast cancer risk and of BRCA1/BRCA2 fene-mutation carriers. Digital Mammography 1998, Eds.: Karssemeijer N, Thijssen M, Hendriks J, van Erning L. Kluwer Academic Publishers, Dordrecht, The Netherlands, pp. 419-422, 1998. 150. Jiang Y, Metz CE, Nishikawa RM: A receiver operating characteristic partial area index for highly sensitive diagnostic tests. Radiology 201: 745-750, 1996. 151. Chan HP, Sahiner B, Wagner RF, Petrick N, Mossoba J: Effects of sample size on classifier design: Quadratic and neural network classifiers. Proc SPIE 3034: 1102-1113, 1997. 152. Nishikawa RM, Papaioannou J, Collins SA: Reproducibility of an automated scheme for the detection of clustered microcalcifications on digital mammograms. Proc SPIE 2710: 24-29, 1996. 153. Nishikawa RM, Giger ML, Doi K, Vyborny CJ: Effect of case selection on the performance of computer-aided detection schemes. Med Phys 21: 265-269, 1994. 154. Nishikawa RM: Mammographic databases. Breast Disease 10: 137-150, 1998. 155. Giger ML: Overview of computer-aided diagnosis in breast imaging. In: Computer Aided Diagnosis, Doi K, MacMahon H, Giger ML, Hoffmann KR (eds). (Elsevier, Amsterdam) (in press), 1999. 156. Kupinski MA, Giger ML: Feature selection with limited datasets. Med Phys 26: 21762182, 1999. 157. Huo Z: Computerized methods for classification of masses and analysis of parenchymal patterns on digitized mammograms. Ph.D. Dissertation, University of Chicago, 1998. 158. Schmidt RA, Nishikawa RM, Osnis RB, Giger ML, Schreibman K, Doi K: Computerized detection of lesions missed by mammography. In: Digital Mammography ‘96 (eds. K. Doi, M. L. Giger, R. M. Nishikawa, R. A. Schmidt). Amsterdam (Elsevier Science (Excerpta Medica International Congress series), pp. 105-110, 1996. 159. te Brake GM, Karssemeijer N, Hendriks HCL: Automated detection of breast carcimomas not detected in a screening program. Radiology 207:465-471, 1998. 160. Giger ML, Doi K, MacMahon H, et al: An “intelligent” workstation for computer-aided diagnosis. Radiographics 1993; 13:647-656. 161. Nishikawa RM, Giger ML, Wolverton DE, Schmidt RA, Comstock CE, Papaioannou J, Collins SA, Doi K: Prospective testing of a clinical mammography workstation for CAD: Analysis of the first 10,000 cases. Digital Mammography 1998, Eds.: Karssemeijer N, Thijssen M, Hendriks J, van Erning L. Kluwer Academic Publishers, Dordrecht, The Netherlands, pp. 401-406.167. 162. Hendricks JHC: Comparison of CAD with double reading in breast cancer screening. Presented at the First Internationa Workshop on Computer-Aided Diagnosis, Chicago, Illinois, Sept. 22, 1998. 26 163. Roehrig J, DOi T, Hasegawa A, Hunt B, Marshall J, Romsdahl H, Schneider A, Sharbaugh R, Zhang W: Clinical results with R2 imagechecker system. Digital Mammography 1998, Eds.: Karssemeijer N, Thijssen M, Hendriks J, van Erning L. Kluwer Academic Publishers, Dordrecht, The Netherlands, pp. 395-400, 1998. 164. Anastasio MA, Kupinski MA, Nishikawa RM: Optimization of FROC analysis of rulebased detection schemes using a multiobjective approach, IEEE Trans on Medical Image 17: 1089-1093, 1998. 165. Burhenne LJW, D’Orsi CJ, Feig SA, Kopans DB, Sickles EA, Tabar L, Vyborny CJ, Castellino RA: The potential contribution of computer-aided detection to the sensitivity of screening mammography. Radiology 215: 554-562, 2000.