Get 20M+ Full-Text Papers For Less Than $1.50/day. Start a 7-Day Trial for You or Your Team.

Learn More →

Radiomics: Images Are More than Pictures, They Are Data

Radiomics: Images Are More than Pictures, They Are Data Original r esearch Special Repo R This copy is for personal use only. To order printed copies, contact [email protected] r adiomics: Images Are More than Pictures, They Are Data Robert J. Gillies, PhD In the past decade, the field of medical image analysis has Paul E. Kinahan, PhD grown exponentially, with an increased number of pattern Hedvig Hricak, MD, PhD, Dr(hc) recognition tools and an increase in data set sizes. These advances have facilitated the development of processes for high-throughput extraction of quantitative features that result in the conversion of images into mineable data and the subsequent analysis of these data for decision support; this practice is termed radiomics. This is in contrast to the traditional practice of treating medical images as pictures intended solely for visual interpretation. Radiomic data contain first-, second-, and higher-order statistics. These data are combined with other patient data and are mined with sophisticated bioinformatics tools to develop models that may potentially improve diagnostic, prognostic, and predictive accuracy. Because radiomics analyses are in- tended to be conducted with standard of care images, it is conceivable that conversion of digital images to mineable data will eventually become routine practice. This report describes the process of radiomics, its challenges, and its potential power to facilitate better clinical decision mak- ing, particularly in the care of patients with cancer. From the Department of Cancer Imaging, H. Lee Moffitt Cancer Center and Research Institute, 12902 Magnolia Dr, Tampa, FL 33612 (R.J.G.); Department of Radiology, University of Washington, Seattle, Wash (P.E.K.); and Department of Radiology, Memorial Sloan-Kettering Cancer Center, New York NY 10065 (H.H.). Received May 29, 2015; revision requested July 6; revision received August 14; accepted September 14; final version accepted October 16. Address correspondence to R.J.G. (e-mail: Robert. [email protected] ). Radiology: Volume 278: Number 2—February 2016 n radiology.rsna.org 563 SPECIAL REPORT: Radiomics Gillies et al ith high-throughput computing, subsequently mine the data for hypo- assessment of prognosis, prediction of it is now possible to rapidly thesis generation, testing, or both. Ra- response to treatment, and monitoring Wextract innumerable quantita- diomics is designed to develop decision of disease status. tive features from tomographic images support tools; therefore, it involves The mining of radiomic data to (computed tomography [CT], magnetic combining radiomic data with other detect correlations with genomic pat- resonance [MR], or positron emission patient characteristics, as available, to terns is known as radiogenomics, and tomography [PET] images). The con- increase the power of the decision sup- it has elicited especially great interest version of digital medical images into port models. As radiomics is intended in the research community. To avoid mineable high-dimensional data, a to extract maximal information from confusion, it should be noted that the process that is known as radiomics, is standard of care images, the creation of term radiogenomics is also used in the motivated by the concept that biomed- databases that combine vast quantities field of radiation oncology to describe ical images contain information that of radiomics data (and ideally other whole-genome analyses aimed at de- reflects underlying pathophysiology and complementary data) from millions of termining the genetic causes of varia- that these relationships can be revealed patients is foreseeable. tions in radiosensitivity (4,5). Hence- via quantitative image analyses. Al- Although radiomics can be applied forward in this article, we will refer to though radiomics is a natural extension to a large number of conditions, it is radiogenomics only as the combination of computer-aided diagnosis and detec- most well developed in oncology be- of radiomic features with genomic data tion (CAD) systems, it is significantly cause of support from the National for the purpose of enabling decision different from them. CAD systems are Cancer Institute (NCI) Quantitative support. The value of radiogenomics usually standalone systems that are Imaging Network (QIN) and other ini- stems from the fact that while virtu- designated by the Food and Drug Ad- tiatives from the NCI Cancer Imaging ally all patients with cancer undergo ministration for use in either the detec- Program. As described in subsequent imaging at some point and often mul- tion or diagnosis of disease (1). Early sections of this article, the potential of tiple times during their care, not all of successes of CAD have been greatest radiomics to contribute to decision sup- them have their disease genomically in breast cancer imaging (2,3). Un- port in oncology has grown as knowl- profiled. Furthermore, when genomic like CAD systems, which are directed edge and analytic tools have evolved. profiling is performed, it is done one toward delivering a single answer (ie, Quantitative image features based on time at one location and is susceptible presence of a lesion or cancer), ra- intensity, shape, size or volume, and diomics is explicitly a process designed texture offer information on tumor phe- to extract a large number of quantita- notype and microenvironment (or hab- Published online before print tive features from digital images, place itat) that is distinct from that provided 10.1148/radiol.2015151169 Content codes: these data in shared databases, and by clinical reports, laboratory test re- Radiology 2016; 278:563–577 sults, and genomic or proteomic assays. Abbreviations: These features, in conjunction with the Advances in Knowledge ADC = apparent diffusion coefficient other information, can be correlated BI-RADS = Breast Imaging Reporting and Data System n Radiomics is defined as the con - with clinical outcomes data and used CAD = computer-aided diagnosis and detection version of images to higher- for evidence-based clinical decision Lung-RADS = Lung Imaging Reporting and Data System dimensional data and the subse- support (Fig 1). Radiomics appears NCI = National Cancer Institute quent mining of these data for to offer a nearly limitless supply of PI-RADS = Prostate Imaging Reporting and Data System improved decision support. QIBA = Quantitative Imaging Biomarkers Alliance imaging biomarkers that could poten- QIN = Quantitative Imaging Network tially aid cancer detection, diagnosis, n Radiomics has been initiated in RSNA = Radiological Society of North America oncology studies, but it is poten- Implications for Patient Care tially applicable to all diseases. Author contributions: Guarantor of integrity of entire study, R.J.G.; study n Radiomics can be performed with n Radiomics is designed to be used concepts/study design or data acquisition or data analysis/ tomographic images from CT, in decision support of precision interpretation, all authors; manuscript drafting or manu- MR imaging, and PET studies. medicine. script revision for important intellectual content, all authors; approval of final version of submitted manuscript, all au- n Image features are extracted n Although relationships between thors; agrees to ensure any questions related to the work from volumes of interest, which radiomics and outcome are are appropriately resolved, all authors; literature research, can be either entire tumors or defined with populations, they all authors; and manuscript editing, all authors defined subvolumes within can be applied to individual tumors, known as habitats. patients. Funding: This research was supported by the National Insti- n Radiomics is a new field, and n Radiomic analysis promises to tutes of Health (grants U54CA143970, U01CA143062, there are substantial challenges increase precision in diagnosis, R01CA190105, and R01CA187532). to its implementation in a clinical assessment of prognosis, and setting. prediction of therapy response. Conflicts of interest are listed at the end of this article. 564 radiology.rsna.org Radiology: Volume 278: Number 2—February 2016 SPECIAL REPORT: Radiomics Gillies et al Figure 1 Figure 1: Flowchart shows the process of radiomics and the use of radiomics in decision support. Patient work-up requires information from disparate sources to be combined into a coherent model to describe where the lesion is, what it is, and what it is doing. Radiomics begins with acquisition of high-quality images. From these images, a region of interest (ROI) that contains either the whole tumor or subregions (ie, habitats) within the tumor can be identified. These are segmented with operator edits and are eventually rendered in three dimensions (3D). Quantitative features are extracted from these rendered volumes to generate a report, which is placed in a database along with other data, such as clinical and genomic data. These data are then mined to develop diagnostic, predictive, or prognostic models for outcomes of interest. to sampling error. Thus, radiogenom- and prognosis. For example, research on multiple levels: One multisection ics has two potential uses, which will has already shown the capacity of ra- or three-dimensional image from one be described in detail in the Examples diomics analyses to help distinguish patient may easily contain millions of of Radiomics Results section. First, a prostate cancer from benign prostate voxels. Also, one tumor (or other ab- subset of the radiomic data can be used tissue or add information about pros- normal entity) may contain hundreds to suggest gene expression or mutation tate cancer aggressiveness (6). In the of measurable features describing size, status that potentially warrants further evaluation of lung cancer and in the shape, and texture. testing. This is important because the evaluation of glioblastoma multiforme, Radiomics analyses epitomize the radiomic data are derived from the radiomics has been shown to be a tool pursuit of precision medicine, in which entire tumor (or tumors) rather than with which to assess patient prognosis molecular and other biomarkers are from just a sample. Thus, radiomics (7). The tools developed for radiomics used to predict the right treatment can provide important information re- can help in daily clinical work, and for the right patient at the right time. garding the sample genomics and can radiologists can play a pivotal role in The availability of robust and validated be used for cross-validation. Second, continuously building the databases biomarkers is essential to move preci- a subset of radiomic features is not that are to be used for future decision sion medicine forward (9). Around the significantly related to gene expression support. world, efforts are underway to improve or mutational data and, hence, has the The suffix -omics is a term that orig- the availability of such biomarkers, potential to provide additional, inde- inated in molecular biology disciplines and in the United States, the effort is pendent information. The combination to describe the detailed characteriza- most notably through The Precision of this subset of radiomic features with tion of biologic molecules such as DNA Medicine Initiative (10,11). This ini- genomic data may increase diagnostic, (genomics), RNA (transcriptomics), tiative will provide funding for a new prognostic, and predictive power. proteins (proteomics), and metabolites model of patient-powered research While radiomics primarily grew (metabolomics). Now, the term is also that promises to accelerate biomedical out of basic research, lately it has also being used in other medical research discoveries and provide clinicians with elicited interest from those in clinical fields that generate complex high-di - new tools, knowledge, and therapies research, as well as those in daily clin- mensional data from single objects or that enable more precise personalized ical practice. For a clinical radiologist, samples (8). One desirable character- care. radiomics has the potential to help istic of -omics data is that these data A major strength of a radiomics with the diagnosis of both common and are mineable and, as such, can be used approach for cancer is that digital ra- rare tumors. Visualization of tumor for exploration and hypothesis genera- diologic images are obtained for almost heterogeneity may prove critical in the tion. The -omics concept readily applies every patient with cancer, and all of assessment of tumor aggressiveness to quantitative tomographic imaging these images are potential sources for Radiology: Volume 278: Number 2—February 2016 radiology.rsna.org 565 SPECIAL REPORT: Radiomics Gillies et al tumor biology. A central hypothesis driv- each with its own challenges (24,25). Table 1 ing radiomics research is that radiomics These steps are shown in Figure 1 Important Aspects of Radiomics has the potential to enable quantitative and include: (a) acquiring the images, measurement of intra- and intertumoral (b) identifying the volumes of interest Aspect heterogeneity. Moreover, radiomics of- (ie, those that may contain prognostic Uses standard-of-care images fers the possibility of longitudinal use in value), (c) segmenting the volumes (ie, Interrogates the entire tumor treatment monitoring and optimization delineating the borders of the volume Can be used to interrogate stroma or in active surveillance. Although such with computer-assisted contouring), (d) Enables longitudinal monitoring applications of radiomics have yet to be extracting and qualifying descriptive fea- explored in depth, they may provide the tures from the volume, (e) using these to most value going forward. populate a searchable database, and (f) radiomics databases (Table 1). In the It should be emphasized that ra- mining these data to develop classifier United States alone, there are approx- diomic and radiogenomic analyses can models to predict outcomes either alone imately 1.6 million new cancer cases be used to identify correlations, not or in combination with additional infor- every year (12). Most of these patients causes; thus, they are not expected to mation, such as demographic, clinical, will undergo multiple CT, MR imaging, enable definitive assessment of genetic comorbidity, or genomic data. We will and PET examinations. In the future, or other contents of tissue through im- discuss these processes in turn. it is possible that image interpretation aging alone. However, correlation of ra- Image Acquisition for all these studies will be augmented diomic data with genomic or other -omic by using radiomics, building an unprec- data could inform not only the decision Modern CT, MR imaging, and com- edented source of big data that will about whether to test for certain gene bined PET/CT units allow for wide expand the potential for discovering alterations in biopsy samples but also variations in acquisition and image re- helpful correlations. While radiomics the choice of biopsy sites. It also could construction protocols, and standardi- will allow better characterization of pa- provide confirmatory information to zation of these protocols across medi- tients and their diseases through new support histopathologic findings. This cal imaging centers is typically lacking. applications of genomics and improved is important, as it is estimated that the This is generally not a problem in the methods of phenotyping, it will also add error rate of cancer histopathology can routine identification of radiologic fea - to the challenges of data management, be as high as 23% (20–23). Errors in tures used in clinical practice. However, as we will discuss later in this article. histopathology are due to both sampling when images are analyzed numerically Radiomics offers important advan- errors and observer variability; thus, to extract meaningful data, variations tages for assessment of tumor biology. there is a great need for additional quan- in acquisition and image reconstruction It is now appreciated that most clinically titative diagnostic information. parameters can introduce changes that relevant solid tumors are highly hetero- We believe that radiomics is rapidly are not due to underlying biologic ef- geneous at the phenotypic, physiologic, expanding beyond a boutique research fects. This has been well recognized in and genomic levels (13–15) and that area and is emerging as a translational the emerging field of quantitative imag - they continue to evolve over time. In this technology. Hence, this is an appropri- ing, in which the intent is to generate emerging era of targeted therapies, it ate time to begin to establish bench- medical images with describable limits is notable that most responses are not marks for data extraction, analysis, of bias and variance. In other words, it durable and that benefit is generally and presentation. The goal of this is not sufficient to report a number or a measured in months, not years. For ex- report is to introduce the practice of set of numbers derived from images; in- ample, this is the case with (a) gefitinib radiomics to a wide audience of prac- stead, we must also be able to provide in patients with epidermal growth factor ticing clinicians, including radiologists, error bars, as is done with every other receptor–mutated lung cancer (16), (b) to engage a broader community in es- credible laboratory measurement. trastuzamab in those with human epider- tablishing benchmarks. In doing so, we There have been multiple efforts to mal growth factor receptor 2 (or HER2) will describe the processes involved in advance quantitative imaging, includ- overexpressing breast cancer (17), and radiomics and the unique information ing definition of acquisition and recon - (c) vemurafenib in those with B-Raf–mu- it offers, as well as its challenges and struction standards, over the past 15 tated melanoma (18). Genomic hetero- their potential solutions. We will also years (26,27). The QIN is a cooperative geneity within tumors and across meta- highlight some of the more recent find - network initiated by the NCI with the static tumor sites in the same patient is ings of importance and, finally, offer a goal of developing quantitative imaging the major cause of treatment failure and vision for radiomics of the future. methods that improve the effectiveness emergence of therapy resistance (19). of clinical trials of new cancer ther- Thus, precision medicine requires not apies (28). The QIN is a major initia- Process of Radiomics only in vitro biomarkers and companion tive from the NCI and can be regarded diagnostics but also spatially and tem- While conceptually simple, the practice as the leading edge of new imaging porally resolved in vivo biomarkers of of radiomics involves discrete steps, methods, including radiomics. Also, the 566 radiology.rsna.org Radiology: Volume 278: Number 2—February 2016 SPECIAL REPORT: Radiomics Gillies et al Volume of Interest Identification Figure 2 Identification of tissue volumes of prog - nostic value is the core of the practice of radiology in oncology. Although at the time of diagnosis cancer can be detect- ed at one tumor site or multiple tumor sites, most patients with cancer metas- tasis have multiple lesions. In either sce- nario, we need to identify tumors and suspected tumors as volumes of interest. However, detailed analysis of subvolumes within tumors (the manifestations of tu- mor heterogeneity) that may have prog- nostic value generally are not captured in a radiology report because of the spatial and contrast limitations of digital images. While heterogeneity is not included in Response Evaluation Criteria in Solid Tumors, version 1.1 (32), a few texture descriptors have been incorporated in more complex diagnostic imaging report- ing and data systems, such as the Breast Imaging Reporting and Data System (BI-RADS) (33), the Prostate Imaging Reporting and Data System (PI-RADS) (34), and the Lung Imaging Reporting and Data System (Lung-RADS) (35). In Figure 2: Habitats in a patient with glioblastoma multiforme. Habitats were the practice of radiomics, so-called sub- defined by combining unenhanced and contrast-enhanced T1-weighted, 120- volumes of interest can be captured and msec echo time T2-weighted, and fluid-attenuated inversion recovery (FLAIR) added to the analyses. The basic philoso- images. Data from each acquisition were sorted into low and high values phy, which has its foundation in process with automated histogram analyses, yielding a potential for eight different engineering, is to capture as much data combinations. In practice, only four distinct combinations were observed. They as possible at the front end and use correspond to the red (low T1, high T2 and FLAIR), yellow (low T1 and T2, high FLAIR), blue (High T1 and FLAIR, low T2), and green (high T1, low FLAIR and T2) downstream database mining to identify areas. Notably, while the identities of individual voxels were determined math- the features with the highest prognostic ematically, they spatially clustered into contiguous regions reflecting different value. This is driven by the knowledge physiologic microenvironments. (Image courtesy of R. A. Gatenby.) that attempting to filter the data at input would be inefficient and would presup - Radiological Society of North America 100 participants were involved in the pose knowledge regarding the value of (RSNA) and the National Institute for creation of the initial QIBA fluorode - the features in classifier models before Biomedical Imaging and Bioengineer- oxyglucose PET/CT profile, which was they were tested. ing have sponsored the Quantitative released in 2014 (30,31). The American Recently, the concept of using image Imaging Biomarkers Alliance (QIBA), Association of Physicists in Medicine is data to identify physiologically distinct which is a major effort in quantitative providing technical guidelines in quanti- regions within lesions has been de- imaging (29). The goal of QIBA is to tative imaging in the form of modality- scribed (36). In this approach, images industrialize quantitative imaging by dependent reports on imager operation with different acquisition parameters bringing together the entire spectrum and testing. Finally, relevant profes- (eg, contrast material–enhanced T1- of groups involved in its development sional societies, such as the American weighted MR imaging, diffusion-weight- and implementation. The main product College of Radiology, RSNA, the Soci- ed, and fluid attenuation sequences) can of QIBA is a new type of a document ety of Nuclear Medicine and Molecular be combined to yield regions with spe- termed a profile that provides a con- Imaging, the International Society of cific combinations of quantitative image sensus on the measurement accuracy of Magnetic Resonance in Medicine, and data. Notably, when this is performed, a quantitative imaging biomarker for a the World Molecular Imaging Society, the combinations reside in spatially specific use and the requirements and are increasingly including aspects of the explicit regions of the tumors (Fig 2). procedures needed to achieve this level bedrock of quantitative imaging in their We have termed these regions habitats of measurement accuracy. More than guidelines. because they represent physiologically Radiology: Volume 278: Number 2—February 2016 radiology.rsna.org 567 SPECIAL REPORT: Radiomics Gillies et al distinct volumes, each with a specific while agnostic features are those that Table 2 combination of blood flow, cell density, attempt to capture lesion heterogeneity Examples of Semantic and Agnostic necrosis, and edema. Additional ra- through quantitative descriptors. Features of Radiomics diomic features can be extracted from Semantic features.—Although se- each of these habitats to obtain highly mantic features are commonly used by Semantic Agnostic granular descriptions of cancer lesions. radiologists to describe lesions, in this Size Histogram (skewness, The distribution of these habitats in pa- article we refer to their quantification kurtosis) tients with glioblastoma multiforme, for with computer assistance. With the Shape Haralick textures example, can enable us to discriminate foreknowledge that semantic features Location Laws textures between cancers that progress quickly are of prognostic value, early investiga- Vascularity Wavelets (,400 days of survival) and those that tions in radiomics developed radiology Spiculation Laplacian transforms are more indolent (37). Furthermore, lexicons, much the same as BI-RADS, Necrosis Minkowski functionals these habitats change after treatment PI-RADS, and Lung-RADS attempt to Attachments or Fractal dimensions (eg, treatment with radiation and tem- do. A watershed article in this regard lepidics azolamide), and the pattern of change came from Segal et al, who, in an early has been observed to be predictive of example of radiogenomics, used a finite response. series of radiologist-scored quantitative are referred to some excellent reviews features to predict gene expression pat- on the subject, with specific reference Segmentation terns in hepatocellular carcinoma (40). to intratumoral heterogeneity (42,43). Segmentation is the most critical, chal- This approach continues to have high Higher-order statistical methods im- lenging, and contentious component value, and there is a movement to cap- pose filter grids on the image to extract of radiomics. It is critical because the ture such semantic data with the aid of repetitive or nonrepetitive patterns. subsequent feature data are gener- computers to achieve higher interread- These include fractal analyses, wherein ated from the segmented volumes. It is er agreement, faster throughput, and patterns are imposed on the image and challenging because many tumors have lower variance. the number of grid elements containing indistinct borders. It is contentious be- Agnostic features.—Agnostic ra- voxels of a specified value is computed cause there are ongoing debates over diomic features on an image are mathe- (44); Minkowski functionals, which as- whether to seek ground truth or re- matically extracted quantitative descrip- sess patterns of voxels whose intensity is producibility and how much to rely on tors, which are generally not part of above a threshold (45); wavelets, which manual or automatic segmentation. the radiologists’ lexicon. These can be are filter transforms that multiply an However, a consensus is emerging that divided into first-, second-, or higher- image by a matrix of complex linear or truth is elusive and that optimum re- order statistical outputs. First-order sta- radial “waves”; and Laplacian transforms producible segmentation is achievable tistics describe the distribution of values of Gaussian bandpass filters that can ex - with computer-aided edge detection of individual voxels without concern for tract areas with increasingly coarse tex- followed by manual curation. It is well spatial relationships. These are generally ture patterns from the image (46). recognized that interoperator variabil- histogram-based methods and reduce a There has been a sustained effort ity of manually contoured tumors is region of interest to single values for to identify, define, and extract more high (38,39). Segmentation of normal mean, median, maximum, minimum, agnostic features. The first such study structures, such as skeletal elements and uniformity or randomness (entropy) used 182 texture features in combina- and organs, can now be achieved with of the intensities on the image, as well tion with 22 semantic features to de- full automation. However, any disease, as the skewness (asymmetry) and kurto- scribe CT images of lung cancer (24). especially cancer, requires operator in- sis (flatness) of the histogram of values. This was followed by a 442-member put because of inter- and intrasubject Second-order statistical descriptors feature set that also contained wavelets morphologic and contrast heterogene- generally are described as “texture” fea- (47). More recently, this has been ex- ity at the initial examination. tures; they describe statistical interre- panded to 662 features that also con- lationships between voxels with similar tain Laplace transforms of Gaussian Feature Extraction and Qualification (or dissimilar) contrast values. Texture fits (46) and 522 features that include The heart of radiomics is the extraction analysis of images was first introduced in texture and fractal dimension features of high-dimension feature data to quan- 1973 by Haralick et al (41). In radiomics, (48). These features potentially can titatively describe attributes of volumes texture analyses can readily provide a be extracted from individual habitats, of interest. In practice, “semantic” and measure of intratumoral heterogeneity. thereby yielding thousands of data ele- “agnostic” features are the two types In practice, there are dozens of methods ments with which to describe each vol- of features extracted in radiomics. and multiple variables that can be used ume of interest, with many volumes of (Table 2). Semantic features are those to extract texture features, resulting interest available in each patient. that are commonly used in the radiology in hundreds of values—far too many Thus, it is readily apparent that the lexicon to describe regions of interest, to elaborate on in this article. Readers number of descriptive image features 568 radiology.rsna.org Radiology: Volume 278: Number 2—February 2016 SPECIAL REPORT: Radiomics Gillies et al second-, and higher-order textures). The Figure 3 classifier models can then be built with the two or three highest-priority fea- tures in each class. In the final analysis, the value of feature sets is determined by their contribution to classifier models created through database mining. Building Databases: Numbers Are King, Quality Is Queen In radiomics and elsewhere, the power of the predictive classifier model is de - pendent on having sufficient data. It has been our experience that a reasonable rule of thumb is that 10 samples (pa- tients) are needed for each feature in a model based on binary classifiers. Fur - thermore, the best models are those that can accommodate additional clinical or genomic covariates, and this increases the need for large high-quality data sets. Radiomics can be performed with as few as 100 patients, although larger data sets provide more power. It is time con- suming to capture and curate large high- quality sets from retrospective data. For Figure 3: Covariance matrix of radiomic features. A total of 219 features example, in a recent study, we curated a were extracted from each non–small cell lung cancer tumor in 235 patients. data set of patients with non–small cell Across all tumors, each feature was individually compared with all other fea- tures by using regression analysis, thereby generating correlation coefficients lung cancer adenocarcinoma who had (R ). Individual features were then clustered and plotted along both axes, and gene expression profiles (46). Within a 2 2 R is shown as a heat map, with areas of high correlation (R . 0.95) shown local database, 285 such patients were in red. Thus, each of the red squares along the diagonal contains a group of readily identified as potential candidates features that are highly correlated with one another and are thus redundant. for such a cohort study. The need to For data analysis, one feature was chosen to be representative of each of these validate these via chart and pathology groups. The representative feature chosen was the one that had the highest review required 188 hours and resulted natural biologic range (interpatient variability) across the entire patient data set, in the loss of 50 patients from the study with the explicit assumption that features that show the highest interpatient cohort because of missing data or equiv- variability will be the most informative. (Image courtesy of Y. Balagurunathan.) ocal histopathologic findings. When his - topathologic findings were equivocal, can approach the complexity of data off-diagonal elements. Clusters of highly a pathologist reviewed the slides; this obtained with gene expression profiling, correlated features can be collapsed into only added to the curation time. Fur- which commonly yields information on one representative feature, usually the ther validation via picture archiving and more than 30 000 different sequences. one with the largest intersubject vari- communication system review of images With such large complexity, there is ability or highest dynamic range. Figure captured with standardized acquisition a danger of overfitting analyses, and 3 also provides a conceptual bridge to and reconstruction parameters required hence, dimensionality must be reduced the other -omics fields, where the data 94 hours and resulted in the attrition of by prioritizing the features (49,50). content of the images is indicated by the an additional 92 patients. Segmentation The most systematic approach is to false-color map. If available, test-retest and extraction of features into the data- first identify features that may be re - data are also extremely helpful, as they base required an additional 145 hours. dundant (ie, those that are highly cor- can help prioritize features on the basis Thus, the curation of a data set of 143 related with one another). Figure 3 is of their reproducibility (51,52). A fur- patients required an initial cohort of 285 a covariance matrix of 219 features ex- ther level of prioritization, described by patients (approximately 50% attrition) tracted from CT scans in 143 patients Aerts et al (47), is to rank order features and required 430 hours of processing, with non–small cell lung cancer. Those within separate categories representing or approximately 3 hours of process- features that are highly correlated (r . different agnostic and semantic classes ing per patient. As these patients were 0.95) with each other are shown as red of features (eg, size, shape, and first-, not filtered for medical or demographic Radiology: Volume 278: Number 2—February 2016 radiology.rsna.org 569 SPECIAL REPORT: Radiomics Gillies et al issues, there was no selection bias. In progression-free survival and disease- Furthermore, these analyses could be the future, capturing images and other free survival or recurrence; however, used to distinguish between two differ- data prospectively and with higher qual- these data are not readily available ent forms of Gleason score 7 disease ity and standards should reduce data and require a dedicated abstraction ef- (4+3 vs 3+4) with 92% accuracy (53). attrition and make the process more fort with chart review. Hence, there is Tumor Prognosis efficient. a pressing need to capture such data and to share data across institutions Seminal radiogenomic studies were the Classifier Modeling and Data Sharing to accumulate sufficient numbers for first to show a relationship between Once large high-quality and well-curat- statistical power. Such data sharing is quantitative image features and gene ed data sets are available, they can be a major initiative of the QIN, whose expression patterns in patients with used for data mining, which refers to members are committed to depositing cancer (40,54,55). In the first of these the process of discovering patterns in well-curated data sets into The Cancer studies, the investigators compared large data sets. This process can use Imaging Archive for public and private semantic radiologist-defined features artificial intelligence, machine learn - data mining efforts. extracted from contrast-enhanced CT ing, or statistical approaches. At one images in patients with hepatocellular end, these include both supervised carcinoma to gene expression patterns Examples of Radiomics Results and nonsupervised machine learning by using machine learning with a neu- approaches, such as neural networks, In the past 10 years, radiomics and ra- ral network. They found that combi- support vector machines, or Bayesian diogenomics research in tomographic nations of 28 imaging traits could be networks. Although these approaches imaging (CT, MR imaging, and PET) used to reconstruct 78% of the global use a priori knowledge through train- has increased dramatically. Two well- gene expression profiles, which in turn ing sets, they are agnostic in that they written and relatively recent reviews were linked to cell proliferation, liver make no assumptions about the mean- describe some of the advances through synthetic function, and patient prog- ing of the individual features. Hence, all 2014 (42,43). In the subsequent sec- nosis (40). In a subsequent study and features are treated with equal weight tion, we will highlight selected find - with a similar approach, the investiga- at the initiation of learning. At the ings, some of which are very recent, tors compared image features extracted other end of the data-mining spectrum that show the potential of radiomics to from MR images to predict global gene are hypothesis-driven approaches that substantially aid clinical care in several expression patterns in patients with cluster features according to predefined areas. glioblastoma multiforme (54). They information content. While both of found that an “infiltrative” imaging phe - Enabling Diagnosis these approaches have merit, the best notype was associated with significantly models are those that are tailored to In a study of 147 men with biopsy-prov- shorter survival (54). a specific medical context and, hence, en prostate cancer, Wibmer et al (6) In patients with lung cancer, there start out with a well-defined endpoint. showed that Haralick texture analysis is incontrovertible evidence for intratu- Ideally, robust models accommo- has the potential to enable differenti- moral heterogeneity on lung CT images date patient features beyond imaging. ation of cancerous from noncancerous (Fig 5). These heterogeneities can be Covariates include genomic profiles (ex - prostate tissue on both T2-weighted captured with features such as spicu- pression, mutation, polymorphisms), MR images and apparent diffusion co- lation or entropy gradients. Grove et histology, serum markers, patient his- efficient (ADC) maps derived from dif - al found these measures to be strong tories, and biomarkers that are quali- fusion-weighted MR images (Fig 4). In prognostic indicators in patients with fied for the specific-use case (Fig 1). In the peripheral zone of the prostate, all early-stage lung cancer (P , .01) (56). practice, not all information is available five features assessed (entropy, inertia, A study by Aerts et al (47) showed that for all patients; hence, models should energy, correlation, and homogeneity) a radiomic signature could be used to also be designed to accommodate differed significantly between benign predict outcome in completely inde- sparse data. As mentioned previously, and cancerous tissue on both types of pendent cohorts of patients with lung the power of the model is entirely de- images; however, in the transition zone, cancer from two separate institutions. pendent on the size and quality of the significant differences were found for Further, this same signature could be data within the database. Quality de- all five features on ADC maps and for applied to cohorts of patients with head pends not only on the image acquisi- two features (inertia and correlation) and neck cancer with equivalent prog- tion conditions but also on the avail- on T2-weighted images. In a follow-up nostic power. Notably, the signature ability and reliability of covariates. For study, these features were used to au- was comprised of the top features from example, overall survival is a common tomatically compute Gleason grade and four classes (size, shape, texture, and endpoint for many studies, but this were found to enable discrimination wavelets) that were prioritized from a includes death from all causes, which between cancers with a Gleason score database of 442 features by using test- may not be related to the disease being of 6 (3+3) and those with a Gleason retest reproducibility and intersubject studied. More exact endpoints include score of 7 of more with 93% accuracy. range. This study and others like it 570 radiology.rsna.org Radiology: Volume 278: Number 2—February 2016 SPECIAL REPORT: Radiomics Gillies et al seven were related to gene expression. Figure 4 When gene expression was assessed via pathways, approximately half of the im- aging features showed strong correla- tion to genomics. These analyses show that power for predicting gene expres- sion patterns, outcomes, and staging of gliomas can be significantly increased with radiomics-based approaches. Recently, Vignati et al performed a thorough prospective radiomic analysis of diffusion- and T2-weighted MR im- aging examinations in 49 patients with prostate cancer (58). Agnostic features extracted from T2-weighted images and ADC maps were compared with more traditional ADC cutoff metrics to test the hypothesis that textures could help differentiate between men with a path- ologic Gleason score of 6 and those with a pathologic Gleason score of 7 or higher. This is an important cut- off, as men with a pathologic Gleason score of 6 may be candidates for active surveillance. For standard ADC cutoff metrics, the area under the receiver operator characteristic curve ranged from 0.82 to 0.85. When ADC and T2 Figure 4: Application of texture analysis to T2-weighted MR images and ADC maps of pros- maps were analyzed for heterogeneity, tate cancer. A lesion in the transition zone is barely discernible on the T2-weighted image (top the area under the curve improved to left) and has higher conspicuity on the ADC map (top right). Texture features were computed on impressive values of 0.92 and 0.96, a per-voxel basis (using a 5 3 5 3 1 pixel window) from manually segmented regions of inter- respectively. Although this study may est identifying the normal peripheral zone (outlined in blue) and cancer (outlined in red). From have been underpowered, it shows the the computed texture features, a machine learning method was applied to distinguish between potential value of quantitative analysis normal and cancerous structures and to stratify the Gleason patterns. Heat map images show of tumor heterogeneity in assessing tu- clear differences between healthy tissue and cancer and depict intratumoral heterogeneity that mor aggressiveness and informing ma- may be useful in assessing tumor aggressiveness and informing fused MR imaging–ultrasonog- jor clinical decisions, such as whether raphy biopsy. to treat the cancer at all. Of note, other investigators have also found entropy demonstrate the potential of radiomics transfer coefficient maps from dynamic determined from ADC maps to be sig- for the identification of a general prog - contrast-enhanced MR images could nificantly associated with the pathologic nostic imaging phenotype existing in be used to distinguish high- and low- Gleason score, even after controlling several forms of cancer. grade gliomas with much higher sta- for the median ADC (6,53). It is well known in the radiology tistical power (P , .00005) than could Treatment Selection community that contrast enhancement median transfer coefficient maps alone at MR imaging is often heterogeneous, (P = .005). In a more recent study, Ge- In a seminal study, Kuo et al identified with complex patterns. In a landmark vaert et al extracted a large number of hepatocellular carcinoma imaging phe- article, Rose et al (44) analyzed the semantic and agnostic features in 55 notypes that correlated with a doxo- pattern of enhancement on dynamic patients with glioma who had under- rubicin drug response gene expression contrast-enhanced MR images in sim- gone gene expression profiling (57). program (55). Their results suggested ulations, phantoms, and 23 patients The feature set was then filtered for re - that radiogenomic analyses could be with glioma by using second-order and producibility, yielding 18 features that used to guide the selection of therapy higher statistical measures to repre- were assessed in three distinct habitats. for individual tumors. More recently, sent enhancement heterogeneity. They Of the agnostic features, most could be a study of 58 women who underwent convincingly showed that complex correlated with the semantic features; treatment for locally advanced breast measures of texture heterogeneity in three of 54 were related to survival, and cancer suggested that texture analysis Radiology: Volume 278: Number 2—February 2016 radiology.rsna.org 571 SPECIAL REPORT: Radiomics Gillies et al lack of standards for validating results, Figure 5 incomplete reporting of results, and un- recognized confounding variables in the databases used, particularly if data are derived retrospectively. Hence, as with any biomarker study, a retrospective radiomics investigation must be vali- dated against a completely independent data set, preferably from another insti- tution. Furthermore, the most rigorous biomarker qualification requires a pro - spective multicenter trial wherein the biomarker is one of the primary end- points (62,63). While standardized tools for geno- mic profiling (GenomeDx for prostate cancer [GenomeDx Biosciences, San Diego, Calif], Oncotype Dx for breast cancer [Genomic Health, Redwood City, Calif]) have been developed, they are not universally agreed upon or ap- plied across medical centers, hamper- ing efforts to share data and reproduce results. Studies have documented these Figure 5: Attenuation gradients of lung CT images. Data are representative of patients with problems in biomedical research gener- (left) and without (right) recurrence after lobectomy. Although differences in texture were visible ally and in molecular-targeted drug de- on the CT images (top), the color-coded attenuation maps (bottom) more dramatically show velopment specifically. A 2009 analysis intratumoral complexity. Maps were generated by separating the attenuation into quartiles, with of biomedical research reports found hotter colors representing higher attenuation. that at least 50% of studies were too poor, insufficient, or incomplete to be of dynamic contrast-enhanced MR im- better-informed decisions about where usable (64). Scientists at Bayer Health- aging could help predict response to to biopsy. Care (Leverkusen, Germany) reported neoadjuvant chemotherapy before its that they were able to successfully re- initiation (59). produce the published results from only Challenges for Radiomics a quarter of 67 seminal studies (65,66). Deciding Where to Biopsy or Resect In this article, we have already discussed Furthermore, when scientists at Amgen It is axiomatic that images can be used technical challenges to the individual (Thousand Oaks, Calif) tried to repli- to guide biopsy. It is our opinion that steps in the process of radiomics. We cate 53 landmark studies in the basic quantitative analyses of regionally dis- will now we present broader concerns science of cancer, they were able to re- tinct radiomic features can also pre- that arise from radiomics as a whole. produce the original results of just six cisely inform biopsy; that is, they can be (67). These issues have become serious Reproducibility used to identify a priori those locations enough that editors from more than 30 within complex tumors that are most Radiomics is a young discipline. As high-impact-factor biomedical journals likely to contain important diagnostic, with therapies motivated by molecular have united to impose common stan- prognostic, or predictive information. biology, radiomics offers great potential dards for statistical testing and to im- This has already been shown with the to accelerate precision medicine. How- prove access to raw data (68,69). The use of PET to overlay functional infor- ever, it is also possible that radiomics standards have been adopted by the mation on CT or MR images to better will undergo the same slow progress National Institutes of Health (8,70). Al- guide biopsies in the abdomen and in already experienced with molecular though these standards were generated patients with bone disease (60,61). biology–based systemic diagnostic tech- to address preclinical data, they can Figures 4, 6, and 7 show recent exam- niques and therapies. That slow pro- be applied across all areas of research ples of applications of radiomics to MR gress can be attributed to a number of and can provide a roadmap for navi- imaging, CT, and PET/CT in patients causes, including technical complexity, gating the complex issues associated with prostate, bladder, and metastatic poor study design (in particular, mix- with acquisition and analysis of high- breast cancer, respectively, and show ing hypothesis generation with hypo- dimensional data inherent in radiomics. the potential of radiomics to enable thesis testing) and overfitting of data, Superb reporting guidelines for clinical 572 radiology.rsna.org Radiology: Volume 278: Number 2—February 2016 SPECIAL REPORT: Radiomics Gillies et al studies have been developed by many Figure 6 organizations. An excellent overview is provided by the Equator network, which promotes the quality and trans- parency of health research (65). Chal- lenges with study design were also identified in the 2012 report Omics from the Institute of Medicine (8). A clear solution to these challenges is to establish benchmarks for the conduct of radiomics studies and for their re- porting in the literature. Big Data In the era of precision medicine, giga- bytes of data are collected for each pa- tient, and radiomics data can provide a significant component of this. The exponential growth in the numbers of patients and the data elements being harvested from each is known collo- quially as “big data”. Big data initiatives are aimed at drawing inferences from large data sets that are not derived from carefully controlled experiments. Although correlations among observa- tions can be vast in number and easy to obtain, causality is much harder to Figure 6: Application of texture analysis to CT images of bladder cancer. On original contrast-enhanced CT assess and establish, partly because image of bladder cancer (top left), a high-attenuation lesion is clearly visible, and there is some evidence of it is a vague and poorly specified con - intratumoral heterogeneity. However, when the texture features of energy (top right), entropy (bottom left), and struct for complex systems. Across big homogeneity (bottom right) are displayed over the source image, intratumoral heterogeneity can be readily data disciplines there are basic ques- appreciated. Other studies have shown that higher intratumoral heterogeneity is associated with a worse prognosis. tions: Will access to massive data be a key to understanding the fundamental Figure 7 questions of basic and applied science? Or, does the vast increase in data con- found analysis, produce computational bottlenecks, and decrease the ability to draw valid causal inferences? As in radiomics, the field of big data is in its early phases. The aforementioned questions were addressed in a meeting on big data that was sponsored by the National Academy of Sciences (71), and the radiomics field will benefit from this effort. Data Sharing The biggest challenge to establishing radiomics-based models as biomarkers to use in decision support is the shar- Figure 7: Application of radiomics to FDG-avid lymph nodes on PET and CT images in a patient with ing of image data and metadata across metastatic breast cancer. Left: Standard PET image shows there is little evidence of intranodal heterogeneity. multiple sites. Multisite trials are re- Right: CT image shows calculation and display of the Haralick co-occurrence statistics with a 9 3 9 3 9 quired to interrogate separate co- voxel matrix and clearly reveals some areas with lower co-occurrence (red), which have higher regional horts of patients and to create data- heterogeneity and would therefore be considered more suspicious for cancer. The results were used to select lymph nodes for image-informed biopsy. bases with sufficient size for statistical Radiology: Volume 278: Number 2—February 2016 radiology.rsna.org 573 SPECIAL REPORT: Radiomics Gillies et al power. Data sharing is a common chal- in which standards are lacking. It will solution is to capture data prospectively lenge in all biomedical research, and be necessary to provide standards for at the point of care. Hence, we envision it must overcome cultural, administra- all aspects of radiomics if the field is to a transition from classic radiology to a tive, regulatory, and personal issues realize its potential. new paradigm in which the radiologist (72). Notably, communities like the actively participates in the curation of Children’s Oncology Group have es- quantitative image databases. Collec- Radiomics: The Next Frontier in Clinical tablished a history and culture of data tion of high-quality image data requires Decision Making sharing (73) and therefore are in a sophisticated content expertise to iden- prime position to expand their efforts Our vision for radiomics is optimistic tity and circumscribe (with computer to include radiomic image analysis. and clear. In the foreseeable future, assistance) and annotate (with a stan- Data sharing in radiomics is especially we expect that data gleaned from ra- dardized and mineable lexicon) the vol- daunting because shared data must in- diologic examinations throughout the umes of interest. To make high-quality clude images and sharing must be in world will be converted into quantita- data curation a reality, we must first compliance with the Health Insurance tive feature data and that these data convince the imaging practitioners of Portability and Accountability Act, will be interfaced with knowledge ba- its value, and we must streamline the as a substantial amount of personal ses to improve diagnostic accuracy and process so it can occur within the lim- health information is needed to build predictive power for decision support. itations of a clinical practice. By play- models of sufficient complexity. Solu - For this to have high penetrance in ing a crucial role in data curation and tions to this challenge are many and clinical settings, practitioners must be analysis of big data, radiologists and can include: (a) large centralized data given an incentive to participate in the physicians alike will be able to make repositories, such as The Cancer Imag- process. How do we get there from radiomics an important, valuable new ing Archive and The Cancer Genome here? Clearly, part of the solution in- dimension of their field. Atlas, wherein access can be limited volves addressing the aforementioned Health Informatics to institutional review board–approved challenges of standardization and data users or data can be stripped of per- sharing. In addition, the data must be To be of maximal value, the various sonal health information; (b) federated collected prospectively. There are cen- kinds of high-quality data that are ob- approaches, wherein each institution tral and critical roles for radiologists tained during the work-up and monitor- maintains their individual data, and to play in identifying and curating data ing of individual patients must interface query models are sent to extract the at the front end and in applying classi- with each other. This is well recog- relevant metadata; or (c) federated ap- fier models at the user end to improve nized, and most large medical centers proaches, wherein the institutions are diagnostic and prognostic accuracy. In are now investing in appropriate elec- all accessible by an honest broker (ie, between, this will be a multidisciplinary tronic medical record systems to make a superuser with multisite institutional effort involving information technolo- patient data accessible in a mineable review board–approved access). No gists, bioinformaticists, statisticians, form. Currently, radiomic data are typ- matter which solution is applied, the and treating physicians. Here, we out- ically not incorporated as part of this infrastructure costs can be substantial. line what needs to be done at the local data stream; however, this is changing and transnational levels in short- and with the adoption of structured radiol- Standards intermediate-term time frames. ogy reporting. The challenge going for- While standards exist or are being de- ward will be to capture radiomic data Curation of High-Quality Data by veloped in many of the areas already as part of the structured report. Radiologists mentioned in this article, there are still Data Sharing gaps. For example, while the value of In current general practice, radiologic test-retest subject or patient image examinations are qualitatively evalu- As discussed previously, the quality of studies is well recognized, many of ated, and the generated reports often classifier models is limited by the size the published studies have small sam- do not use a standard lexicon, despite a of the data sets used to create them. ple sizes. Ideally, these studies should number of efforts to embrace a uniform Even if one institution were to capture be combined to provide a meta-anal- lexicon, such as RadLex® (74). (If used all of its radiomic data prospectively, it ysis; however, as noted earlier, there routinely, annotations with RadLex®- would be years before sufficient power are often problems with how results type image features could greatly con- could be generated. Additionally, these are reported. While there are guide- tribute to mineable databases [75,76].) data are a moving target, as there are lines for reporting, none yet exist for Furthermore, once images are ar- continuous improvements in medical the reporting of quantitative imaging chived, they are rarely reaccessed. image acquisition. Differences in im- results, let alone for the reporting of Although massive image repositories age acquisition and reconstruction are much more complex radiomics results. exist, they are virtually inaccessible covariates that must be incorporated in Testing of the core technology of tex- for curation because of the limitations the mining of quantitative image data ture analysis also is among the areas described previously. The only viable and therefore increase the amount of 574 radiology.rsna.org Radiology: Volume 278: Number 2—February 2016 SPECIAL REPORT: Radiomics Gillies et al therapy by evaluating patient responses to data required for model building by Radiomics Resources radiation treatment. Carcinogenesis 2015; an order of magnitude. The solution is Readers may find the following resources 36(3):307–317. for multi-institutional, national, or in- helpful: QIN, http://imaging.cancer.gov/ ternational consortia to agree to share 5. West CM, Barnett GC. Genetics and geno- programsandresources/specializedinitia- mics of radiotherapy toxicity: towards pre- data either through centralized or dis- tives/qin; QIBA, http://rsna.org/QIBA. diction. Genome Med 2011;3(8):52. tributed (federated) networks. These aspx; The Cancer Imaging Archive, challenges have been met and solved 6. Wibmer A, Hricak H, Gondo T, et al. Ha- http://www.cancerimagingarchive.net; ralick texture analysis of prostate MRI: in the basic sciences of gene expres- Food and Drug Administration CAD utility for differentiating non-cancerous sion, sequencing, and protein struc- Guidance, http://www.fda.gov/Regula- prostate from prostate cancer and differenti- ture databases. They are beginning to toryInformation/Guidances/ucm187249. ating prostate cancers with different Gleason be solved through sharing of oncology htm; and National Institutes of Health scores. Eur Radiol 2015;25(10):2840–2850. metadata (www.oriencancer.org or the Principles and Standards of Research 7. Coroller TP, Grossmann P, Hou Y, et al. CT- Children’s Oncology Group), and some Reporting, http://www.nih.gov/about/ based radiomic signature predicts distant sharing of image files has been enabled reporting-preclinical-research.htm. metastasis in lung adenocarcinoma. Radio- by The Cancer Imaging Archive (77). ther Oncol 2015;114(3):345–350. Acknowledgments: This article was an out- To date, collaborative efforts to growth of the 2014 Radiological Society of North 8. Committee on the Review of Omics-Based develop quantitative image-driven bio- America/American Association of Physicists in Tests for Predicting Patient Outcomes in markers have been remarkable. Ra- Medicine Plenary Session. The authors thank Clinical Trials, Board on Health Care Ser- the following colleagues for their substantial in- diomics has tremendous potential to vices, Board on Health Sciences Policy, Insti- put: Daniel Seeburg, MD (Johns Hopkins Uni- further enrich image interpretations tute of Medicine. Evolution of Translational versity), for his timely review of an early draft and to expand the horizons of imaging Omics: Lessons Learned and the Path For- of the manuscript; Olya Stringfield, PhD (Moffitt ward. Micheel CM, Nass SJ, Omenn GS, eds. toward greater precision and extrac- Cancer Center), for production of images; Rob- ert A. Gatenby, MD (Moffitt Cancer Center), Washington, DC: National Academies Press. tion of in vivo biologic information. To for providing images, inspiration. and edits; fully exploit the potential of radiomics, 9. Bates SE. It’s all about the test: the com- Yoganand Balagurunathan, PhD (Moffitt Cancer we need to embrace an interdisciplin- plexity of companion diagnostic co-develop- Center), for his review of the manuscript; Sandy ment in personalized medicine. Clin Cancer Napel, PhD (Stanford University), for providing ary shared vision and make a joint a comprehensive list of relevant references; and Res 2014;20(6):1418. commitment. Ada Muellner, MS (Memorial Sloan-Kettering 10. Collins FS, Varmus H. A new initiative on pre- Cancer Center), for her spectacular editing. The Radiology Reading Room of the Future cision medicine. N Engl J Med 2015;372(9): We also acknowledge funding and support from 793–795. RSNA’s sponsorship of QIBA and the NCI’s spon- The aforementioned scenario will en- sorship of QIN. tail a reading room wherein practic- 11. Office of the Press Secretary. Fact sheet: ing radiologists interact with picture Disclosures of Conflicts of Interest: R.J.G. Ac- President Obama’s precision medicine tivities related to the present article: none to initiative. The White House Web site. archiving and communication system disclose. Activities not related to the present http://www.whitehouse.gov/the-press- software to identify, segment, and ex- article: is on the advisory board of and is an in- office/2015/01/30/fact-sheet-president- tract features from regions of interest. vestor in Health Myne. Other relationships: has obama-s-precision-medicine-initiative. Up- a patent pending for Radiology Reading Room If prior studies obtained in the same dated January 30, 2015. Accessed April 4, of the Future. P.E.K. Activities related to the patient are available, the previous re- present article: none to disclose. Activities not gions of interest will be automatically related to the present article: received a grant 12. Siegel R, Ma J, Zou Z, Jemal A. Cancer identified by the reading software. As from GE Healthcare, is the cofounder of PET/X. statistics, 2014. CA Cancer J Clin 2014; Other relationships: none to disclose. H.H. dis- part of the reading, the extracted size, 64(1):9–29. closed no relevant relationships. shape, location, and textural features 13. Yachida S, Jones S, Bozic I, et al. Distant will be automatically uploaded to a metastasis occurs late during the genetic shared database and algorithmically References evolution of pancreatic cancer. Nature 2010; compared with prior images to enable 467(7319):1114–1117. 1. Institute of Medicine I. Evolution of Transla- more precise diagnoses. Such capa- tional Omics. Lessons learned and the path 14. Gerlinger M, Rowan AJ, Horswell S, et al. bilities are nearly at hand, as most forward. Washington, DC: National Acad- Intratumor heterogeneity and branched evo- emies Press, 2012. picture archiving and communication lution revealed by multiregion sequencing. N systems have the capability to coreg- Engl J Med 2012;366(10):883–892. 2. Giger ML. Update on the potential of com- ister current images with prior images puter-aided diagnosis for breast cancer. Fu- 15. Sottoriva A, Spiteri I, Piccirillo SG, et al. and perform user-interactive segmen- ture Oncol 2010;6(1):1–4. Intratumor heterogeneity in human glioblas- tation. For the foreseeable future, toma reflects cancer evolutionary dynamics. 3. Doi K. Computer-aided diagnosis in medi- the field of radiomics research will be Proc Natl Acad Sci U S A 2013;110(10): cal imaging: historical review, current status 4009–4014. concentrated on improving classifier and future potential. Comput Med Imaging Graph 2007;31(4-5):198–211. models to provide the most accurate 16. Sequist LV, Martins RG, Spigel D, et al. possible diagnoses and, hence, better 4. Guo Z, Shu Y, Zhou H, Zhang W, Wang H. First-line gefitinib in patients with advanced patient care and outcomes. non-small-cell lung cancer harboring so- Radiogenomics helps to achieve personalized Radiology: Volume 278: Number 2—February 2016 radiology.rsna.org 575 SPECIAL REPORT: Radiomics Gillies et al matic EGFR mutations. J Clin Oncol 2008; Historical Perspective and Planned Goals. 41. Haralick RM, Shanmugam K, Dinstein I. 26(15):2442–2449. Transl Oncol 2014;7(1):1–4. Textural features for image classification. IEEE Trans Syst Man Cybern 1973;SMC- 17. Burstein HJ, Harris LN, Marcom PK, et al. 29. Buckler AJ, Bresolin L, Dunnick NR, Sul- 3(6):610–621. Trastuzumab and vinorelbine as first-line livan DC; for the Group. A collaborative en- therapy for HER2-overexpressing metastatic terprise for multi-stakeholder participation 42. Davnall F, Yip CS, Ljungqvist G, et al. As- breast cancer: multicenter phase II trial in the advancement of quantitative imaging. sessment of tumor heterogeneity: an emerg- with clinical outcomes, analysis of serum Radiology 2011;258(3):906–914. ing imaging tool for clinical practice? Insights tumor markers as predictive factors, and Imaging 2012;3(6):573–589. 30. Kinahan P, Wahl R, Shao L, Frank R, Perl- cardiac surveillance algorithm. J Clin Oncol man E. The QIBA profile for quantitative 43. O’Connor JP, Rose CJ, Waterton JC, Cara- 2003;21(15):2889–2895. FDG-PET/CT oncology imaging [abstr]. J no RA, Parker GJ, Jackson A. Imaging in- 18. Wagle N, Emery C, Berger MF, et al. Dis- Nucl Med 2014;55(Suppl 1):1520. tratumor heterogeneity: role in therapy secting therapeutic resistance to RAF inhi- response, resistance, and clinical outcome. 31. FDG-PET/CT Technical Committee. FDG- bition in melanoma by tumor genomic pro- Clin Cancer Res 2015;21(2):249–257. PET/CT as an Imaging Biomarker Measuring filing. J Clin Oncol 2011;29(22):3085–3096. Response to Cancer Therapy, Quantitative 44. Rose CJ, Mills SJ, O’Connor JP, et al. Quan- 19. Sequist LV, Waltman BA, Dias-Santagata Imaging Biomarkers Alliance, Version 1.05, tifying spatial heterogeneity in dynamic con- D, et al. Genotypic and histological evolu- Publicly Reviewed Version. RSNA Web site. trast-enhanced MRI parameter maps. Magn tion of lung cancers acquiring resistance to http://www.rsna.org/qiba/. Published De- Reson Med 2009;62(2):488–499. EGFR inhibitors. Sci Transl Med 2011;3(75): cember 11, 2013. Accessed May 20, 2015. 45. Larkin TJ, Canuto HC, Kettunen MI, et al. 75ra26. 32. van Persijn van Meerten EL, Gelderblom Analysis of image heterogeneity using 2D 20. Clauson J, Hsieh YC, Acharya S, Rademak- H, Bloem JL. RECIST revised: implications Minkowski functionals detects tumor re- er AW, Morrow M. Results of the Lynn Sage for the radiologist: a review article on the sponses to treatment. Magn Reson Med 2014; Second-Opinion Program for local therapy modified RECIST guideline. Eur Radiol 71(1):402–410. in patients with breast carcinoma: changes 2010;20(6):1456–1467. 46. Grossmann P, Grove O, El-Hachem N, et in management and determinants of where 33. Burnside ES, Sickles EA, Bassett LW, et al. Identification of molecular phenotypes care is delivered. Cancer 2002;94(4):889– al. The ACR BI-RADS experience: learning in lung cancer by integrating radiomics and from history. J Am Coll Radiol 2009;6(12): genomics. Sci Transl Med (in press). 21. Nguyen PL, Schultz D, Renshaw AA, et al. 851–860. 47. Aerts HJ, Velazquez ER, Leijenaar RT, et al. The impact of pathology review on treat- 34. Bomers JG, Barentsz JO. Standardization Decoding tumour phenotype by noninvasive ment recommendations for patients with of multiparametric prostate MR imaging imaging using a quantitative radiomics ap- adenocarcinoma of the prostate. Urol Oncol using PI-RADS. BioMed Res Int 2014;2014: proach. Nat Commun 2014;5:4006. 2004;22(4):295–299. 48. Pickup L, Talwar A, Stalin S, et al. Lung 22. Staradub VL, Messenger KA, Hao N, Wiley 35. Kazerooni EA, Armstrong MR, Amorosa JK, nodule classification using learnt texture EL, Morrow M. Changes in breast cancer et al. ACR CT accreditation program and the features on a single patient population [ab- therapy because of pathology second opin- lung cancer screening program designation. J str]. In: Radiological Society of North Amer- ions. Ann Surg Oncol 2002;9(10):982–987. Am Coll Radiol 2015;12(1):38–42. ica Scientific Assembly and Annual Meeting 23. Kronz JD, Westra WH, Epstein JI. Mandatory Program. Oak Brook, Ill: Radiological Soci- 36. Gatenby RA, Grove O, Gillies RJ. Quantita- second opinion surgical pathology at a large ety of North America, 2015; SSM06. tive imaging in cancer evolution and ecology. referral hospital. Cancer 1999;86(11): Radiology 2013;269(1):8–15. 49. Schabath MB, Massion PP, Thompson ZJ, 2426–2435. et al. Behavior of incidence lung cancers in 37. Zhou M, Hall L, Goldgof D, et al. Radiolog- 24. Kumar V, Gu Y, Basu S, et al. Radiomics: the CT arm of the National Lung Screening ically defined ecological dynamics and clin - the process and the challenges. Magn Reson Trial. JAMA (in press). ical outcomes in glioblastoma multiforme: Imaging 2012;30(9):1234–1248. preliminary results. Transl Oncol 2014;7(1): 50. Balagurunathan Y, Kumar V, Gu Y, et al. Test- 25. Lambin P, Rios-Velazquez E, Leijenaar R, et 5–13. retest reproducibility analysis of lung CT al. Radiomics: extracting more information image features. J Digit Imaging 2014;27(6): 38. Rios Velazquez E, Aerts HJ, Gu Y, et al. A from medical images using advanced feature 805–823. semiautomatic CT-based ensemble segmenta- analysis. Eur J Cancer 2012;48(4):441–446. tion of lung tumors: comparison with oncolo- 51. Lin LI. A concordance correlation coefficient 26. Clarke LP, Sriram RD, Schilling LB. Imaging gists’ delineations and with the surgical speci- to evaluate reproducibility. Biometrics 1989; as a Biomarker: Standards for Change Mea- men. Radiother Oncol 2012;105(2):167–173. 45(1):255–268. surements in Therapy workshop summary. 39. van Dam IE, van Sörnsen de Koste JR, 52. Carrasco JL, King TS, Chinchilli VM. The Acad Radiol 2008;15(4):501–530. Hanna GG, Muirhead R, Slotman BJ, Senan concordance correlation coefficient for re - 27. Shankar LK, Hoffman JM, Bacharach S, et S. Improving target delineation on 4-dimen- peated measures estimated by variance al. Consensus recommendations for the use sional CT scans in stage I NSCLC using a de- components. J Biopharm Stat 2009;19(1): of 18F-FDG PET as an indicator of therapeu- formable registration tool. Radiother Oncol 90–105. tic response in patients in National Cancer 2010;96(1):67–72. 53. Fehr D, Veeraraghavan H, Wibmer A, et al. Institute Trials. J Nucl Med 2006;47(6):1059– 40. Segal E, Sirlin CB, Ooi C, et al. Decoding Automatic classification of prostate cancer global gene expression programs in liver Gleason scores from multi-parametric mag- 28. Clarke LP, Nordstrom RJ, Zhang H, et al. cancer by noninvasive imaging. Nat Biotech- netic resonance images. Proc Natl Acad Sci The Quantitative Imaging Network: NCI’s nol 2007;25(6):675–680. USA (in press). 576 radiology.rsna.org Radiology: Volume 278: Number 2—February 2016 SPECIAL REPORT: Radiomics Gillies et al 54. Diehn M, Nardini C, Wang DS, et al. Identifi - ical impact. Eur J Nucl Med Mol Imaging 70. Landis SC, Amara SG, Asadullah K, et al. cation of noninvasive imaging surrogates for 2010;37(11):2027–2036. A call for transparent reporting to optimize brain tumor gene-expression modules. Proc the predictive value of preclinical research. 61. Tatli S, Gerbaudo VH, Mamede M, Tun- Natl Acad Sci U S A 2008;105(13):5213– Nature 2012;490(7419):187–191. cali K, Shyn PB, Silverman SG. Abdominal masses sampled at PET/CT-guided percuta- 71. Drawing Causal Inference from Big Data. neous biopsy: initial experience with regis- 55. Kuo MD, Gollub J, Sirlin CB, Ooi C, Chen Arthur M. Sackler Colloquia of the National tration of prior PET/CT images. Radiology X. Radiogenomic analysis to identify im- Academy of Sciences. https://www.youtube. 2010;256(1):305–311. aging phenotypes associated with drug re- com/playlist?list=PLGJm1x3XQeK0NgFO sponse gene expression programs in hepa- X2Z7Wt-P5RU5Zv0Hv. Published April 2, 62. Simon RM, Paik S, Hayes DF. Use of ar- tocellular carcinoma. J Vasc Interv Radiol 2015. Accessed May 5, 2015. chived specimens in evaluation of prognos- 2007;18(7):821–831. tic and predictive biomarkers. J Natl Cancer 72. Nelson B. Data sharing: empty archives. Na- Inst 2009;101(21):1446–1452. 56. Grove O, Berglund AE, Schabath MB, et ture 2009;461(7261):160–163. al. Quantitative computed tomographic de- 63. Hayes DF, Bast RC, Desch CE, et al. Tumor 73. Devidas M, London WB, Anderson JR. The scriptors associate tumor shape complexity marker utility grading system: a framework use of central laboratories and remote elec- and intratumor heterogeneity with prog- to evaluate clinical utility of tumor markers. tronic data capture to risk-adjust therapy for nosis in lung adenocarcinoma. PLoS One J Natl Cancer Inst 1996;88(20):1456–1466. pediatric acute lymphoblastic leukemia and 2015;10(3):e0118261. neuroblastoma. Semin Oncol 2010;37(1):53– 64. Chalmers I, Glasziou P. Avoidable waste in 57. Gevaert O, Mitchell LA, Achrol AS, et al. 59. the production and reporting of research ev- Glioblastoma multiforme: exploratory ra- idence. Lancet 2009;374(9683):86–89. 74. Heilbrun ME. Evaluating RadLex and real diogenomic analysis by using quantitative world radiology reporting: are we there yet? 65. Simera I, Moher D, Hirst A, Hoey J, Schulz image features. Radiology 2014;273(1):168– Acad Radiol 2013;20(11):1327–1328. KF, Altman DG. Transparent and accurate reporting increases reliability, utility, and 75. Kahn CE Jr. Annotation of figures from the 58. Vignati A, Mazzetti S, Giannini V, et al. Tex- impact of your research: reporting guide- biomedical imaging literature: a comparative ture features on T2-weighted magnetic res- lines and the EQUATOR Network. BMC analysis of RadLex and other standardized onance imaging: new potential biomarkers Med 2010;8:24. vocabularies. Acad Radiol 2014;21(3):384– for prostate cancer aggressiveness. Phys 66. Prinz F, Schlange T, Asadullah K. Believe it Med Biol 2015;60(7):2685–2701. or not: how much can we rely on published 76. Mabotuwana T, Lee MC, Cohen-Solal EV, 59. Teruel JRHM, Heldahl MG, Goa PE, et al. data on potential drug targets? Nat Rev Chang P. Mapping institution-specific study Dynamic contrast-enhanced MRI texture Drug Discov 2011;10(9):712. descriptions to RadLex Playbook entries. J analysis for pretreatment prediction of clin- 67. Begley CG, Ellis LM. Drug development: Digit Imaging 2014;27(3):321–330. ical and pathological response to neoadju- Raise standards for preclinical cancer re- vant chemotherapy in patients with locally 77. Clark K, Vendt B, Smith K, et al. The Can- search. Nature 2012;483(7391):531–533. advanced breast cancer. NMR Biomed 2014; cer Imaging Archive (TCIA): maintaining 27(8):887–896. 68. Journals unite for reproducibility. Nature and operating a public information reposi- 2014;515(7525):7. tory. J Digit Imaging 2013;26(6):1045–1057. 60. Klaeser B, Wiskirchen J, Wartenberg J, et al. PET/CT-guided biopsies of metabolically 69. McNutt M. Journals unite for reproducibil- active bone lesions: applications and clin- ity. Science 2014;346(6210):679. Radiology: Volume 278: Number 2—February 2016 radiology.rsna.org 577 http://www.deepdyve.com/assets/images/DeepDyve-Logo-lg.png Radiology Pubmed Central

Radiomics: Images Are More than Pictures, They Are Data

Gillies, Robert J.; Kinahan, Paul E.; Hricak, Hedvig
Radiology , Volume 278 (2) – Nov 18, 2015
Download PDF
15 pages

Loading next page...
 
/lp/pubmed-central/radiomics-images-are-more-than-pictures-they-are-data-eMsUcJ077N

References (86)

Publisher
Pubmed Central
Copyright
2015 by the Radiological Society of North America, Inc.
ISSN
0033-8419
eISSN
1527-1315
DOI
10.1148/radiol.2015151169
Publisher site
See Article on Publisher Site

Abstract

Original r esearch Special Repo R This copy is for personal use only. To order printed copies, contact [email protected] r adiomics: Images Are More than Pictures, They Are Data Robert J. Gillies, PhD In the past decade, the field of medical image analysis has Paul E. Kinahan, PhD grown exponentially, with an increased number of pattern Hedvig Hricak, MD, PhD, Dr(hc) recognition tools and an increase in data set sizes. These advances have facilitated the development of processes for high-throughput extraction of quantitative features that result in the conversion of images into mineable data and the subsequent analysis of these data for decision support; this practice is termed radiomics. This is in contrast to the traditional practice of treating medical images as pictures intended solely for visual interpretation. Radiomic data contain first-, second-, and higher-order statistics. These data are combined with other patient data and are mined with sophisticated bioinformatics tools to develop models that may potentially improve diagnostic, prognostic, and predictive accuracy. Because radiomics analyses are in- tended to be conducted with standard of care images, it is conceivable that conversion of digital images to mineable data will eventually become routine practice. This report describes the process of radiomics, its challenges, and its potential power to facilitate better clinical decision mak- ing, particularly in the care of patients with cancer. From the Department of Cancer Imaging, H. Lee Moffitt Cancer Center and Research Institute, 12902 Magnolia Dr, Tampa, FL 33612 (R.J.G.); Department of Radiology, University of Washington, Seattle, Wash (P.E.K.); and Department of Radiology, Memorial Sloan-Kettering Cancer Center, New York NY 10065 (H.H.). Received May 29, 2015; revision requested July 6; revision received August 14; accepted September 14; final version accepted October 16. Address correspondence to R.J.G. (e-mail: Robert. [email protected] ). Radiology: Volume 278: Number 2—February 2016 n radiology.rsna.org 563 SPECIAL REPORT: Radiomics Gillies et al ith high-throughput computing, subsequently mine the data for hypo- assessment of prognosis, prediction of it is now possible to rapidly thesis generation, testing, or both. Ra- response to treatment, and monitoring Wextract innumerable quantita- diomics is designed to develop decision of disease status. tive features from tomographic images support tools; therefore, it involves The mining of radiomic data to (computed tomography [CT], magnetic combining radiomic data with other detect correlations with genomic pat- resonance [MR], or positron emission patient characteristics, as available, to terns is known as radiogenomics, and tomography [PET] images). The con- increase the power of the decision sup- it has elicited especially great interest version of digital medical images into port models. As radiomics is intended in the research community. To avoid mineable high-dimensional data, a to extract maximal information from confusion, it should be noted that the process that is known as radiomics, is standard of care images, the creation of term radiogenomics is also used in the motivated by the concept that biomed- databases that combine vast quantities field of radiation oncology to describe ical images contain information that of radiomics data (and ideally other whole-genome analyses aimed at de- reflects underlying pathophysiology and complementary data) from millions of termining the genetic causes of varia- that these relationships can be revealed patients is foreseeable. tions in radiosensitivity (4,5). Hence- via quantitative image analyses. Al- Although radiomics can be applied forward in this article, we will refer to though radiomics is a natural extension to a large number of conditions, it is radiogenomics only as the combination of computer-aided diagnosis and detec- most well developed in oncology be- of radiomic features with genomic data tion (CAD) systems, it is significantly cause of support from the National for the purpose of enabling decision different from them. CAD systems are Cancer Institute (NCI) Quantitative support. The value of radiogenomics usually standalone systems that are Imaging Network (QIN) and other ini- stems from the fact that while virtu- designated by the Food and Drug Ad- tiatives from the NCI Cancer Imaging ally all patients with cancer undergo ministration for use in either the detec- Program. As described in subsequent imaging at some point and often mul- tion or diagnosis of disease (1). Early sections of this article, the potential of tiple times during their care, not all of successes of CAD have been greatest radiomics to contribute to decision sup- them have their disease genomically in breast cancer imaging (2,3). Un- port in oncology has grown as knowl- profiled. Furthermore, when genomic like CAD systems, which are directed edge and analytic tools have evolved. profiling is performed, it is done one toward delivering a single answer (ie, Quantitative image features based on time at one location and is susceptible presence of a lesion or cancer), ra- intensity, shape, size or volume, and diomics is explicitly a process designed texture offer information on tumor phe- to extract a large number of quantita- notype and microenvironment (or hab- Published online before print tive features from digital images, place itat) that is distinct from that provided 10.1148/radiol.2015151169 Content codes: these data in shared databases, and by clinical reports, laboratory test re- Radiology 2016; 278:563–577 sults, and genomic or proteomic assays. Abbreviations: These features, in conjunction with the Advances in Knowledge ADC = apparent diffusion coefficient other information, can be correlated BI-RADS = Breast Imaging Reporting and Data System n Radiomics is defined as the con - with clinical outcomes data and used CAD = computer-aided diagnosis and detection version of images to higher- for evidence-based clinical decision Lung-RADS = Lung Imaging Reporting and Data System dimensional data and the subse- support (Fig 1). Radiomics appears NCI = National Cancer Institute quent mining of these data for to offer a nearly limitless supply of PI-RADS = Prostate Imaging Reporting and Data System improved decision support. QIBA = Quantitative Imaging Biomarkers Alliance imaging biomarkers that could poten- QIN = Quantitative Imaging Network tially aid cancer detection, diagnosis, n Radiomics has been initiated in RSNA = Radiological Society of North America oncology studies, but it is poten- Implications for Patient Care tially applicable to all diseases. Author contributions: Guarantor of integrity of entire study, R.J.G.; study n Radiomics can be performed with n Radiomics is designed to be used concepts/study design or data acquisition or data analysis/ tomographic images from CT, in decision support of precision interpretation, all authors; manuscript drafting or manu- MR imaging, and PET studies. medicine. script revision for important intellectual content, all authors; approval of final version of submitted manuscript, all au- n Image features are extracted n Although relationships between thors; agrees to ensure any questions related to the work from volumes of interest, which radiomics and outcome are are appropriately resolved, all authors; literature research, can be either entire tumors or defined with populations, they all authors; and manuscript editing, all authors defined subvolumes within can be applied to individual tumors, known as habitats. patients. Funding: This research was supported by the National Insti- n Radiomics is a new field, and n Radiomic analysis promises to tutes of Health (grants U54CA143970, U01CA143062, there are substantial challenges increase precision in diagnosis, R01CA190105, and R01CA187532). to its implementation in a clinical assessment of prognosis, and setting. prediction of therapy response. Conflicts of interest are listed at the end of this article. 564 radiology.rsna.org Radiology: Volume 278: Number 2—February 2016 SPECIAL REPORT: Radiomics Gillies et al Figure 1 Figure 1: Flowchart shows the process of radiomics and the use of radiomics in decision support. Patient work-up requires information from disparate sources to be combined into a coherent model to describe where the lesion is, what it is, and what it is doing. Radiomics begins with acquisition of high-quality images. From these images, a region of interest (ROI) that contains either the whole tumor or subregions (ie, habitats) within the tumor can be identified. These are segmented with operator edits and are eventually rendered in three dimensions (3D). Quantitative features are extracted from these rendered volumes to generate a report, which is placed in a database along with other data, such as clinical and genomic data. These data are then mined to develop diagnostic, predictive, or prognostic models for outcomes of interest. to sampling error. Thus, radiogenom- and prognosis. For example, research on multiple levels: One multisection ics has two potential uses, which will has already shown the capacity of ra- or three-dimensional image from one be described in detail in the Examples diomics analyses to help distinguish patient may easily contain millions of of Radiomics Results section. First, a prostate cancer from benign prostate voxels. Also, one tumor (or other ab- subset of the radiomic data can be used tissue or add information about pros- normal entity) may contain hundreds to suggest gene expression or mutation tate cancer aggressiveness (6). In the of measurable features describing size, status that potentially warrants further evaluation of lung cancer and in the shape, and texture. testing. This is important because the evaluation of glioblastoma multiforme, Radiomics analyses epitomize the radiomic data are derived from the radiomics has been shown to be a tool pursuit of precision medicine, in which entire tumor (or tumors) rather than with which to assess patient prognosis molecular and other biomarkers are from just a sample. Thus, radiomics (7). The tools developed for radiomics used to predict the right treatment can provide important information re- can help in daily clinical work, and for the right patient at the right time. garding the sample genomics and can radiologists can play a pivotal role in The availability of robust and validated be used for cross-validation. Second, continuously building the databases biomarkers is essential to move preci- a subset of radiomic features is not that are to be used for future decision sion medicine forward (9). Around the significantly related to gene expression support. world, efforts are underway to improve or mutational data and, hence, has the The suffix -omics is a term that orig- the availability of such biomarkers, potential to provide additional, inde- inated in molecular biology disciplines and in the United States, the effort is pendent information. The combination to describe the detailed characteriza- most notably through The Precision of this subset of radiomic features with tion of biologic molecules such as DNA Medicine Initiative (10,11). This ini- genomic data may increase diagnostic, (genomics), RNA (transcriptomics), tiative will provide funding for a new prognostic, and predictive power. proteins (proteomics), and metabolites model of patient-powered research While radiomics primarily grew (metabolomics). Now, the term is also that promises to accelerate biomedical out of basic research, lately it has also being used in other medical research discoveries and provide clinicians with elicited interest from those in clinical fields that generate complex high-di - new tools, knowledge, and therapies research, as well as those in daily clin- mensional data from single objects or that enable more precise personalized ical practice. For a clinical radiologist, samples (8). One desirable character- care. radiomics has the potential to help istic of -omics data is that these data A major strength of a radiomics with the diagnosis of both common and are mineable and, as such, can be used approach for cancer is that digital ra- rare tumors. Visualization of tumor for exploration and hypothesis genera- diologic images are obtained for almost heterogeneity may prove critical in the tion. The -omics concept readily applies every patient with cancer, and all of assessment of tumor aggressiveness to quantitative tomographic imaging these images are potential sources for Radiology: Volume 278: Number 2—February 2016 radiology.rsna.org 565 SPECIAL REPORT: Radiomics Gillies et al tumor biology. A central hypothesis driv- each with its own challenges (24,25). Table 1 ing radiomics research is that radiomics These steps are shown in Figure 1 Important Aspects of Radiomics has the potential to enable quantitative and include: (a) acquiring the images, measurement of intra- and intertumoral (b) identifying the volumes of interest Aspect heterogeneity. Moreover, radiomics of- (ie, those that may contain prognostic Uses standard-of-care images fers the possibility of longitudinal use in value), (c) segmenting the volumes (ie, Interrogates the entire tumor treatment monitoring and optimization delineating the borders of the volume Can be used to interrogate stroma or in active surveillance. Although such with computer-assisted contouring), (d) Enables longitudinal monitoring applications of radiomics have yet to be extracting and qualifying descriptive fea- explored in depth, they may provide the tures from the volume, (e) using these to most value going forward. populate a searchable database, and (f) radiomics databases (Table 1). In the It should be emphasized that ra- mining these data to develop classifier United States alone, there are approx- diomic and radiogenomic analyses can models to predict outcomes either alone imately 1.6 million new cancer cases be used to identify correlations, not or in combination with additional infor- every year (12). Most of these patients causes; thus, they are not expected to mation, such as demographic, clinical, will undergo multiple CT, MR imaging, enable definitive assessment of genetic comorbidity, or genomic data. We will and PET examinations. In the future, or other contents of tissue through im- discuss these processes in turn. it is possible that image interpretation aging alone. However, correlation of ra- Image Acquisition for all these studies will be augmented diomic data with genomic or other -omic by using radiomics, building an unprec- data could inform not only the decision Modern CT, MR imaging, and com- edented source of big data that will about whether to test for certain gene bined PET/CT units allow for wide expand the potential for discovering alterations in biopsy samples but also variations in acquisition and image re- helpful correlations. While radiomics the choice of biopsy sites. It also could construction protocols, and standardi- will allow better characterization of pa- provide confirmatory information to zation of these protocols across medi- tients and their diseases through new support histopathologic findings. This cal imaging centers is typically lacking. applications of genomics and improved is important, as it is estimated that the This is generally not a problem in the methods of phenotyping, it will also add error rate of cancer histopathology can routine identification of radiologic fea - to the challenges of data management, be as high as 23% (20–23). Errors in tures used in clinical practice. However, as we will discuss later in this article. histopathology are due to both sampling when images are analyzed numerically Radiomics offers important advan- errors and observer variability; thus, to extract meaningful data, variations tages for assessment of tumor biology. there is a great need for additional quan- in acquisition and image reconstruction It is now appreciated that most clinically titative diagnostic information. parameters can introduce changes that relevant solid tumors are highly hetero- We believe that radiomics is rapidly are not due to underlying biologic ef- geneous at the phenotypic, physiologic, expanding beyond a boutique research fects. This has been well recognized in and genomic levels (13–15) and that area and is emerging as a translational the emerging field of quantitative imag - they continue to evolve over time. In this technology. Hence, this is an appropri- ing, in which the intent is to generate emerging era of targeted therapies, it ate time to begin to establish bench- medical images with describable limits is notable that most responses are not marks for data extraction, analysis, of bias and variance. In other words, it durable and that benefit is generally and presentation. The goal of this is not sufficient to report a number or a measured in months, not years. For ex- report is to introduce the practice of set of numbers derived from images; in- ample, this is the case with (a) gefitinib radiomics to a wide audience of prac- stead, we must also be able to provide in patients with epidermal growth factor ticing clinicians, including radiologists, error bars, as is done with every other receptor–mutated lung cancer (16), (b) to engage a broader community in es- credible laboratory measurement. trastuzamab in those with human epider- tablishing benchmarks. In doing so, we There have been multiple efforts to mal growth factor receptor 2 (or HER2) will describe the processes involved in advance quantitative imaging, includ- overexpressing breast cancer (17), and radiomics and the unique information ing definition of acquisition and recon - (c) vemurafenib in those with B-Raf–mu- it offers, as well as its challenges and struction standards, over the past 15 tated melanoma (18). Genomic hetero- their potential solutions. We will also years (26,27). The QIN is a cooperative geneity within tumors and across meta- highlight some of the more recent find - network initiated by the NCI with the static tumor sites in the same patient is ings of importance and, finally, offer a goal of developing quantitative imaging the major cause of treatment failure and vision for radiomics of the future. methods that improve the effectiveness emergence of therapy resistance (19). of clinical trials of new cancer ther- Thus, precision medicine requires not apies (28). The QIN is a major initia- Process of Radiomics only in vitro biomarkers and companion tive from the NCI and can be regarded diagnostics but also spatially and tem- While conceptually simple, the practice as the leading edge of new imaging porally resolved in vivo biomarkers of of radiomics involves discrete steps, methods, including radiomics. Also, the 566 radiology.rsna.org Radiology: Volume 278: Number 2—February 2016 SPECIAL REPORT: Radiomics Gillies et al Volume of Interest Identification Figure 2 Identification of tissue volumes of prog - nostic value is the core of the practice of radiology in oncology. Although at the time of diagnosis cancer can be detect- ed at one tumor site or multiple tumor sites, most patients with cancer metas- tasis have multiple lesions. In either sce- nario, we need to identify tumors and suspected tumors as volumes of interest. However, detailed analysis of subvolumes within tumors (the manifestations of tu- mor heterogeneity) that may have prog- nostic value generally are not captured in a radiology report because of the spatial and contrast limitations of digital images. While heterogeneity is not included in Response Evaluation Criteria in Solid Tumors, version 1.1 (32), a few texture descriptors have been incorporated in more complex diagnostic imaging report- ing and data systems, such as the Breast Imaging Reporting and Data System (BI-RADS) (33), the Prostate Imaging Reporting and Data System (PI-RADS) (34), and the Lung Imaging Reporting and Data System (Lung-RADS) (35). In Figure 2: Habitats in a patient with glioblastoma multiforme. Habitats were the practice of radiomics, so-called sub- defined by combining unenhanced and contrast-enhanced T1-weighted, 120- volumes of interest can be captured and msec echo time T2-weighted, and fluid-attenuated inversion recovery (FLAIR) added to the analyses. The basic philoso- images. Data from each acquisition were sorted into low and high values phy, which has its foundation in process with automated histogram analyses, yielding a potential for eight different engineering, is to capture as much data combinations. In practice, only four distinct combinations were observed. They as possible at the front end and use correspond to the red (low T1, high T2 and FLAIR), yellow (low T1 and T2, high FLAIR), blue (High T1 and FLAIR, low T2), and green (high T1, low FLAIR and T2) downstream database mining to identify areas. Notably, while the identities of individual voxels were determined math- the features with the highest prognostic ematically, they spatially clustered into contiguous regions reflecting different value. This is driven by the knowledge physiologic microenvironments. (Image courtesy of R. A. Gatenby.) that attempting to filter the data at input would be inefficient and would presup - Radiological Society of North America 100 participants were involved in the pose knowledge regarding the value of (RSNA) and the National Institute for creation of the initial QIBA fluorode - the features in classifier models before Biomedical Imaging and Bioengineer- oxyglucose PET/CT profile, which was they were tested. ing have sponsored the Quantitative released in 2014 (30,31). The American Recently, the concept of using image Imaging Biomarkers Alliance (QIBA), Association of Physicists in Medicine is data to identify physiologically distinct which is a major effort in quantitative providing technical guidelines in quanti- regions within lesions has been de- imaging (29). The goal of QIBA is to tative imaging in the form of modality- scribed (36). In this approach, images industrialize quantitative imaging by dependent reports on imager operation with different acquisition parameters bringing together the entire spectrum and testing. Finally, relevant profes- (eg, contrast material–enhanced T1- of groups involved in its development sional societies, such as the American weighted MR imaging, diffusion-weight- and implementation. The main product College of Radiology, RSNA, the Soci- ed, and fluid attenuation sequences) can of QIBA is a new type of a document ety of Nuclear Medicine and Molecular be combined to yield regions with spe- termed a profile that provides a con- Imaging, the International Society of cific combinations of quantitative image sensus on the measurement accuracy of Magnetic Resonance in Medicine, and data. Notably, when this is performed, a quantitative imaging biomarker for a the World Molecular Imaging Society, the combinations reside in spatially specific use and the requirements and are increasingly including aspects of the explicit regions of the tumors (Fig 2). procedures needed to achieve this level bedrock of quantitative imaging in their We have termed these regions habitats of measurement accuracy. More than guidelines. because they represent physiologically Radiology: Volume 278: Number 2—February 2016 radiology.rsna.org 567 SPECIAL REPORT: Radiomics Gillies et al distinct volumes, each with a specific while agnostic features are those that Table 2 combination of blood flow, cell density, attempt to capture lesion heterogeneity Examples of Semantic and Agnostic necrosis, and edema. Additional ra- through quantitative descriptors. Features of Radiomics diomic features can be extracted from Semantic features.—Although se- each of these habitats to obtain highly mantic features are commonly used by Semantic Agnostic granular descriptions of cancer lesions. radiologists to describe lesions, in this Size Histogram (skewness, The distribution of these habitats in pa- article we refer to their quantification kurtosis) tients with glioblastoma multiforme, for with computer assistance. With the Shape Haralick textures example, can enable us to discriminate foreknowledge that semantic features Location Laws textures between cancers that progress quickly are of prognostic value, early investiga- Vascularity Wavelets (,400 days of survival) and those that tions in radiomics developed radiology Spiculation Laplacian transforms are more indolent (37). Furthermore, lexicons, much the same as BI-RADS, Necrosis Minkowski functionals these habitats change after treatment PI-RADS, and Lung-RADS attempt to Attachments or Fractal dimensions (eg, treatment with radiation and tem- do. A watershed article in this regard lepidics azolamide), and the pattern of change came from Segal et al, who, in an early has been observed to be predictive of example of radiogenomics, used a finite response. series of radiologist-scored quantitative are referred to some excellent reviews features to predict gene expression pat- on the subject, with specific reference Segmentation terns in hepatocellular carcinoma (40). to intratumoral heterogeneity (42,43). Segmentation is the most critical, chal- This approach continues to have high Higher-order statistical methods im- lenging, and contentious component value, and there is a movement to cap- pose filter grids on the image to extract of radiomics. It is critical because the ture such semantic data with the aid of repetitive or nonrepetitive patterns. subsequent feature data are gener- computers to achieve higher interread- These include fractal analyses, wherein ated from the segmented volumes. It is er agreement, faster throughput, and patterns are imposed on the image and challenging because many tumors have lower variance. the number of grid elements containing indistinct borders. It is contentious be- Agnostic features.—Agnostic ra- voxels of a specified value is computed cause there are ongoing debates over diomic features on an image are mathe- (44); Minkowski functionals, which as- whether to seek ground truth or re- matically extracted quantitative descrip- sess patterns of voxels whose intensity is producibility and how much to rely on tors, which are generally not part of above a threshold (45); wavelets, which manual or automatic segmentation. the radiologists’ lexicon. These can be are filter transforms that multiply an However, a consensus is emerging that divided into first-, second-, or higher- image by a matrix of complex linear or truth is elusive and that optimum re- order statistical outputs. First-order sta- radial “waves”; and Laplacian transforms producible segmentation is achievable tistics describe the distribution of values of Gaussian bandpass filters that can ex - with computer-aided edge detection of individual voxels without concern for tract areas with increasingly coarse tex- followed by manual curation. It is well spatial relationships. These are generally ture patterns from the image (46). recognized that interoperator variabil- histogram-based methods and reduce a There has been a sustained effort ity of manually contoured tumors is region of interest to single values for to identify, define, and extract more high (38,39). Segmentation of normal mean, median, maximum, minimum, agnostic features. The first such study structures, such as skeletal elements and uniformity or randomness (entropy) used 182 texture features in combina- and organs, can now be achieved with of the intensities on the image, as well tion with 22 semantic features to de- full automation. However, any disease, as the skewness (asymmetry) and kurto- scribe CT images of lung cancer (24). especially cancer, requires operator in- sis (flatness) of the histogram of values. This was followed by a 442-member put because of inter- and intrasubject Second-order statistical descriptors feature set that also contained wavelets morphologic and contrast heterogene- generally are described as “texture” fea- (47). More recently, this has been ex- ity at the initial examination. tures; they describe statistical interre- panded to 662 features that also con- lationships between voxels with similar tain Laplace transforms of Gaussian Feature Extraction and Qualification (or dissimilar) contrast values. Texture fits (46) and 522 features that include The heart of radiomics is the extraction analysis of images was first introduced in texture and fractal dimension features of high-dimension feature data to quan- 1973 by Haralick et al (41). In radiomics, (48). These features potentially can titatively describe attributes of volumes texture analyses can readily provide a be extracted from individual habitats, of interest. In practice, “semantic” and measure of intratumoral heterogeneity. thereby yielding thousands of data ele- “agnostic” features are the two types In practice, there are dozens of methods ments with which to describe each vol- of features extracted in radiomics. and multiple variables that can be used ume of interest, with many volumes of (Table 2). Semantic features are those to extract texture features, resulting interest available in each patient. that are commonly used in the radiology in hundreds of values—far too many Thus, it is readily apparent that the lexicon to describe regions of interest, to elaborate on in this article. Readers number of descriptive image features 568 radiology.rsna.org Radiology: Volume 278: Number 2—February 2016 SPECIAL REPORT: Radiomics Gillies et al second-, and higher-order textures). The Figure 3 classifier models can then be built with the two or three highest-priority fea- tures in each class. In the final analysis, the value of feature sets is determined by their contribution to classifier models created through database mining. Building Databases: Numbers Are King, Quality Is Queen In radiomics and elsewhere, the power of the predictive classifier model is de - pendent on having sufficient data. It has been our experience that a reasonable rule of thumb is that 10 samples (pa- tients) are needed for each feature in a model based on binary classifiers. Fur - thermore, the best models are those that can accommodate additional clinical or genomic covariates, and this increases the need for large high-quality data sets. Radiomics can be performed with as few as 100 patients, although larger data sets provide more power. It is time con- suming to capture and curate large high- quality sets from retrospective data. For Figure 3: Covariance matrix of radiomic features. A total of 219 features example, in a recent study, we curated a were extracted from each non–small cell lung cancer tumor in 235 patients. data set of patients with non–small cell Across all tumors, each feature was individually compared with all other fea- tures by using regression analysis, thereby generating correlation coefficients lung cancer adenocarcinoma who had (R ). Individual features were then clustered and plotted along both axes, and gene expression profiles (46). Within a 2 2 R is shown as a heat map, with areas of high correlation (R . 0.95) shown local database, 285 such patients were in red. Thus, each of the red squares along the diagonal contains a group of readily identified as potential candidates features that are highly correlated with one another and are thus redundant. for such a cohort study. The need to For data analysis, one feature was chosen to be representative of each of these validate these via chart and pathology groups. The representative feature chosen was the one that had the highest review required 188 hours and resulted natural biologic range (interpatient variability) across the entire patient data set, in the loss of 50 patients from the study with the explicit assumption that features that show the highest interpatient cohort because of missing data or equiv- variability will be the most informative. (Image courtesy of Y. Balagurunathan.) ocal histopathologic findings. When his - topathologic findings were equivocal, can approach the complexity of data off-diagonal elements. Clusters of highly a pathologist reviewed the slides; this obtained with gene expression profiling, correlated features can be collapsed into only added to the curation time. Fur- which commonly yields information on one representative feature, usually the ther validation via picture archiving and more than 30 000 different sequences. one with the largest intersubject vari- communication system review of images With such large complexity, there is ability or highest dynamic range. Figure captured with standardized acquisition a danger of overfitting analyses, and 3 also provides a conceptual bridge to and reconstruction parameters required hence, dimensionality must be reduced the other -omics fields, where the data 94 hours and resulted in the attrition of by prioritizing the features (49,50). content of the images is indicated by the an additional 92 patients. Segmentation The most systematic approach is to false-color map. If available, test-retest and extraction of features into the data- first identify features that may be re - data are also extremely helpful, as they base required an additional 145 hours. dundant (ie, those that are highly cor- can help prioritize features on the basis Thus, the curation of a data set of 143 related with one another). Figure 3 is of their reproducibility (51,52). A fur- patients required an initial cohort of 285 a covariance matrix of 219 features ex- ther level of prioritization, described by patients (approximately 50% attrition) tracted from CT scans in 143 patients Aerts et al (47), is to rank order features and required 430 hours of processing, with non–small cell lung cancer. Those within separate categories representing or approximately 3 hours of process- features that are highly correlated (r . different agnostic and semantic classes ing per patient. As these patients were 0.95) with each other are shown as red of features (eg, size, shape, and first-, not filtered for medical or demographic Radiology: Volume 278: Number 2—February 2016 radiology.rsna.org 569 SPECIAL REPORT: Radiomics Gillies et al issues, there was no selection bias. In progression-free survival and disease- Furthermore, these analyses could be the future, capturing images and other free survival or recurrence; however, used to distinguish between two differ- data prospectively and with higher qual- these data are not readily available ent forms of Gleason score 7 disease ity and standards should reduce data and require a dedicated abstraction ef- (4+3 vs 3+4) with 92% accuracy (53). attrition and make the process more fort with chart review. Hence, there is Tumor Prognosis efficient. a pressing need to capture such data and to share data across institutions Seminal radiogenomic studies were the Classifier Modeling and Data Sharing to accumulate sufficient numbers for first to show a relationship between Once large high-quality and well-curat- statistical power. Such data sharing is quantitative image features and gene ed data sets are available, they can be a major initiative of the QIN, whose expression patterns in patients with used for data mining, which refers to members are committed to depositing cancer (40,54,55). In the first of these the process of discovering patterns in well-curated data sets into The Cancer studies, the investigators compared large data sets. This process can use Imaging Archive for public and private semantic radiologist-defined features artificial intelligence, machine learn - data mining efforts. extracted from contrast-enhanced CT ing, or statistical approaches. At one images in patients with hepatocellular end, these include both supervised carcinoma to gene expression patterns Examples of Radiomics Results and nonsupervised machine learning by using machine learning with a neu- approaches, such as neural networks, In the past 10 years, radiomics and ra- ral network. They found that combi- support vector machines, or Bayesian diogenomics research in tomographic nations of 28 imaging traits could be networks. Although these approaches imaging (CT, MR imaging, and PET) used to reconstruct 78% of the global use a priori knowledge through train- has increased dramatically. Two well- gene expression profiles, which in turn ing sets, they are agnostic in that they written and relatively recent reviews were linked to cell proliferation, liver make no assumptions about the mean- describe some of the advances through synthetic function, and patient prog- ing of the individual features. Hence, all 2014 (42,43). In the subsequent sec- nosis (40). In a subsequent study and features are treated with equal weight tion, we will highlight selected find - with a similar approach, the investiga- at the initiation of learning. At the ings, some of which are very recent, tors compared image features extracted other end of the data-mining spectrum that show the potential of radiomics to from MR images to predict global gene are hypothesis-driven approaches that substantially aid clinical care in several expression patterns in patients with cluster features according to predefined areas. glioblastoma multiforme (54). They information content. While both of found that an “infiltrative” imaging phe - Enabling Diagnosis these approaches have merit, the best notype was associated with significantly models are those that are tailored to In a study of 147 men with biopsy-prov- shorter survival (54). a specific medical context and, hence, en prostate cancer, Wibmer et al (6) In patients with lung cancer, there start out with a well-defined endpoint. showed that Haralick texture analysis is incontrovertible evidence for intratu- Ideally, robust models accommo- has the potential to enable differenti- moral heterogeneity on lung CT images date patient features beyond imaging. ation of cancerous from noncancerous (Fig 5). These heterogeneities can be Covariates include genomic profiles (ex - prostate tissue on both T2-weighted captured with features such as spicu- pression, mutation, polymorphisms), MR images and apparent diffusion co- lation or entropy gradients. Grove et histology, serum markers, patient his- efficient (ADC) maps derived from dif - al found these measures to be strong tories, and biomarkers that are quali- fusion-weighted MR images (Fig 4). In prognostic indicators in patients with fied for the specific-use case (Fig 1). In the peripheral zone of the prostate, all early-stage lung cancer (P , .01) (56). practice, not all information is available five features assessed (entropy, inertia, A study by Aerts et al (47) showed that for all patients; hence, models should energy, correlation, and homogeneity) a radiomic signature could be used to also be designed to accommodate differed significantly between benign predict outcome in completely inde- sparse data. As mentioned previously, and cancerous tissue on both types of pendent cohorts of patients with lung the power of the model is entirely de- images; however, in the transition zone, cancer from two separate institutions. pendent on the size and quality of the significant differences were found for Further, this same signature could be data within the database. Quality de- all five features on ADC maps and for applied to cohorts of patients with head pends not only on the image acquisi- two features (inertia and correlation) and neck cancer with equivalent prog- tion conditions but also on the avail- on T2-weighted images. In a follow-up nostic power. Notably, the signature ability and reliability of covariates. For study, these features were used to au- was comprised of the top features from example, overall survival is a common tomatically compute Gleason grade and four classes (size, shape, texture, and endpoint for many studies, but this were found to enable discrimination wavelets) that were prioritized from a includes death from all causes, which between cancers with a Gleason score database of 442 features by using test- may not be related to the disease being of 6 (3+3) and those with a Gleason retest reproducibility and intersubject studied. More exact endpoints include score of 7 of more with 93% accuracy. range. This study and others like it 570 radiology.rsna.org Radiology: Volume 278: Number 2—February 2016 SPECIAL REPORT: Radiomics Gillies et al seven were related to gene expression. Figure 4 When gene expression was assessed via pathways, approximately half of the im- aging features showed strong correla- tion to genomics. These analyses show that power for predicting gene expres- sion patterns, outcomes, and staging of gliomas can be significantly increased with radiomics-based approaches. Recently, Vignati et al performed a thorough prospective radiomic analysis of diffusion- and T2-weighted MR im- aging examinations in 49 patients with prostate cancer (58). Agnostic features extracted from T2-weighted images and ADC maps were compared with more traditional ADC cutoff metrics to test the hypothesis that textures could help differentiate between men with a path- ologic Gleason score of 6 and those with a pathologic Gleason score of 7 or higher. This is an important cut- off, as men with a pathologic Gleason score of 6 may be candidates for active surveillance. For standard ADC cutoff metrics, the area under the receiver operator characteristic curve ranged from 0.82 to 0.85. When ADC and T2 Figure 4: Application of texture analysis to T2-weighted MR images and ADC maps of pros- maps were analyzed for heterogeneity, tate cancer. A lesion in the transition zone is barely discernible on the T2-weighted image (top the area under the curve improved to left) and has higher conspicuity on the ADC map (top right). Texture features were computed on impressive values of 0.92 and 0.96, a per-voxel basis (using a 5 3 5 3 1 pixel window) from manually segmented regions of inter- respectively. Although this study may est identifying the normal peripheral zone (outlined in blue) and cancer (outlined in red). From have been underpowered, it shows the the computed texture features, a machine learning method was applied to distinguish between potential value of quantitative analysis normal and cancerous structures and to stratify the Gleason patterns. Heat map images show of tumor heterogeneity in assessing tu- clear differences between healthy tissue and cancer and depict intratumoral heterogeneity that mor aggressiveness and informing ma- may be useful in assessing tumor aggressiveness and informing fused MR imaging–ultrasonog- jor clinical decisions, such as whether raphy biopsy. to treat the cancer at all. Of note, other investigators have also found entropy demonstrate the potential of radiomics transfer coefficient maps from dynamic determined from ADC maps to be sig- for the identification of a general prog - contrast-enhanced MR images could nificantly associated with the pathologic nostic imaging phenotype existing in be used to distinguish high- and low- Gleason score, even after controlling several forms of cancer. grade gliomas with much higher sta- for the median ADC (6,53). It is well known in the radiology tistical power (P , .00005) than could Treatment Selection community that contrast enhancement median transfer coefficient maps alone at MR imaging is often heterogeneous, (P = .005). In a more recent study, Ge- In a seminal study, Kuo et al identified with complex patterns. In a landmark vaert et al extracted a large number of hepatocellular carcinoma imaging phe- article, Rose et al (44) analyzed the semantic and agnostic features in 55 notypes that correlated with a doxo- pattern of enhancement on dynamic patients with glioma who had under- rubicin drug response gene expression contrast-enhanced MR images in sim- gone gene expression profiling (57). program (55). Their results suggested ulations, phantoms, and 23 patients The feature set was then filtered for re - that radiogenomic analyses could be with glioma by using second-order and producibility, yielding 18 features that used to guide the selection of therapy higher statistical measures to repre- were assessed in three distinct habitats. for individual tumors. More recently, sent enhancement heterogeneity. They Of the agnostic features, most could be a study of 58 women who underwent convincingly showed that complex correlated with the semantic features; treatment for locally advanced breast measures of texture heterogeneity in three of 54 were related to survival, and cancer suggested that texture analysis Radiology: Volume 278: Number 2—February 2016 radiology.rsna.org 571 SPECIAL REPORT: Radiomics Gillies et al lack of standards for validating results, Figure 5 incomplete reporting of results, and un- recognized confounding variables in the databases used, particularly if data are derived retrospectively. Hence, as with any biomarker study, a retrospective radiomics investigation must be vali- dated against a completely independent data set, preferably from another insti- tution. Furthermore, the most rigorous biomarker qualification requires a pro - spective multicenter trial wherein the biomarker is one of the primary end- points (62,63). While standardized tools for geno- mic profiling (GenomeDx for prostate cancer [GenomeDx Biosciences, San Diego, Calif], Oncotype Dx for breast cancer [Genomic Health, Redwood City, Calif]) have been developed, they are not universally agreed upon or ap- plied across medical centers, hamper- ing efforts to share data and reproduce results. Studies have documented these Figure 5: Attenuation gradients of lung CT images. Data are representative of patients with problems in biomedical research gener- (left) and without (right) recurrence after lobectomy. Although differences in texture were visible ally and in molecular-targeted drug de- on the CT images (top), the color-coded attenuation maps (bottom) more dramatically show velopment specifically. A 2009 analysis intratumoral complexity. Maps were generated by separating the attenuation into quartiles, with of biomedical research reports found hotter colors representing higher attenuation. that at least 50% of studies were too poor, insufficient, or incomplete to be of dynamic contrast-enhanced MR im- better-informed decisions about where usable (64). Scientists at Bayer Health- aging could help predict response to to biopsy. Care (Leverkusen, Germany) reported neoadjuvant chemotherapy before its that they were able to successfully re- initiation (59). produce the published results from only Challenges for Radiomics a quarter of 67 seminal studies (65,66). Deciding Where to Biopsy or Resect In this article, we have already discussed Furthermore, when scientists at Amgen It is axiomatic that images can be used technical challenges to the individual (Thousand Oaks, Calif) tried to repli- to guide biopsy. It is our opinion that steps in the process of radiomics. We cate 53 landmark studies in the basic quantitative analyses of regionally dis- will now we present broader concerns science of cancer, they were able to re- tinct radiomic features can also pre- that arise from radiomics as a whole. produce the original results of just six cisely inform biopsy; that is, they can be (67). These issues have become serious Reproducibility used to identify a priori those locations enough that editors from more than 30 within complex tumors that are most Radiomics is a young discipline. As high-impact-factor biomedical journals likely to contain important diagnostic, with therapies motivated by molecular have united to impose common stan- prognostic, or predictive information. biology, radiomics offers great potential dards for statistical testing and to im- This has already been shown with the to accelerate precision medicine. How- prove access to raw data (68,69). The use of PET to overlay functional infor- ever, it is also possible that radiomics standards have been adopted by the mation on CT or MR images to better will undergo the same slow progress National Institutes of Health (8,70). Al- guide biopsies in the abdomen and in already experienced with molecular though these standards were generated patients with bone disease (60,61). biology–based systemic diagnostic tech- to address preclinical data, they can Figures 4, 6, and 7 show recent exam- niques and therapies. That slow pro- be applied across all areas of research ples of applications of radiomics to MR gress can be attributed to a number of and can provide a roadmap for navi- imaging, CT, and PET/CT in patients causes, including technical complexity, gating the complex issues associated with prostate, bladder, and metastatic poor study design (in particular, mix- with acquisition and analysis of high- breast cancer, respectively, and show ing hypothesis generation with hypo- dimensional data inherent in radiomics. the potential of radiomics to enable thesis testing) and overfitting of data, Superb reporting guidelines for clinical 572 radiology.rsna.org Radiology: Volume 278: Number 2—February 2016 SPECIAL REPORT: Radiomics Gillies et al studies have been developed by many Figure 6 organizations. An excellent overview is provided by the Equator network, which promotes the quality and trans- parency of health research (65). Chal- lenges with study design were also identified in the 2012 report Omics from the Institute of Medicine (8). A clear solution to these challenges is to establish benchmarks for the conduct of radiomics studies and for their re- porting in the literature. Big Data In the era of precision medicine, giga- bytes of data are collected for each pa- tient, and radiomics data can provide a significant component of this. The exponential growth in the numbers of patients and the data elements being harvested from each is known collo- quially as “big data”. Big data initiatives are aimed at drawing inferences from large data sets that are not derived from carefully controlled experiments. Although correlations among observa- tions can be vast in number and easy to obtain, causality is much harder to Figure 6: Application of texture analysis to CT images of bladder cancer. On original contrast-enhanced CT assess and establish, partly because image of bladder cancer (top left), a high-attenuation lesion is clearly visible, and there is some evidence of it is a vague and poorly specified con - intratumoral heterogeneity. However, when the texture features of energy (top right), entropy (bottom left), and struct for complex systems. Across big homogeneity (bottom right) are displayed over the source image, intratumoral heterogeneity can be readily data disciplines there are basic ques- appreciated. Other studies have shown that higher intratumoral heterogeneity is associated with a worse prognosis. tions: Will access to massive data be a key to understanding the fundamental Figure 7 questions of basic and applied science? Or, does the vast increase in data con- found analysis, produce computational bottlenecks, and decrease the ability to draw valid causal inferences? As in radiomics, the field of big data is in its early phases. The aforementioned questions were addressed in a meeting on big data that was sponsored by the National Academy of Sciences (71), and the radiomics field will benefit from this effort. Data Sharing The biggest challenge to establishing radiomics-based models as biomarkers to use in decision support is the shar- Figure 7: Application of radiomics to FDG-avid lymph nodes on PET and CT images in a patient with ing of image data and metadata across metastatic breast cancer. Left: Standard PET image shows there is little evidence of intranodal heterogeneity. multiple sites. Multisite trials are re- Right: CT image shows calculation and display of the Haralick co-occurrence statistics with a 9 3 9 3 9 quired to interrogate separate co- voxel matrix and clearly reveals some areas with lower co-occurrence (red), which have higher regional horts of patients and to create data- heterogeneity and would therefore be considered more suspicious for cancer. The results were used to select lymph nodes for image-informed biopsy. bases with sufficient size for statistical Radiology: Volume 278: Number 2—February 2016 radiology.rsna.org 573 SPECIAL REPORT: Radiomics Gillies et al power. Data sharing is a common chal- in which standards are lacking. It will solution is to capture data prospectively lenge in all biomedical research, and be necessary to provide standards for at the point of care. Hence, we envision it must overcome cultural, administra- all aspects of radiomics if the field is to a transition from classic radiology to a tive, regulatory, and personal issues realize its potential. new paradigm in which the radiologist (72). Notably, communities like the actively participates in the curation of Children’s Oncology Group have es- quantitative image databases. Collec- Radiomics: The Next Frontier in Clinical tablished a history and culture of data tion of high-quality image data requires Decision Making sharing (73) and therefore are in a sophisticated content expertise to iden- prime position to expand their efforts Our vision for radiomics is optimistic tity and circumscribe (with computer to include radiomic image analysis. and clear. In the foreseeable future, assistance) and annotate (with a stan- Data sharing in radiomics is especially we expect that data gleaned from ra- dardized and mineable lexicon) the vol- daunting because shared data must in- diologic examinations throughout the umes of interest. To make high-quality clude images and sharing must be in world will be converted into quantita- data curation a reality, we must first compliance with the Health Insurance tive feature data and that these data convince the imaging practitioners of Portability and Accountability Act, will be interfaced with knowledge ba- its value, and we must streamline the as a substantial amount of personal ses to improve diagnostic accuracy and process so it can occur within the lim- health information is needed to build predictive power for decision support. itations of a clinical practice. By play- models of sufficient complexity. Solu - For this to have high penetrance in ing a crucial role in data curation and tions to this challenge are many and clinical settings, practitioners must be analysis of big data, radiologists and can include: (a) large centralized data given an incentive to participate in the physicians alike will be able to make repositories, such as The Cancer Imag- process. How do we get there from radiomics an important, valuable new ing Archive and The Cancer Genome here? Clearly, part of the solution in- dimension of their field. Atlas, wherein access can be limited volves addressing the aforementioned Health Informatics to institutional review board–approved challenges of standardization and data users or data can be stripped of per- sharing. In addition, the data must be To be of maximal value, the various sonal health information; (b) federated collected prospectively. There are cen- kinds of high-quality data that are ob- approaches, wherein each institution tral and critical roles for radiologists tained during the work-up and monitor- maintains their individual data, and to play in identifying and curating data ing of individual patients must interface query models are sent to extract the at the front end and in applying classi- with each other. This is well recog- relevant metadata; or (c) federated ap- fier models at the user end to improve nized, and most large medical centers proaches, wherein the institutions are diagnostic and prognostic accuracy. In are now investing in appropriate elec- all accessible by an honest broker (ie, between, this will be a multidisciplinary tronic medical record systems to make a superuser with multisite institutional effort involving information technolo- patient data accessible in a mineable review board–approved access). No gists, bioinformaticists, statisticians, form. Currently, radiomic data are typ- matter which solution is applied, the and treating physicians. Here, we out- ically not incorporated as part of this infrastructure costs can be substantial. line what needs to be done at the local data stream; however, this is changing and transnational levels in short- and with the adoption of structured radiol- Standards intermediate-term time frames. ogy reporting. The challenge going for- While standards exist or are being de- ward will be to capture radiomic data Curation of High-Quality Data by veloped in many of the areas already as part of the structured report. Radiologists mentioned in this article, there are still Data Sharing gaps. For example, while the value of In current general practice, radiologic test-retest subject or patient image examinations are qualitatively evalu- As discussed previously, the quality of studies is well recognized, many of ated, and the generated reports often classifier models is limited by the size the published studies have small sam- do not use a standard lexicon, despite a of the data sets used to create them. ple sizes. Ideally, these studies should number of efforts to embrace a uniform Even if one institution were to capture be combined to provide a meta-anal- lexicon, such as RadLex® (74). (If used all of its radiomic data prospectively, it ysis; however, as noted earlier, there routinely, annotations with RadLex®- would be years before sufficient power are often problems with how results type image features could greatly con- could be generated. Additionally, these are reported. While there are guide- tribute to mineable databases [75,76].) data are a moving target, as there are lines for reporting, none yet exist for Furthermore, once images are ar- continuous improvements in medical the reporting of quantitative imaging chived, they are rarely reaccessed. image acquisition. Differences in im- results, let alone for the reporting of Although massive image repositories age acquisition and reconstruction are much more complex radiomics results. exist, they are virtually inaccessible covariates that must be incorporated in Testing of the core technology of tex- for curation because of the limitations the mining of quantitative image data ture analysis also is among the areas described previously. The only viable and therefore increase the amount of 574 radiology.rsna.org Radiology: Volume 278: Number 2—February 2016 SPECIAL REPORT: Radiomics Gillies et al therapy by evaluating patient responses to data required for model building by Radiomics Resources radiation treatment. Carcinogenesis 2015; an order of magnitude. The solution is Readers may find the following resources 36(3):307–317. for multi-institutional, national, or in- helpful: QIN, http://imaging.cancer.gov/ ternational consortia to agree to share 5. West CM, Barnett GC. Genetics and geno- programsandresources/specializedinitia- mics of radiotherapy toxicity: towards pre- data either through centralized or dis- tives/qin; QIBA, http://rsna.org/QIBA. diction. Genome Med 2011;3(8):52. tributed (federated) networks. These aspx; The Cancer Imaging Archive, challenges have been met and solved 6. Wibmer A, Hricak H, Gondo T, et al. Ha- http://www.cancerimagingarchive.net; ralick texture analysis of prostate MRI: in the basic sciences of gene expres- Food and Drug Administration CAD utility for differentiating non-cancerous sion, sequencing, and protein struc- Guidance, http://www.fda.gov/Regula- prostate from prostate cancer and differenti- ture databases. They are beginning to toryInformation/Guidances/ucm187249. ating prostate cancers with different Gleason be solved through sharing of oncology htm; and National Institutes of Health scores. Eur Radiol 2015;25(10):2840–2850. metadata (www.oriencancer.org or the Principles and Standards of Research 7. Coroller TP, Grossmann P, Hou Y, et al. CT- Children’s Oncology Group), and some Reporting, http://www.nih.gov/about/ based radiomic signature predicts distant sharing of image files has been enabled reporting-preclinical-research.htm. metastasis in lung adenocarcinoma. Radio- by The Cancer Imaging Archive (77). ther Oncol 2015;114(3):345–350. Acknowledgments: This article was an out- To date, collaborative efforts to growth of the 2014 Radiological Society of North 8. Committee on the Review of Omics-Based develop quantitative image-driven bio- America/American Association of Physicists in Tests for Predicting Patient Outcomes in markers have been remarkable. Ra- Medicine Plenary Session. The authors thank Clinical Trials, Board on Health Care Ser- the following colleagues for their substantial in- diomics has tremendous potential to vices, Board on Health Sciences Policy, Insti- put: Daniel Seeburg, MD (Johns Hopkins Uni- further enrich image interpretations tute of Medicine. Evolution of Translational versity), for his timely review of an early draft and to expand the horizons of imaging Omics: Lessons Learned and the Path For- of the manuscript; Olya Stringfield, PhD (Moffitt ward. Micheel CM, Nass SJ, Omenn GS, eds. toward greater precision and extrac- Cancer Center), for production of images; Rob- ert A. Gatenby, MD (Moffitt Cancer Center), Washington, DC: National Academies Press. tion of in vivo biologic information. To for providing images, inspiration. and edits; fully exploit the potential of radiomics, 9. Bates SE. It’s all about the test: the com- Yoganand Balagurunathan, PhD (Moffitt Cancer we need to embrace an interdisciplin- plexity of companion diagnostic co-develop- Center), for his review of the manuscript; Sandy ment in personalized medicine. Clin Cancer Napel, PhD (Stanford University), for providing ary shared vision and make a joint a comprehensive list of relevant references; and Res 2014;20(6):1418. commitment. Ada Muellner, MS (Memorial Sloan-Kettering 10. Collins FS, Varmus H. A new initiative on pre- Cancer Center), for her spectacular editing. The Radiology Reading Room of the Future cision medicine. N Engl J Med 2015;372(9): We also acknowledge funding and support from 793–795. RSNA’s sponsorship of QIBA and the NCI’s spon- The aforementioned scenario will en- sorship of QIN. tail a reading room wherein practic- 11. Office of the Press Secretary. Fact sheet: ing radiologists interact with picture Disclosures of Conflicts of Interest: R.J.G. Ac- President Obama’s precision medicine tivities related to the present article: none to initiative. The White House Web site. archiving and communication system disclose. Activities not related to the present http://www.whitehouse.gov/the-press- software to identify, segment, and ex- article: is on the advisory board of and is an in- office/2015/01/30/fact-sheet-president- tract features from regions of interest. vestor in Health Myne. Other relationships: has obama-s-precision-medicine-initiative. Up- a patent pending for Radiology Reading Room If prior studies obtained in the same dated January 30, 2015. Accessed April 4, of the Future. P.E.K. Activities related to the patient are available, the previous re- present article: none to disclose. Activities not gions of interest will be automatically related to the present article: received a grant 12. Siegel R, Ma J, Zou Z, Jemal A. Cancer identified by the reading software. As from GE Healthcare, is the cofounder of PET/X. statistics, 2014. CA Cancer J Clin 2014; Other relationships: none to disclose. H.H. dis- part of the reading, the extracted size, 64(1):9–29. closed no relevant relationships. shape, location, and textural features 13. Yachida S, Jones S, Bozic I, et al. Distant will be automatically uploaded to a metastasis occurs late during the genetic shared database and algorithmically References evolution of pancreatic cancer. Nature 2010; compared with prior images to enable 467(7319):1114–1117. 1. Institute of Medicine I. Evolution of Transla- more precise diagnoses. Such capa- tional Omics. Lessons learned and the path 14. Gerlinger M, Rowan AJ, Horswell S, et al. bilities are nearly at hand, as most forward. Washington, DC: National Acad- Intratumor heterogeneity and branched evo- emies Press, 2012. picture archiving and communication lution revealed by multiregion sequencing. N systems have the capability to coreg- Engl J Med 2012;366(10):883–892. 2. Giger ML. Update on the potential of com- ister current images with prior images puter-aided diagnosis for breast cancer. Fu- 15. Sottoriva A, Spiteri I, Piccirillo SG, et al. and perform user-interactive segmen- ture Oncol 2010;6(1):1–4. Intratumor heterogeneity in human glioblas- tation. For the foreseeable future, toma reflects cancer evolutionary dynamics. 3. Doi K. Computer-aided diagnosis in medi- the field of radiomics research will be Proc Natl Acad Sci U S A 2013;110(10): cal imaging: historical review, current status 4009–4014. concentrated on improving classifier and future potential. Comput Med Imaging Graph 2007;31(4-5):198–211. models to provide the most accurate 16. Sequist LV, Martins RG, Spigel D, et al. possible diagnoses and, hence, better 4. Guo Z, Shu Y, Zhou H, Zhang W, Wang H. First-line gefitinib in patients with advanced patient care and outcomes. non-small-cell lung cancer harboring so- Radiogenomics helps to achieve personalized Radiology: Volume 278: Number 2—February 2016 radiology.rsna.org 575 SPECIAL REPORT: Radiomics Gillies et al matic EGFR mutations. J Clin Oncol 2008; Historical Perspective and Planned Goals. 41. Haralick RM, Shanmugam K, Dinstein I. 26(15):2442–2449. Transl Oncol 2014;7(1):1–4. Textural features for image classification. IEEE Trans Syst Man Cybern 1973;SMC- 17. Burstein HJ, Harris LN, Marcom PK, et al. 29. Buckler AJ, Bresolin L, Dunnick NR, Sul- 3(6):610–621. Trastuzumab and vinorelbine as first-line livan DC; for the Group. A collaborative en- therapy for HER2-overexpressing metastatic terprise for multi-stakeholder participation 42. Davnall F, Yip CS, Ljungqvist G, et al. As- breast cancer: multicenter phase II trial in the advancement of quantitative imaging. sessment of tumor heterogeneity: an emerg- with clinical outcomes, analysis of serum Radiology 2011;258(3):906–914. ing imaging tool for clinical practice? Insights tumor markers as predictive factors, and Imaging 2012;3(6):573–589. 30. Kinahan P, Wahl R, Shao L, Frank R, Perl- cardiac surveillance algorithm. J Clin Oncol man E. The QIBA profile for quantitative 43. O’Connor JP, Rose CJ, Waterton JC, Cara- 2003;21(15):2889–2895. FDG-PET/CT oncology imaging [abstr]. J no RA, Parker GJ, Jackson A. Imaging in- 18. Wagle N, Emery C, Berger MF, et al. Dis- Nucl Med 2014;55(Suppl 1):1520. tratumor heterogeneity: role in therapy secting therapeutic resistance to RAF inhi- response, resistance, and clinical outcome. 31. FDG-PET/CT Technical Committee. FDG- bition in melanoma by tumor genomic pro- Clin Cancer Res 2015;21(2):249–257. PET/CT as an Imaging Biomarker Measuring filing. J Clin Oncol 2011;29(22):3085–3096. Response to Cancer Therapy, Quantitative 44. Rose CJ, Mills SJ, O’Connor JP, et al. Quan- 19. Sequist LV, Waltman BA, Dias-Santagata Imaging Biomarkers Alliance, Version 1.05, tifying spatial heterogeneity in dynamic con- D, et al. Genotypic and histological evolu- Publicly Reviewed Version. RSNA Web site. trast-enhanced MRI parameter maps. Magn tion of lung cancers acquiring resistance to http://www.rsna.org/qiba/. Published De- Reson Med 2009;62(2):488–499. EGFR inhibitors. Sci Transl Med 2011;3(75): cember 11, 2013. Accessed May 20, 2015. 45. Larkin TJ, Canuto HC, Kettunen MI, et al. 75ra26. 32. van Persijn van Meerten EL, Gelderblom Analysis of image heterogeneity using 2D 20. Clauson J, Hsieh YC, Acharya S, Rademak- H, Bloem JL. RECIST revised: implications Minkowski functionals detects tumor re- er AW, Morrow M. Results of the Lynn Sage for the radiologist: a review article on the sponses to treatment. Magn Reson Med 2014; Second-Opinion Program for local therapy modified RECIST guideline. Eur Radiol 71(1):402–410. in patients with breast carcinoma: changes 2010;20(6):1456–1467. 46. Grossmann P, Grove O, El-Hachem N, et in management and determinants of where 33. Burnside ES, Sickles EA, Bassett LW, et al. Identification of molecular phenotypes care is delivered. Cancer 2002;94(4):889– al. The ACR BI-RADS experience: learning in lung cancer by integrating radiomics and from history. J Am Coll Radiol 2009;6(12): genomics. Sci Transl Med (in press). 21. Nguyen PL, Schultz D, Renshaw AA, et al. 851–860. 47. Aerts HJ, Velazquez ER, Leijenaar RT, et al. The impact of pathology review on treat- 34. Bomers JG, Barentsz JO. Standardization Decoding tumour phenotype by noninvasive ment recommendations for patients with of multiparametric prostate MR imaging imaging using a quantitative radiomics ap- adenocarcinoma of the prostate. Urol Oncol using PI-RADS. BioMed Res Int 2014;2014: proach. Nat Commun 2014;5:4006. 2004;22(4):295–299. 48. Pickup L, Talwar A, Stalin S, et al. Lung 22. Staradub VL, Messenger KA, Hao N, Wiley 35. Kazerooni EA, Armstrong MR, Amorosa JK, nodule classification using learnt texture EL, Morrow M. Changes in breast cancer et al. ACR CT accreditation program and the features on a single patient population [ab- therapy because of pathology second opin- lung cancer screening program designation. J str]. In: Radiological Society of North Amer- ions. Ann Surg Oncol 2002;9(10):982–987. Am Coll Radiol 2015;12(1):38–42. ica Scientific Assembly and Annual Meeting 23. Kronz JD, Westra WH, Epstein JI. Mandatory Program. Oak Brook, Ill: Radiological Soci- 36. Gatenby RA, Grove O, Gillies RJ. Quantita- second opinion surgical pathology at a large ety of North America, 2015; SSM06. tive imaging in cancer evolution and ecology. referral hospital. Cancer 1999;86(11): Radiology 2013;269(1):8–15. 49. Schabath MB, Massion PP, Thompson ZJ, 2426–2435. et al. Behavior of incidence lung cancers in 37. Zhou M, Hall L, Goldgof D, et al. Radiolog- 24. Kumar V, Gu Y, Basu S, et al. Radiomics: the CT arm of the National Lung Screening ically defined ecological dynamics and clin - the process and the challenges. Magn Reson Trial. JAMA (in press). ical outcomes in glioblastoma multiforme: Imaging 2012;30(9):1234–1248. preliminary results. Transl Oncol 2014;7(1): 50. Balagurunathan Y, Kumar V, Gu Y, et al. Test- 25. Lambin P, Rios-Velazquez E, Leijenaar R, et 5–13. retest reproducibility analysis of lung CT al. Radiomics: extracting more information image features. J Digit Imaging 2014;27(6): 38. Rios Velazquez E, Aerts HJ, Gu Y, et al. A from medical images using advanced feature 805–823. semiautomatic CT-based ensemble segmenta- analysis. Eur J Cancer 2012;48(4):441–446. tion of lung tumors: comparison with oncolo- 51. Lin LI. A concordance correlation coefficient 26. Clarke LP, Sriram RD, Schilling LB. Imaging gists’ delineations and with the surgical speci- to evaluate reproducibility. Biometrics 1989; as a Biomarker: Standards for Change Mea- men. Radiother Oncol 2012;105(2):167–173. 45(1):255–268. surements in Therapy workshop summary. 39. van Dam IE, van Sörnsen de Koste JR, 52. Carrasco JL, King TS, Chinchilli VM. The Acad Radiol 2008;15(4):501–530. Hanna GG, Muirhead R, Slotman BJ, Senan concordance correlation coefficient for re - 27. Shankar LK, Hoffman JM, Bacharach S, et S. Improving target delineation on 4-dimen- peated measures estimated by variance al. Consensus recommendations for the use sional CT scans in stage I NSCLC using a de- components. J Biopharm Stat 2009;19(1): of 18F-FDG PET as an indicator of therapeu- formable registration tool. Radiother Oncol 90–105. tic response in patients in National Cancer 2010;96(1):67–72. 53. Fehr D, Veeraraghavan H, Wibmer A, et al. Institute Trials. J Nucl Med 2006;47(6):1059– 40. Segal E, Sirlin CB, Ooi C, et al. Decoding Automatic classification of prostate cancer global gene expression programs in liver Gleason scores from multi-parametric mag- 28. Clarke LP, Nordstrom RJ, Zhang H, et al. cancer by noninvasive imaging. Nat Biotech- netic resonance images. Proc Natl Acad Sci The Quantitative Imaging Network: NCI’s nol 2007;25(6):675–680. USA (in press). 576 radiology.rsna.org Radiology: Volume 278: Number 2—February 2016 SPECIAL REPORT: Radiomics Gillies et al 54. Diehn M, Nardini C, Wang DS, et al. Identifi - ical impact. Eur J Nucl Med Mol Imaging 70. Landis SC, Amara SG, Asadullah K, et al. cation of noninvasive imaging surrogates for 2010;37(11):2027–2036. A call for transparent reporting to optimize brain tumor gene-expression modules. Proc the predictive value of preclinical research. 61. Tatli S, Gerbaudo VH, Mamede M, Tun- Natl Acad Sci U S A 2008;105(13):5213– Nature 2012;490(7419):187–191. cali K, Shyn PB, Silverman SG. Abdominal masses sampled at PET/CT-guided percuta- 71. Drawing Causal Inference from Big Data. neous biopsy: initial experience with regis- 55. Kuo MD, Gollub J, Sirlin CB, Ooi C, Chen Arthur M. Sackler Colloquia of the National tration of prior PET/CT images. Radiology X. Radiogenomic analysis to identify im- Academy of Sciences. https://www.youtube. 2010;256(1):305–311. aging phenotypes associated with drug re- com/playlist?list=PLGJm1x3XQeK0NgFO sponse gene expression programs in hepa- X2Z7Wt-P5RU5Zv0Hv. Published April 2, 62. Simon RM, Paik S, Hayes DF. Use of ar- tocellular carcinoma. J Vasc Interv Radiol 2015. Accessed May 5, 2015. chived specimens in evaluation of prognos- 2007;18(7):821–831. tic and predictive biomarkers. J Natl Cancer 72. Nelson B. Data sharing: empty archives. Na- Inst 2009;101(21):1446–1452. 56. Grove O, Berglund AE, Schabath MB, et ture 2009;461(7261):160–163. al. Quantitative computed tomographic de- 63. Hayes DF, Bast RC, Desch CE, et al. Tumor 73. Devidas M, London WB, Anderson JR. The scriptors associate tumor shape complexity marker utility grading system: a framework use of central laboratories and remote elec- and intratumor heterogeneity with prog- to evaluate clinical utility of tumor markers. tronic data capture to risk-adjust therapy for nosis in lung adenocarcinoma. PLoS One J Natl Cancer Inst 1996;88(20):1456–1466. pediatric acute lymphoblastic leukemia and 2015;10(3):e0118261. neuroblastoma. Semin Oncol 2010;37(1):53– 64. Chalmers I, Glasziou P. Avoidable waste in 57. Gevaert O, Mitchell LA, Achrol AS, et al. 59. the production and reporting of research ev- Glioblastoma multiforme: exploratory ra- idence. Lancet 2009;374(9683):86–89. 74. Heilbrun ME. Evaluating RadLex and real diogenomic analysis by using quantitative world radiology reporting: are we there yet? 65. Simera I, Moher D, Hirst A, Hoey J, Schulz image features. Radiology 2014;273(1):168– Acad Radiol 2013;20(11):1327–1328. KF, Altman DG. Transparent and accurate reporting increases reliability, utility, and 75. Kahn CE Jr. Annotation of figures from the 58. Vignati A, Mazzetti S, Giannini V, et al. Tex- impact of your research: reporting guide- biomedical imaging literature: a comparative ture features on T2-weighted magnetic res- lines and the EQUATOR Network. BMC analysis of RadLex and other standardized onance imaging: new potential biomarkers Med 2010;8:24. vocabularies. Acad Radiol 2014;21(3):384– for prostate cancer aggressiveness. Phys 66. Prinz F, Schlange T, Asadullah K. Believe it Med Biol 2015;60(7):2685–2701. or not: how much can we rely on published 76. Mabotuwana T, Lee MC, Cohen-Solal EV, 59. Teruel JRHM, Heldahl MG, Goa PE, et al. data on potential drug targets? Nat Rev Chang P. Mapping institution-specific study Dynamic contrast-enhanced MRI texture Drug Discov 2011;10(9):712. descriptions to RadLex Playbook entries. J analysis for pretreatment prediction of clin- 67. Begley CG, Ellis LM. Drug development: Digit Imaging 2014;27(3):321–330. ical and pathological response to neoadju- Raise standards for preclinical cancer re- vant chemotherapy in patients with locally 77. Clark K, Vendt B, Smith K, et al. The Can- search. Nature 2012;483(7391):531–533. advanced breast cancer. NMR Biomed 2014; cer Imaging Archive (TCIA): maintaining 27(8):887–896. 68. Journals unite for reproducibility. Nature and operating a public information reposi- 2014;515(7525):7. tory. J Digit Imaging 2013;26(6):1045–1057. 60. Klaeser B, Wiskirchen J, Wartenberg J, et al. PET/CT-guided biopsies of metabolically 69. McNutt M. Journals unite for reproducibil- active bone lesions: applications and clin- ity. Science 2014;346(6210):679. Radiology: Volume 278: Number 2—February 2016 radiology.rsna.org 577

Journal

Radiology – Pubmed Central

Published: Nov 18, 2015

There are no references for this article.