Journal of Bone Research

Journal of Bone Research
Open Access

ISSN: 2572-4916

+44 1478 350008

Review Article - (2015) Volume 3, Issue 3

A Systematic Literature Review of Imaging Definitions for Detection of Bone and Bone Marrow Metastases in Neuroblastoma Patients

Gitta Bleeker1, Doris Heijkoop1, Elvira C Van Dalen1, Leontien C Kremer1, Anne M Smets2, Eline E Deurloo2, Berthe L Van Eck-Smit3, Huib N Caron1 and Godelieve A Tytgat1*
1Department of Paediatric Oncology, Emma Children’s Hospital, Academic Medical Centre (AMC), University of Amsterdam, Amsterdam, The Netherlands
2Department of Radiology, Academic Medical Centre (AMC), University of Amsterdam, Amsterdam, The Netherlands
3Department of Nuclear Medicine, Academic Medical Centre (AMC), University of Amsterdam, The Netherlands
*Corresponding Author: Godelieve A Tytgat, M.D., Department of Paediatric Oncology, Emma Children’s Hospital, Academic Medical Centre (AMC), University of Amsterdam, PO Box 22700, 1100 DE Amsterdam, The Netherlands Email:

Abstract

Objective: The presence of bone and bone marrow (BM) metastases in neuroblastoma patients are independent adverse prognostic factors, so precise and consistent definitions of both categories on imaging are important. The objectives of this systematic review were to identify all definitions reported for detection of bone and bone marrow metastases on imaging in neuroblastoma to determine diagnostic accuracies of the most frequently used definitions for detection of bone and/or BM metastases on each imaging technique.

Methods: We searched MEDLINE/PubMed (1945 to April 2013) and EMBASE/Ovid (1980 to April 2013). Full-text original studies were included if they reported definitions of bone and/or BM metastases on diagnostic imaging of children with suspected metastatic neuroblastoma. As reference standard for bone metastases bone scintigraphy was used and for bone marrow metastases bone marrow biopsies/aspirates. Methodological quality was assessed.

Results: Of 403 identified studies (plus one relevant reference), 131 were assessed in full-text and 31 finally included, 23 described BM metastases and 18 bone metastases. No uniform definitions of bone and bone marrow metastases were reported for each imaging method. On MIBG scintigraphy bone metastases were mostly defined as “focal” and BM metastases as “diffuse” and on MRI both definitions were used for BM metastases. The diagnostic accuracy of different diagnostic methods to detect bone (reference test bone scintigraphy) or BM (reference test bone marrow biopsies/ trephines) metastases varied widely.

Conclusion: No uniform definitions of bone and bone marrow metastases were reported for each imaging method and concerning the diagnostic accuracy no general conclusions could be drawn.

<

Keywords: Neuroblastoma; Bone metastases; Bone marrow metastases; Diagnostic imaging; MIBG scintigraphy; MRI

Introduction

Overall survival (OS) of high-risk neuroblastoma patients is about 40% despite intense multi-modality treatment [1-4], although the addition of anti-GD2 therapy might improve this outcome [5]. Highrisk neuroblastoma is defined by the presence of distant metastases and/ or the presence of biological factors like amplification of the MYCN gene (MNA). Because bone and bone marrow metastases are reported to be independent poor prognostic factors [6-10], it is necessary to have clear and uniform definitions of respectively bone metastases and bone marrow metastases, as well as a need of clear discrimination of both types of metastatic disease.

Ladenstein and co-workers reported an inferior outcome for patients with bone marrow metastases at initial diagnosis and furthermore an inferior, but not identical outcome for patients with bone and bone marrow metastases before mega-therapy [10]. In patients with metastatic disease younger than one year at diagnosis, the presence of bone metastases is reported to be associated with an inferior outcome [7]. According to the consensus paper of the International Neuroblastoma Risk Group (INRG), for the diagnosis of stage MS a maximum of 10% bone marrow invasion is allowed, and bone metastases are not [11,12]. In contrast to protocols for bone marrow involvement, no standardized protocols for the definition and differentiation of bone from bone marrow metastases on imaging are reported.

Because many studies report on the presence of both bone marrow and bone metastases detected on different imaging methods, such as metaiodobenzylguanidine (MIBG) scintigraphy, computed tomography (CT), magnetic resonance imaging (MRI), bone scintigraphy, like Technetium-99m-methylene diphosphanate (99mTc- MDP) scintigraphy, and fluorine-18-fluorodeoxy-glucose positron emission tomography (18F-FDG-PET) [7,13,14], we wondered what definitions were used in these papers.

So the objectives of this systematic review were to identify all published definitions of bone and bone marrow metastases on all different imaging modalities; and to determine the diagnostic accuracy to detect bone and bone marrow metastases, with these published definitions on the different imaging methods.

Methods

Preferred Reporting Items for Systematic Reviews and Meta- Analyses (PRISMA) guidelines were followed for this review [15].

Search strategy

We searched MEDLINE/PubMed (from 1945 to April 2013) with a combination of the following controlled vocabulary and text words:

((“Neuroblastoma”[Mesh:noexp] OR neuroblastoma [tiab] OR neuroblastomas[tiab]) AND (“Bone Neoplasms/secondary”[Mesh] OR bone metasta*[tiab] OR “Bone Marrow Neoplasms/secondary”[Mesh] OR bone marrow metasta*[tiab] OR skeletal metasta*[tiab]) AND (“Diagnostic Imaging”[Mesh] OR diagnostic imaging[tiab] OR radionuclide imaging[tiab] OR scintigraphy[tiab] OR SPECT[tiab] OR radiography[tiab] OR x-ray[tiab] OR *tomography[tiab] OR ultrasonography[tiab] OR magnetic resonance imaging[tiab] OR MRI[tiab] OR PET[tiab] OR MIBG[tiab] OR 3-Iodobenzylguanidine[tw] OR technetium Tc 99m medronate[tw] OR fluorodeoxyglucose f18[tw] OR fdg[tiab] OR radiopharmaceuticals[pa]))

(Abbreviations: Mesh: Medical subject headings; Noexp: not exploded; Tiab: title and abstract; Tw: text word (=title and abstract); Pa: pharmacological.action; *: zero or more characters)

We also searched EMBASE/Ovid (from 1980 to April 2013) with a combination of the following controlled vocabulary and text words:

i. neuroblastoma/

ii. (neuroblastoma or neuroblastomas).ti,ab.

iii. 1 or 2

iv. bone metastasis/

v. bone marrow metastasis/

vi. (bone adj2 metastas*).ti,ab.

vii. (skeletal adj2 metastas*).ti,ab.

viii. or/4-7

ix. exp radiopharmaceutical agent/

x. exp diagnostic imaging/

xi. diagnostic imaging.ti,ab.

xii. radionuclide imaging.ti,ab.

xiii. scintigraphy.ti,ab.

xiv. SPECT.ti,ab.

xv. exp bone radiography/

xvi. radiography.ti,ab.

xvii. x-ray.ti,ab.

xviii. exp *tomography/

xix. *tomography/.ti,ab.

xx. ultrasonography.ti,ab.

xxi. magnetic resonance imaging.ti,ab.

xxii. MRI.ti,ab.

xxiii. PET.ti,ab.

xxiv. MIBG.ti,ab.

xxv. Iodobenzylguanidine.tw.

xxvi. Tc 99m.tw.

xxvii. fluorodeoxyglucose.tw.

xxviii. fdg.tw.

xxix. or/9-28

xxx. 3 and 8 and 29

(Abbreviations: Pa: pharmacological.action; /: word as subject heading (EMBASE); Ti.ab: title and abstract; Exp: explode; Tw: textword (=title and abstract); *: zero or more characters)

The resulting list of articles was supplemented through crosschecking of reference lists of relevant articles and review articles. If studies were reported in conference proceedings, we searched for full publications.

Eligibility criteria for study selection

Objective 1: Definitions of bone and bone marrow metastases: Studies were included if they reported on the use of diagnostic imaging in children (<18 years old) with neuroblastoma, English language was used, they described definitions of bone and/or bone marrow metastases, and if they were original reports (no reviews) and were reported as full-text studies. We clearly stated reasons for exclusion for any study considered for the review. In case of duplicate publications we used the most recent paper.

Objective 2: Diagnostic accuracy of all imaging techniques to detect bone and/or bone marrow metastases: From all studies that were included for objective 1, only studies that compared the results of an index test with a reference test, thus providing sensitivity and specificity, or data to calculate them, were included.

The literature was studied for bone and bone marrow metastases separately. Unless stated differently, in this review of the literature, we used bone marrow biopsies or aspirates as reference standard for bone marrow metastases [12] and bone scintigraphy for bone metastases. The results were described separately for bone and bone marrow metastases.

For both objective 1 and 2, two reviewers independently assessed all potentially eligible studies. First selection was performed on title and abstract. Next, the full text versions of the selected articles were reviewed to determine their eligibility for inclusion in the study. Disagreements were resolved by discussion.

Data collection

Objective 1: Definitions of bone and bone marrow metastases

The following items were scored for each included study:

a) Study population: age, sex, stage of neuroblastoma, primary or recurrent neuroblastoma, in- and exclusion criteria, number of subjects (including number eligible for the study, number enrolled in the study);

b) Description of diagnostic imaging method describing bone and/ or bone marrow metastases;

c) Used definitions of bone and/or bone marrow metastases.

Objective 2: Diagnostic accuracy of all imaging techniques to detect bone and/or bone marrow metastases

We extracted data on the following additional items:

a) True positive, false negative, false positive and true negative findings on diagnostic modalities for the detection of bone and/or bone marrow metastases (if not enough data were available to calculate sensitivity and/or specificity ourselves, we used the sensitivity and/or specificity as reported in the article);

b) The number of assessments (a patient could have more than one scan, e.g. at initial diagnosis and/or one or more during treatment or follow-up) and/or the number of lesions in the sensitivity/specificity analyses (out of the total number of eligible patients and/or lesions); and

c) The reference standards used in the sensitivity/specificity analyses.

For both objectives 1 and 2, two review authors performed data extraction independently, using standardized forms.

Methodological quality assessment

For each study eligible for objective 2, the risk of bias and concerns regarding applicability on four domains were scored namely patient selection, index test, reference standard, and flow of patients through the study and timing of the index test(s) and reference standard (flow and timing) according to a modified version of the QUADAS-2 tool (Table 1) [16]. Because studies could report on bone and/or bone marrow metastases, we scored them separately.

Domain Patient Selection Index Test Reference Test Flow and Timing
Description Describe methods of patient selection. Describe the index test and how it was conducted and interpreted. Describe the reference standard and how it was conducted and interpreted. Describe any patients who did not receive the index test(s) and/or reference standard or who were excluded from the 2x2 table.
SignallingQuestions - Consecutive or random sample?
- Case-control design avoided?
- Inappropriate exclusions avoided?
- Interpreted without knowledge of the reference test? - Likely to correctly classify the target condition?
- Interpreted without knowledge of the index test?
- Appropriate interval between index and reference test (< 2 weeks)?
- Did all patients receive the reference standard?
- Did all patients receive the same reference standard?
- Were all patients included in the analysis (high risk bias if >10% missing data)?
Risk of Bias Could the selection of patients have introduced bias? Could the interpretation of the index test have introduced bias? Could the reference standard, its conduct, or its interpretation have introduced bias? Could the patient flow have introduced bias
Concerns Applicability Are there concerns that the included patients do not match the review question? Are there concerns that the index test or interpretation differ from the review question? Are there concerns that the target condition as defined by the reference standard does not match the review question?  
*According to a modified version of the QUADAS-2 tool

Table 1: Methodological quality assessment [16]*.

Two authors independently assessed methodological quality of the studies. Disagreements were resolved by discussion.

Statistical analyses

We calculated sensitivity and/or specificity using two-by-two tables (consisting of true positives, false negatives, false positives and true negatives) in MS-Excel. Unless stated differently, we used bone marrow biopsies or aspirates as reference standard for bone marrow metastases [12] and bone scintigraphy for bone metastases. Sensitivity was calculated as true positives divided by all positives (true positives + false negatives) and specificity as true negatives divided by all negatives (true negatives + false positives). Imaging could be performed at initial diagnosis and during treatment and follow-up. Therefore, analyses on diagnostic accuracy were performed on assessment level for all included studies. If data per lesion were available, we also calculated and reported the sensitivity and specificity at lesion level.

Pooled analysis only included studies that used the same index test, the same reference standard and the same definitions of bone and bone marrow metastases and if at least 10 patients (in total) were available. Data of 123I- and 131I-MIBG scans were not analysed separately, because it was reported in the literature that there is no significant difference in results by type of scan [17].

Results

Selection of articles

The search of the electronic databases of MEDLINE/PubMed and EMBASE/Ovid (in April 2013) yielded a total of 403 references (Figure 1). A total of 30 studies fulfilled the inclusion criteria of this review. Screening the reference lists of reviews and relevant articles identified one additional study, so a total of 31 studies were eligible for inclusion in this review.

archive-bone-marrow-research-Flow-chart-Inclusion

Figure 1: Flow chart Inclusion of studies.

The reasons for exclusion for 101 studies are provided in Supplementary Table 1.

Characteristics of included studies

Objective 1: Definitions of bone and bone marrow metastases: Of the 31 studies, 8 studies formulated a definition of bone metastases [8,18-24], 13 of bone marrow metastases [25-37] and 10 defined both types of metastases [38-47] (Table 2). The included studies used different imaging methods to detect bone or bone marrow metastases (per study often more than one imaging method was used). The most frequent reported imaging technique was MIBG scintigraphy (15 of 31 studies). 8 studies reported on 123I-MIBG [27,31,36,38-40,44,45], 4 on 131I-MIBG [23,29,33,46] and in 3 studies it was not clear [8,43,47].

Study MIBG n=15 MRI n=12 BS n=10 RAD n=8 CT n=6 PET n=1 MIBI n=1 SC n=1 3F8 n=1 PEN n=1 TI n=1
Type of Metastases B BM B BM B BM B BM B BM B BM B BM B BM B BM B BM B BM
Howman-Giles et al.[21]         + -                                
aChirathivat et al.[18]             + - + -                        
bCohen et al.[26]     - +                     - +            
Shulkin et al.[33] - +                                        
Fletcher et al.[28]     - +                         - +        
Corbett et al.[27] - + - +                                    
Hanna et al.[41]     - +     + -                            
dRuzal Shapiro et al. [22]             + - + -                        
cHaddad et al.[37]     - +                                    
Shulkin et al.[23] + -     + -                                
Osmanagoaglu et al.[45] + +     + -                                
Tanabe et al.[34]     - +                                    
Turba et al[47] - +     + - + -                            
Claudiani et al.[38] + +     + -                                
eGiammarile et al.[39] + +     + -                                
HadjDjilani et al.[40] + +     + +                                
Tanabe et al.[35]     - +                                    
fLebtahi et al.[31] - + - +                                    
Minard et al.[8] + -         + -                            
Kushner et al.[43] + +                 + +                    
gOkuyama et al.[44] + +     + - + - + -                     + +
Juweid et al. [42]         + -                         + +    
Siegel et al.[24]     + - + -     + -                        
hUeno et al.[36] - + - +                                    
iGrover et al[20]             + - + -                        
Frappaz et al.[29] - +                                        
Shah Syed et al.[46] + +                                        
Berberoglu et al.[25]                         - +                
Goo et al.[30]     - +                                    
Meyer, et al.[32]     - +                                    
jChu et al.[19]             + - + -                        
Definitions 9 13 1 11 10 1 8 0 6 0 1 1 0 1 0 1 0 1 1 1 1 1
MIBG: MetaiodobenzylguanidineScintigraphy; MRI: Magnetic Resonance Imaging; BS: Bone Scintigraphy; RAD: Conventional Radiology; CT: Computed Tomography; PET: 18F-FDG-Positron Emission Tomography; MIBI: 99mTc-Methoxyisobutylisonitrile Scintigraphy;SC: 99mTc-sulphur Colloid Scintigraphy; 3F8: 131I-3F8 Monoclonal Anti-body Scintigraphy; PEN: 111In PentetreotideScintigraphy; TI: 201Tl Scintigraphy; B: Bone Metastases; BM: Bone Marrow Metastases.
a: Case series: two cases; b: One of seven patients had a neuroblastoma; c: Case series: three cases; d: Two of six patients had a neuroblastoma; e: Case report: one case; f: Case series: four cases; g: Eight patients + case series of three of these patients; h: Case report: one case; i: Case report: one case. Only contrast CT of head was performed; j: Case series: seven cases.

Table 2: Overview of all studies reporting a definition of bone and bone marrow metastases, detected on different imaging methods.

12 studies reported on MRI, with a variety of MRI sequences: 1 reported on MRI with short TI inversion recovery (STIR), and MRIgadolinium (gad)-T1, as well on MRI-T1 and MRI-T2 [41], 1 on MRISTIR, MRI-gadolinium (gad)-T1 and MRI-T1 [32]; 1 on MRI-STIR only [24]; 7 on MRI-T1 or -T2 [22,26-28,30,34,35]. 2 studies did not define the type of MRI [31,36]. 10 studies reported on bone scintigraphy [21,23,24,38-40,42,44,45,47]; 8 on conventional radiographs [8,18-20,23,37,44,47]; 6 on computed tomography (CT) [18-20,24,37,44]; and 1 on 18F-FDG PET (with or without CT) [43], 99mTc-sestamibi (MIBI) scintigraphy [52], 99mTc-sulphur colloid scintigraphy [26], 131I-3F8 monoclonal anti-body scintigraphy [28], Indium (111In) pentetreotide scintigraphy [42]; and Thallium (201Tl) scintigraphy [44].

Reported definitions of bone metastases: In Table 3A, all the definitions of bone metastases per imaging technique for the 18 included studies, are listed. For reasons of clarity and interpretation, in the following text, similar definitions were grouped together in larger categories per imaging technique; however, these might not be identical to the exact definitions per study as reported in Table 3A.

In 9 out of 15 studies (60%) reporting on MIBG scintigraphy, a definition of bone metastases was given. Five studies gave clear definitions of bone metastases: “focal uptake” (n=3) [43,45,46] or “hotspots” (n=2) [40,44]. In 4 other studies the definition was more ambiguous: “any uptake” (n=3) [8,23,38] or “more intense uptake” (n=1) [39].

Of the 12 studies reporting on MRI, only 1 (8%) gave a definition of bone metastases i.e., “areas of cortical bone destruction” [24].

All studies reporting on bone scintigraphy (n=10) gave definitions of bone metastases. 5 studies gave clear definitions: “focal uptake” (n=3) [21,24,42] and “hotspots” (n=2) [40,45]. 5 other studies gave less clear definitions: “any uptake” (n=2) [23,47], “abnormal uptake” (n=2) [38,44], or “uptake throughout the skeleton with focal lesions that coalesce to produce a relatively diffuse image” (n=1) [39].

For conventional radiography, all 8 studies gave definitions of bone metastases: “osteolytic lesions”, “periosteal reaction”, “sunray spiculation”, “pathological fractures” (n=6) [18-20,22,37,44]. Two studies defined bone metastases as “all abnormalities” on radiography [8,47].

For CT, all six studies reported on definitions of bone metastases: “sunray spiculation” (n=2) [18,20] and “bone destruction with characteristic periosteal reaction” (n=4) [18,24,37,44]. For the other nuclear imaging techniques, definitions, if available, are shown in Table 3A.

Reported definitions of bone marrow metastases: Table 3B reports on the definitions of bone marrow metastases provided in the 23 included studies. Again, for reasons of clarity and interpretation, in the following text, similar definitions were grouped together in larger categories per imaging technique; however, these might not be identical to the exact definitions per study as reported in Table 3B.

13 of the 15 studies (87%) reporting on MIBG scintigraphy gave a definition of bone marrow metastases. 11 studies [22,26-28,30-32,34-36,41] gave clear definitions: “diffuse uptake” (n=10, 77%) [31,33,38-40,43-47] or “hotspots” (n=1) [36]. Other, more ambiguous, definitions were “any uptake” (n=1) [29] or “abnormalities within the bone marrow compartment” (n=1) [27].

Of 12 studies reporting on MRI, 11 (92%) defined bone marrow metastases. Definitions were clear in 5: “focal areas” (n=1) [42], “diffuse and nodular type of lesions” (n=2) [28,35], or “diffuse abnormal uptake” (n=2) [26,31]. Furthermore, four studies (36%) reported “low intensity on T1 and high intensity on T2” [22,34,35,41]. Three studies gave a more ambiguous definition: “presence or absence of abnormalities” [27], “abnormal shadow” [36] and “hyper-intensity” [30].

Another study on MRI gave only specific definitions for vertebral metastases: “moderately heterogeneous or focal signal variations” or “greater signal than or equal to the signal intensity of cerebrospinal fluid” [32].

Of the 10 studies that reported on bone scintigraphy, 1 (10%) gave a definition of bone marrow metastases: “diffuse uptake” [40]. For the other nuclear imaging techniques, definitions, if available, are shown in Table 3B.

Objective 2: Diagnostic accuracy of all imaging techniques to detect bone and/or bone marrow metastases: 14 of the 31 included studies (45%) were included for objective 2 (Table 4A). They all reported definitions of bone and/or bone marrow metastases on an index test (imaging technique). They all used bone scintigraphy as a reference test to detect bone metastases and bone marrow biopsies/ aspirates as a reference test to detect bone marrow metastases. Patient characteristics of the included patients for this objective are shown in Supplementary Table 2.

Diagnostic accuracy for the detection of bone metastases: The sensitivities and specificities for the detection of bone metastases are given in Table 4A for all studies. The sensitivity and specificity on all reported imaging techniques and with different definitions, ranged widely, from 0.17 to 1.00 in 6 studies [38,40,43-46] and from 0.00 to 1.00 in 7 studies [38,40,42-46] respectively (Table 4A). For most imaging techniques, pooling was not possible, because of the small number of studies with the same techniques and definitions.

Only 5 studies that defined bone metastases on MIBG scintigraphy as “focal” or “hotspots” [40,43-46], could be pooled for their sensitivity and specificity, resulting in a sensitivity of 0.88 and specificity of 0.79 (MIBG with different radio-isotopes was used) (Table 4B).

Diagnostic accuracy for the detection of bone marrow metastases: The sensitivity and specificity for the detection of bone marrow metastases, on different imaging techniques and with different definitions, ranged from 0.26 to 1.00 in 11 studies [27,32,34,38,40,41,43-47], and 0.41 to 1.00 for 12 studies [25,27,29,32,40-47], respectively (Table 4A). For 18F-FDG-PET scintigraphy, the sensitivity and specificity of the whole-body scans were shown. Because the brain is always 18F-FDG positive, skull lesions are hard to visualize and therefore the diagnostic accuracy improved if skull lesions were excluded with a sensitivity of 0.63 [43].

The pooled sensitivity and specificity of the 7 studies that defined bone marrow metastases as “diffuse uptake” [38,40,43-47] was 0.60 and 0.85, respectively (Table 4C) (MIBG with different radio-isotopes was used).

Methodological quality of the studies evaluating the diagnostic test accuracy: Table 5 shows the overall quality of the 14 included studies for objective 2. Seven reported on bone metastases and all 14 on bone marrow metastases. Studies that reported on more than one index test were scored separately for each test, because they had the same scores on all QUADASitems for each index test, only one QUADAS-score per study is shown in Table 5.

For the detection of bone metastases the risk of bias in patient selection was low in 2 (29%) [40-46] and unclear in 5 studies (71%) [38,42-45]. The concerns regarding applicability were low in 4 (57%) [40,42,45,46] and unclear in 3 (43%) [38,43,44]. In 2 of the 5 studies with an unclear risk of bias in patient selection, the applicability concerns were low [42,45] and in 3 unclear [38,43,44].

Concerning bone marrow metastases, the risk of bias in patient selection was low in 7 (50%) [29,32,34,40,45-47] and unclear in 7 studies (50%) [25,27,38,41-44]. The concerns regarding applicability were low in 9 (64%) [27,29,32,34,40,42,45-47] and unclear in 5 (36%) [25,38,41,43,44]. In 2 of the 7 studies with an unclear risk of bias in patient selection, the applicability concerns were low [27,42] and in 5 unclear [25,38,41,43,44].

For the detection of bone metastases the risk of bias in the interpretation of the index test was high in 1 (14%) [40], because MIBG scans and bone scans were evaluated simultaneously. There are no applicability concerns for this study. The risk of bias was low in 3 (43%) [42,43,46] and unclear in 3 studies (43%) [38,44,45]. The applicability concerns were low in 6 (86%) [40,42-46] and unclear in 1 (14%) study [38]. Of the 4 studies with an unclear risk of bias, the applicability concerns were low in 2 [44,45] and unclear in 1 [38].

Concerning bone marrow metastases, the risk of bias in the interpretation of the index test was low in 8 (57%) [25,27,32,34,41-43,46] and unclear in 6 (43%) studies [29,38,40,44,45,47]. The applicability concerns were low in 12 (86%) [25,27,32,34,40-47] and unclear in 2 (14%) studies [29,38]. Of the 6 studies with an unclear risk of bias, the applicability concerns were low in 4 [40,44,45,47] and unclear in 2 [29,38].

For the detection of bone metastases the risk of bias in the interpretation of the reference test was high in 1 (14%), because MIBG scans and bone scans were evaluated simultaneously [40]. However, the applicability concerns were low for this study. The risk of bias was low in 6 (86%) [38,42-46]. The concerns regarding applicability were low for all 7 studies [38,40,42-46].

Concerning bone marrow metastases, the risk of bias on the reference standards was low in 13 (93%) [25,27,29,32,34,38,40-46] and unclear in one (7%) [47]. For all studies the concerns regarding applicability were low [25,27,29,32,34,38,40-47].

For the detection of bone metastases the risk of bias on flow and timing was low in 3 studies (43%) [40,42,43] and high in 4 studies (37%) [38,44,45,46]. Of the 97 patients included in one study bone metastases were reported in only 12 patients [38]. Of the 5 patients in the second study one was excluded from the analyses because of missing data (no bone scan was performed) [44]. In the third study only 12 of the 18 patients had bone scintigraphy [46]. Of one study the number of assessments was 72, but the number of patients was unclear [45].

Concerning bone marrow metastases the risk of bias on flow and timing was low in nine (64%) [25,27,29,32,40,43,45-47], unclear in 1 [34] and high in 4 (29%) studies. The reason for a high risk of bias was in the first study that three patients were not imaged with all possible MRI-sequences described in the study [41]. Of the 97 patients included in the second study bone marrow metastases were described in only 34 patients [38]. From 5 patients in the third study one was excluded from the analysis, because it was unknown whether the bone marrow was investigated [44]. In the last study, in 1 of 9 patients bone marrow biopsy was not performed [42].

Discussion

Because the presence of bone and bone marrow metastases are independent adverse prognostic factors, it is crucial to use consistent definitions of both bone and bone marrow metastases.

This systematic review showed that many studies did not provide a definition for bone and/or bone marrow metastases (n=41) or did not distinguish between both types of metastases (n=15). 31 studies did provide definitions (n=23 for bone marrow and n=18 for bone metastases), but these were not uniform. When focussing on the most used definitions for bone metastases, these were predominantly “focal uptake” on MIBG, bone scintigraphy and 18F-FDG-PET-CT. For bone marrow metastases, these were mainly described as “diffuse lesions” on MIBG scintigraphy, but on MRI as both “diffuse” and “focal lesions”.

So we conclude that, because of the wide variety of the definitions, results of different studies should be interpreted with caution, and international standards are needed. Along with the variable definitions for bone and bone marrow metastases, the sensitivity and specificity varied widely between the studies.

For MIBG scintigraphy pooling of data on diagnostic accuracy was possible for some of the studies. When using the definitions as “focal uptake” or “hotspots” to detect bone metastases, the sensitivity was 0.88, but the specificity was 0.79. The low specificity can be explained by the 29 of 202 (14%) lesions that were thought to be false positive (according to the bone scintigraphy), that might in fact be true positive, because bone remodelling was not yet present on bone scintigraphy.

To be able to define the diagnostic value of all different definitions on all different imaging techniques, ideally, uniform reference standards should be used.

Since the late 1970s, bone scintigraphy has been the main diagnostic method for the detection of cortical skeletal metastases [48,49]. In this review, bone scintigraphy was therefore used as the reference standard to detect bone metastases. It portrays bone metastases of an osteoblastic type, while bone metastases of neuroblastomas are generally of the osteoclastic type [50]. As a result, bone metastases of neuroblastoma are depicted at a more advanced stage of the disease, when bone remodelling takes place and smaller lesions might be missed [51]. It has a reported sensitivity and specificity of 70-78% and 51%, respectively [52,53]. So false-negative as well as false-positive results are a problem when using this imaging technique as a reference standard. Nowadays, bone scintigraphy is usually not required, except in cases in which the primary tumor is not MIBG-avid or MIBG-positivity cannot be confirmed (i.e., if the primary tumor has been removed before examination). However, many studies in this review were published in an era where bone scintigraphy was mostly used as the reference standard.

Ideally, bone lesions should be confirmed by histology, but it is not common practice to biopsy bone metastases. Only in case of a single equivocal lesion on MIBG, the INRG recommends confirmation by another imaging modality (plain radiographs, and if negative, MRI and/ or biopsy) [12,52]. 6 of the evaluated studies in this review described a case of a suspected bone lesion that was biopsied. Only 3 out of 6 were confirmed neuroblastoma bone metastases (Supplementary Table 3). Due to the small numbers no conclusions could be drawn from these studies.

The reference standard for bone marrow metastases, in this review, were bone marrow biopsies (trephines) or aspirates from iliac crests, as recommended by the INRG [11,12]. Generally, if one of these samples is positive, patients are considered to have bone marrow invasion. However, bone marrow can be infiltrated in other sites outside the pelvic area. Imaging techniques that show tumor infiltration in other localizations can be considered erroneously false positive compared to their reference standard bone marrow biopsies/aspirates. For example, in the study of Hanna et al. 4 MRI-T1 and STIR scans were judged false positive compared to the reference standard bone marrow biopsies/aspirates [41]. This might be caused by lesions on the MRI that are located outside the iliac crests. However, the study did not report on the localisation of the false-positive lesions. 1 of these 4 MRI’s was false-positive during follow-up, and therefore post-therapy marrow signal alteration might also explain this false-positive result. Claudiani et al. reported a false positive MIBG scan [38], as stated above, if this was caused by a bone marrow metastasis located outside the iliac crest, this might in fact have been a true positive. The pooled MIBG scintigraphy data for the bone marrow metastases (definition “diffuse uptake”) generated a low sensitivity of only 0.60, caused by a lot of false-negative results (50 of 364, 14%) on MIBG-scintigraphy. As bone marrow biopsies/aspirates were the golden standard, these results were really false-negative. One can imagine that a low level of infiltrating tumor cells can accumulate MIBG, but fail to be detected on whole body scans. In comparison with a sensitivity of 90-100% for the detection of neuroblastoma in general, the sensitivity to detect bone marrow metastases is rather low [48].

Despite the extensive use of MIBG scoring methods such as the Curie and SIOPEN method to assess response to therapy, MIBG avidity is not allocated to either bone or bone marrow [53-55]. The difficulty of MIBG scintigraphy to differentiate bone from bone marrow metastases is also a consequence of its two-dimensional nature. Routine use of SPECT-CT may overcome some of these limitations.

MRI has a unique soft tissue contrast, and therefore anatomical localization of lesions with MRI is very well possible. Furthermore MRI might provide early detection of bone marrow infiltration by the tumor before osseous destruction becomes apparent on radiography or CT or before metabolic changes occur on bone scintigraphy or PET-CT [56-58]. However, in children, in contrast to adults, it can be difficult to differentiate highly cellular hematopoietic marrow (red marrow) from metastatic disease [59]. 18F-FDG-PET-CT can reveal early malignant bone marrow infiltration because of its increased glucose metabolism. In combination with CT, anatomical localization of abnormal signal is very well possible. However, the brain massively accumulates 18F-FDGPET and therefore metastatic lesions in the skull can be missed [48]. Also, accumulation in brown adipose tissue and cytokine-mediated diffusely hyper-metabolic bone marrow, as can be seen with the use of G-CSF, might result in false-positive images [60-63]. Data on 18F-FDGPET- CT in patients with neuroblastoma are still limited and it is mostly used in patients with neuroblastoma, when the tumor does not, or weakly accumulate MIBG [48].

Neuroblastoma is a unique tumor, because already at diagnosis 50% of patients present with metastatic disease. Therefore it is possible to study diagnostic imaging in these patients before any treatment. We do not know of any childhood or adult tumor with such an extensive number of patients with metastatic disease at diagnosis. Therefore, it was not possible to reliably compare our results with other types of cancer.

From the literature, we concluded that the distinction between bone and bone marrow metastases on imaging can be difficult. Furthermore, we concluded that for the detection of bone and bone marrow metastases on imaging, there are currently no unambiguous definitions, so the value of the prognostic significance as reported in the past should be interpreted with caution. Because of this problem, when using imaging modalities such as MIBG scans, the term skeletal might be more appropriate, where both bone and bone marrow lesions are combined. Brisse et al. described MIBG uptake using the term “skeletal metastases, in bone and bone marrow in a recent consensus paper on imaging of neuroblastoma [52]. In 15 of the 101 excluded studies in this review, this term was also used, but only 5 out of these 15 studies gave a definition of “skeletal metastases” (Supplementary Table 4).

The quality of the included studies in objective 2 was hard to assess. The scoring of QUADAS-items resulted mostly in an unclear risk of bias, because of the many missing items. We strongly recommend describing these items more precisely in future papers.

Future Perspectives

When focusing on MIBG scans, the use of SPECT-CT might enable this technique to better differentiate between localisations of the metastases. Because addition of CT increases the radiation burden, it is not recommendable to add this technique routinely for each lesion. However, the available MIBG-SPECT-CT’s of body parts should be investigated for the possibility to differentiate bone and bone marrow metastases.

Another technique that shows promise is MRI imaging, especially combined with the high sensitivity of MIBG, this would be ideal to identify all involved skeletal lesions. Therefore, these two imaging techniques should be compared to see whether it is possible to differentiate between bone and bone marrow metastases on these imaging methods.

Implications for Clinical Practice

Up till now, MIBG scintigraphy is the most widely used imaging method to diagnose neuroblastoma, but no uniform definitions of bone and bone marrow metastases are described for this technique. The role of 18F-FDG-PET-CT and MRI in the detection of osteomedullary metastases in patients with neuroblastoma is still under investigation.

Because currently no uniform definitions are available to differentiate between bone and bone marrow, we suggest using the term “skeletal” or “osteomedullary” metastases until it is possible to discriminate these on imaging.

Acknowledgements

We would like to thank AGE Leenders, Medical library, Academic Medical Centre (AMC), University of Amsterdam, Amsterdam, The Netherlands, for his work on the search of this review.

Funding

This work was supported by Kika (Children Cancer Free Foundation) and Tom Voûte foundation.

References

  1. Brodeur GM (2003) Neuroblastoma: biological insights into a clinical enigma. Nat Rev Cancer 3: 203-216.
  2. Maris JM, Hogarty MD, Bagatell R, Cohn SL (2007) Neuroblastoma. Lancet 369:2106-2120.
  3. Maris JM (2010) Recent advances in neuroblastoma. N Engl J Med 362:2202-2211.
  4. Park JR, Eggert A, Caron H (2008)Neuroblastoma: biology, prognosis, and treatment. Pediatr Clin North Am 55: 97-120.
  5. Yu AL, Gilman AL, Ozkaynak MF, London WB, Kreissman SG, et al. (2010) Anti-GD2 antibody with GM-CSF, interleukin-2, and isotretinoinfor neuroblastoma. N Engl J Med 363:1324-1334.
  6. Cotterill SJ, Pearson AD, Pritchard J, Foot AB, Roald B, et al. (2000) Clinical prognostic factors in 1277 patients with neuroblastoma: results of The European Neuroblastoma Study Group 'Survey' 1982-1992. Eur J Cancer 36:901-908.
  7. DuBois SG, Kalika Y, Lukens JN, Brodeur GM, Seeger RC, et al. (1999) Metastatic sites in stage IV and IVS neuroblastoma correlate with age, tumor biology, and survival. JPediatr Hematol Oncol 21:181-189.
  8. Minard V, Hartmann O, Peyroulet MC, Michon J, Coze C, et al. (2000)Adverse outcome of infants with metastatic neuroblastoma, MYCN amplification and/or bone lesions: results of the French society of pediatric oncology. Br J Cancer 83:973-979.
  9. Moroz V, Machin D, Faldum A, Hero B, Iehara T, et al. (2011) Changes over three decades in outcome and the prognostic influence of age-at-diagnosis in young patients with neuroblastoma: a report from the International Neuroblastoma Risk Group Project. Eur J Cancer 247:561-571.
  10. Ladenstein R, Philip T, Lasset C, Hartmann O, Garaventa A, et al. (1998) Multivariate analysis of risk factors in stage 4 neuroblastoma patients over the age of one year treated with megatherapy and stem-cell transplantation: a report from the European Bone Marrow Transplantation Solid Tumor Registry. J ClinOncol 16:953-965.
  11. Beiske K, Burchill SA, Cheung IY, Hiyama E, Seeger RC, et al. (2009) Consensus criteria for sensitive detection of minimal neuroblastoma cells in bone marrow, blood and stem cell preparations by immunocytology and QRT-PCR: recommendations by the International Neuroblastoma Risk Group Task Force. Br J Cancer 100:1627-1637.
  12. Monclair T, Brodeur GM, Ambros PF, Brisse HJ, Cecchetto G, et al. (2009) The International Neuroblastoma Risk Group (INRG) staging system: an INRG Task Force report. J ClinOncol 27:298-303.
  13. Olivier P, Colarinha P, Fettich J, Fischer S, Frokier J, et al. (2003) Guidelines for radioiodinated MIBG scintigraphy in children. Eur J Nucl Med Mol Imaging 30:B45-B50.
  14. Charron M, Brown M, Rowland P, Mirro J (1996) Pain palliation with strontium-89 in children with metastatic disease. Med Pediatr Oncol 26:393-396.
  15. Moher D, Liberati A, Tetzlaff J, Altman DG (2009)Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement.Ann Intern Med 151: 264-269.
  16. Whiting PF, Rutjes AW, Westwood ME, Mallett S, Deeks JJ, et al. (2011) QUADAS-2: a revised tool for the quality assessment of diagnostic accuracy studies. Ann Intern Med 155:529-536.
  17. Naranjo A, Parisi MT, Shulkin BL, London WB, Matthay KK, et al. (2011) Comparison of 123I-metaiodobenzylguanidine (MIBG) and 131I-MIBG semi-quantitative scores in predicting survival in patients with stage 4 neuroblastoma: a report from the Children's Oncology Group. Pediatr Blood Cancer 56:1041-1045.
  18. Chirathivat S, Post MJ (1980) CT demonstration of dural metastases in neuroblastoma. J Comput Assist Tomogr 14:316-319.
  19. Chu CM, Rasalkar DD, Hu YJ, Cheng FW, Li CK, et al. (2011) Clinical presentations and imaging findings of neuroblastoma beyond abdominal mass and a review of imaging algorithm. Br J Radio l84:81-91.
  20. Grover SB,Pati NK, Saluja S, Bhowmik KT (2003) Solitary calvarial metastases: an unusual presentation of thoracic neuroblastoma. Indian J Cancer 40:120-122.
  21. Howman-Giles RB, Gilday DL, Ash JM (1979) Radionuclide skeletal survey in neuroblastoma. Radiology 131:497-502.
  22. Ruzal-Shapiro C, Berdon WE, Cohen MD, Abramson SJ (1991) MR imaging of diffuse bone marrow replacement in pediatric patients with cancer. Radiology 181:587-589.
  23. Shulkin BL, Shapiro B, Hutchinson RJ (1992) Iodine-131-metaiodobenzylguanidine and bone scintigraphy for the detection of neuroblastoma. J Nucl Med 33:1735-1740.
  24. Siegel MJ, Ishwaran H, Fletcher BD, Meyer JS, Hoffer FA, et al. (2002) Staging of neuroblastoma at imaging: report of the radiology diagnostic oncology group. Radiology 223:168-175.
  25. Berberoğlu K, Unal SN, Kebudi R, Türkmen C, Cantez S (2005) Role of 99mTc-hexakis-2-methoxyisobutylisonitrile for detecting marrow metastases in childhood solid tumours. Nucl Med Commun26:1075-1080.
  26. Cohen MD, Klatte EC, Baehner R, Smith JA, Martin-Simmerman P, et al. (1984) Magnetic resonance imaging of bone marrow disease in children. Radiology 151:715-718.
  27. Corbett R, Olliff J, Fairley N, Moyes J, Husband J, et al. (1991) A prospective comparison between magnetic resonance imaging, meta-iodobenzylguanidinescintigraphy and marrow histology/cytology in neuroblastoma. Eur J Cancer 27:1560-1564.
  28. Fletcher BD, Miraldi FD, Cheung NK (1989) Comparison of radiolabeled monoclonal antibody and magnetic resonance imaging in the detection of metastatic neuroblastoma in bone marrow: preliminary results. PediatrRadiol 20:72-75.
  29. Frappaz D, Combaret V, Desuzinges C, Bouffet E, Bailly C, et al. (2004) Can MIBG scan replace the need for bone marrow assessment at diagnosis and reassessment in stage 4 neuroblastomas? Bull Cancer 91:E253-E260.
  30. Goo HW, Choi SH, Ghim T, Moon HN, Seo JJ (2005) Whole-body MRI of paediatric malignant tumours: comparison with conventional oncological imaging methods. PediatrRadiol35:766-773.
  31. Lebtahi N, Gudinchet F, Nenadov-Beck M, Beck D, BischofDelaloye A (1997) Evaluating bone marrow metastasis of neuroblastoma with iodine-123-MIBG scintigraphy and MRI. J Nucl Med 38:1389-1392.
  32. Meyer JS, Siegel MJ, Farooqui SO, Jaramillo D, Fletcher BD, et al. (2005) Which MRI sequence of the spine best reveals bone-marrow metastases of neuroblastoma? PediatrRadiol35:778-785.
  33. Shulkin BL, Shen SW, Sisson JC, Shapiro B (1987) Iodine-131 MIBG scintigraphy of the extremities in metastatic pheochromocytoma and neuroblastoma. J Nucl Med 28:315-318.
  34. Tanabe M, Takahashi H, Ohnuma N, Iwai J, Yoshida H (1993) Evaluation of bone marrow metastasis of neuroblastoma and changes after chemotherapy by MRI. Med PediatrOncol21:54-59.
  35. Tanabe M, Ohnuma N, Iwai J, Yoshida H, Takahashi H, et al. (1995) Bone marrow metastasis of neuroblastoma analyzed by MRI and its influence on prognosis. Med PediatrOncol24:292-299.
  36. Ueno S, Yokoyama S, Hirakawa H, Yabe H, Suzuki Y, et al. (2002) Use of real-time magnetic resonance guidance to assist bone biopsy in pediatric malignancy. Pediatrics 109:E18.
  37. Haddad A, Azouz EM, al-Jedher S, Baruchel S, Bernstein ML, et al. (1992) Metastatic neuroblastoma presenting as a mandibular mass. Can AssocRadiol J 43:436-440.
  38. Claudiani F, Stimamiglio P, Bertolazzi L, Cabria M, Conte M, et al. (1995) Radioiodinated meta-iodobenzylguanidine in the diagnosis of childhood neuroblastoma. Q J Nucl Med 39: 21-24.
  39. Giammarile F, Olier AL, Lumbroso J (1995) Diffuse bone marrow uptake [123I]MIBG in neuroblastoma: an "MIBG super scan" case report. Q J Nucl Med 39:119-121.
  40. Hadj-Djilani NL, Lebtahi NE, Delaloye AB, Laurini R, Beck D (1995) Diagnosis and follow-up of neuroblastoma by means of iodine-123 metaiodobenzylguanidinescintigraphy and bone scan, and the influence of histology. Eur J Nucl Med 22:322-329.
  41. Hanna SL, Fletcher BD, Fairclough DL, Jenkins JH 3rd, Le AH (1991) Magnetic resonance imaging of disseminated bone marrow disease in patients treated for malignancy. Skeletal Radiol20:79-84.
  42. Juweid ME, Menda Y, O'Dorisio MS, Bushnell D, Blake M, et al. (2002) 111In-pentetreotide versus bone scintigraphy in the detection of bony metastases of neuroblastoma. Nucl Med Commun23:983-989.
  43. Kushner BH, Yeung HW, Larson SM, Kramer K, Cheung NK (2001) Extending positron emission tomography scan utility to high-risk neuroblastoma: fluorine-18 fluorodeoxyglucose positron emission tomography as sole imaging modality in follow-up of patients. J ClinOncol19:3397-3405.
  44. Okuyama C, Ushuima Y, Nakamura T, Kikkawa M, Nishimura T (2001) Thallium-201 scintigraphy of neuroblastoma: different results for primary tumors and skeletal lesions. Ann Nucl Med 15:171-177.
  45. Osmanagaoglu K, Lippens M, Benoit Y, Obrie E, Schelstraete K, et al. (1993) A comparison of iodine-123 meta-iodobenzylguanidinescintigraphy and single bone marrow aspiration biopsy in the diagnosis and follow-up of 26 children with neuroblastoma. Eur J Nucl Med 20:1154-1160.
  46. Shah Syed GM,Naseer H, Usmani GN, Cheema MA (2004) Role of iodine-131 MIBG scanning in the management of paediatric patients with neuroblastoma. Med PrincPract13:196-200.
  47. Turba E, Fagioli G, Mancini AF, Rosito P, Galli A, et al. (1993) Evaluation of stage 4 neuroblastoma patients by means of MIBG and 99mTc-MDP scintigraphy. J NuclBiol Med 37:107-114.
  48. Sharp SE, Parisi MT, Gelfand MJ, Yanik GA, Shulkin BL (2013) Functional-metabolic imaging of neuroblastoma. Q J Nucl Med Mol Imaging 57: 6-20.
  49. Kushner BH (2004) Neuroblastoma: a disease requiring a multitude of imaging studies. J Nucl Med 45: 1172-1188.
  50. Vinholes J, Vinholes APB, Coleman RE (1996) Imaging of metastatic bone disease. Imaging 8: 256-273.
  51. Kumar J, Seith A, Kumar A, Sharma R, Bakhshi S, et al. (2008) Whole-body MR imaging with the use of parallel imaging for detection of skeletal metastases in pediatric patients with small-cell neoplasms: Comparison with skeletal scintigraphy and FDG PET/CT. PediatrRadiol 38: 953-962.
  52. Brisse HJ, McCarville MB, Granata C, Krug KB, Wootton-Gorges SL, et al (2011) Guidelines for imaging and staging of neuroblastic tumors: Consensus report from the International Neuroblastoma Risk Group Project. Radiology 261: 243-257.
  53. Sharp SE, Gelfand MJ, Shulkin BL (2011) Pediatrics: diagnosis of neuroblastoma. SeminNucl Med 41: 345-353.
  54. Ady N, Zucker JM, Asselain B, Edeline V, Bonnin F, et al (1995) A new 123I-MIBG whole body scan scoring method--application to the prediction of the response of metastases to induction chemotherapy in stage IV neuroblastoma. Eur J Cancer 31A: 256-261.
  55. Matthay KK, Shulkin B, Ladenstein R, Michon J, Giammarile F, et al. (2010) Criteria for evaluation of disease extent by (123)I-metaiodobenzylguanidine scans in neuroblastoma: a report for the International Neuroblastoma Risk Group (INRG) Task Force. Br J Cancer 102: 1319-1326.
  56. Burdiles A, Babyn PS (2009) Pediatric bone marrow MR imaging. MagnReson Imaging Clin N Am 17: 391-409.
  57. Hwang S, Panicek DM (2007) Magnetic resonance imaging of bone marrow in oncology, Part 2. Skeletal Radiol 36: 1017-1027.
  58. Nour-Eldin NEA, Abdelmonem O, Tawfik AM, Naguib NNN, Klingebiel T, et al (2012) Pediatric primary and metastatic neuroblastoma: MRI findings. Pictorial review. MagnReson Imaging 30: 893-906.
  59. Hwang S, Panicek DM (2007) Magnetic resonance imaging of bone marrow in oncology, Part 1. Skeletal Radiol 36: 913-920.
  60. Portwine C, Marriott C, Barr RD (2010) PET imaging for pediatric oncology: An assessment of the evidence. Pediatr Blood Cancer 55: 1048-1061.
  61. Kushner BH, LaQuaglia MP, Kramer K, Cheung NK (2004)Radically different treatment recommendations for newly diagnosed neuroblastoma: pitfalls in assessment of risk. J PediatrHematolOncol26:35-39.
  62. Labreveux de CervensC, Hartmann O, Bonnin F, Couanet D, Valteau-Couanet D, et al. (1994) What is the prognostic value of osteomedullary uptake on MIBG scan in neuroblastoma patients under one year of age? Med PediatrOncol 22:107-114.
  63. Siegel MJ, Acharyya S, Hoffer FA, Wyly JB, Friedmann AM, et al. (2013) Whole-body MR imaging for staging of malignant tumors in pediatric patients: results of the American College of Radiology Imaging Network 6660 Trial. Radiology 266:599-609.
Citation: Bleeker G, Heijkoop D, van Dalen EC, Kremer LC, Smets AM, et al. (2015) A Systematic Literature Review of Imaging Definitions for Detection of Bone and Bone Marrow Metastases in Neuroblastoma Patients. J Bone Marrow Res 3:163.

Copyright: © 2015 Bleeker G, et al. This is an open-access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Top