Background and Objectives for the Systematic Review

Background and Objectives

Hematopoietic stem-cell transplantation (HSCT) refers to a procedure in which hematopoietic stem cells are infused to restore bone marrow function in patients, and is categorized by the source of the stem cells. The role of HSCT in pediatric diseases depends in part on the indication for which it is being used. Autologous transplants involve harvesting the patient’s own stem cells and then returning them, typically after the patient has received doses of chemotherapy that are myeloablative. Allogeneic HSCT uses stem cells from a human leukocyte antigen (HLA) -matched donor, either related or unrelated, and, in malignant diseases, exploits a graft-versus-tumor effect. In many of the nonmalignant diseases, donor hematopoietic stem cells are used to “replace” the patient’s abnormal bone marrow with a donor’s normal one. The third source of hematopoietic stem cells is from the umbilical cord and placenta.

In the pediatric population, HSCT is used to treat a wide variety of diseases, both malignant and nonmalignant. Malignant diseases in the pediatric population include hematologic (leukemias and lymphomas) and nonhematologic (or solid tumors) which include primary brain tumors, and those derived from the peripheral nervous system (neuroblastoma), soft tissues or bone (Ewing’s sarcoma, rhabdomyosarcoma), kidney (Wilms tumor) and eye (retinoblastoma).

A wide variety of nonmalignant conditions may also be treated with HSCT, and in the pediatric population include hemoglobinopathies, bone marrow failure (BMF) syndromes, primary immune deficiencies (PID), autoimmune diseases and inherited metabolic diseases (IMD).

For many of these diseases, HSCT is a well-established treatment. For example, the literature for the use of HSCT in the hematologic malignancies is relatively robust, including randomized, controlled trials that date back 10 to 15 years, and it has become common and accepted medical practice. For other less-common diseases, for example, the primary immunodeficiencies and hemoglobinopathies, the evidence consists of case series and case reports, but the results after HSCT have shown improved outcomes and/or cure. In other pediatric diseases, HSCT is not well established, is evolving, or has been shown not to be beneficial. Although our approach to this topic does not fit the usual AHRQ comparative effectiveness review (CER) format, we feel that the stakeholders would be best served by addressing this topic in part as a narrative review for those diseases in which HSCT in pediatric patients is a well-established treatment option, with the remainder of the diseases approached as a systematic comparative effectiveness review to guide decision makers as to the comparative effectiveness of HSCT to comparators. (See Tables 1 and 2 in Methods [Section VI]).

The success of treating many of the pediatric diseases with HSCT has resulted in an increased number of long-term survivors. Despite this increase in survival, both immediate harms associated with HSCT (e.g., graft-versus-host disease and opportunistic infections) and long-term effects (e.g., growth and development, cognitive ability, future fertility, and secondary malignancies) need to be taken into account. The comparative benefits and harms of HSCT versus other therapies in the pediatric population for these various diseases form the basis for the key questions developed for this topic.

The Topic Nominator requested a broad and comprehensive review of the literature for which diseases HSCT may be a treatment option in the pediatric population. To achieve this, we will address six key questions (see “The Key Questions,” following).

The Key Questions

The proposed Key Questions were posted for public comment. Following the public posting of the key questions, several comments were received and discussed. In summary, the public comments addressed three main points. First, while successes have been seen with HSCT in many pediatric conditions, the measurement of comparative outcomes after HSCT is difficult due to the rarity of the conditions (e.g., retinoblastoma) and/or the number of transplants completed (e.g., autoimmune diseases). Second, comparative harms data are equally difficult to obtain, but further separating out the harms associated with HSCT from the harms associated with other prior treatments or disease natural history is not possible in many cases. Third, it was suggested that we contact the Pediatric Blood and Marrow Transplant Consortium and Center for International Blood and Marrow Transplant Research (CIBMTR) to see if they could provide some pertinent information. These comments were discussed and the resulting six key questions are listed below, followed by the PICOTS (Patient, Intervention, Comparator, Outcome, Timing, Setting) for the three indications addressed by the systematic review. No major changes were made to the key questions following the public comments.

Key Question 1. For pediatric patients with malignant solid tumors, what is the comparative effectiveness of HSCT and conventional chemotherapy regarding overall survival, long-term consequences of HSCT, and quality of life?

Key Question 2. For pediatric patients with malignant solid tumors, what are the comparative harms of HSCT and conventional chemotherapy regarding adverse effects of treatment, long term consequences of HSCT, and impaired quality of life?

Key Question 3. For pediatric patients with inherited metabolic diseases, what is the comparative effectiveness of HSCT, enzyme-replacement therapy (ERT), substrate reduction with iminosugars, and chaperones regarding overall survival, cure, long-term consequences of HSCT, and quality of life?

Key Question 4. For pediatric patients with inherited metabolic diseases, what are the comparative harms of HSCT, enzyme-replacement therapy (ERT), substrate reduction with iminosugars, and chaperones regarding adverse effects of treatment, long term consequences of HSCT, and impaired quality of life?

Key Question 5. For pediatric patients with autoimmune diseases, what is the comparative effectiveness of HSCT, immunosuppressants, target biologic therapies, and low dose chemotherapy regarding overall survival, cure, and remission?

Key Question 6. For pediatric patients with autoimmune diseases, what are the comparative harms of HSCT, immunosuppressants, target biologic therapies, and low dose chemotherapy regarding adverse effects of treatment, long term consequences of HSCT, and impaired quality of life?

PICOTS

Indication 1. Malignant Solid Tumors (Key Questions 1 and 2)

P:

Pediatric patients with malignant solid tumors including rhabdomyosarcoma and retinoblastoma

I:

Hematopoietic stem cell transplantation (HSCT)

C:

Conventional chemotherapy

O:

Overall survival (OS); long-term consequences of HSCT; quality of life (QOL)

T:

All durations of follow-up will be included

S:

Inpatient

Indication 2. Inherited Metabolic Disease (Key Questions 3 and 4)

P:

Pediatric patients with inherited metabolic diseases

I:

Hematopoietic stem cell transplantation (HSCT)

C:

Enzyme-replacement therapy (ERT), substrate reduction with iminosugars, chaperones

O:

OS; cure; long term consequences of HSCT; QOL

T:

All durations of follow-up will be included

S:

Inpatient

Indication 3. Autoimmune Disease (Key Questions 5 and 6)

P:

Pediatric patients with autoimmune diseases

I:

Hematopoietic stem cell transplantation (HSCT)

C:

Immunosuppressants, targeted biologic therapies, low dose chemotherapy

O:

Remission, survival, cure

T:

All durations of follow-up will be included

S:

Inpatient

Important refinement points:

In addition to the public comments, the six key questions were discussed with a group of Key Informants. This discussion led to further refinement and clarification of the scope of the project. The three major points from that discussion are detailed below.

• It was reiterated that the topic will be handled as both a narrative review (for diseases for which HSCT is a well-established treatment option) and as a systematic review (for those diseases for which HSCT is either not a well-established or not a recommended treatment option). Key Informants provided two initial lists of diseases for which they believed HSCT to either be established (narrative) or investigational (systematic). After internal review of the lists and the literature, some changes were made (e.g., adding tandem transplants in the systematic review). After the creation of the Technical Expert Panel (TEP), a final review of the list was completed by its members.
• Further detailing that “inherited metabolic diseases” include the following:
  • mucopolysaccharidoses
  • sphingolipidoses
  • glycocoproteinoses
  • other lipidoses
  • glycogen storage diseases
  • multiple enzyme deficiencies
  • lysosomal transport defects
  • peroxisomal storage disorders
• Restating that the decision not to divide the review into type or source of transplant (allogeneic versus autologous versus reduced-intensity allogeneic, related versus unrelated) or by different conditioning regimens was made at the recommendation of the Key Informants. This is due to the variation in treatment approaches by different institutions and practitioners, as well as the individualization of treatment being made based on patient history, treatment goals, overall patient health status, etc. However, these issues will be addressed in a narrative summary in the background section of the CER.

Analytic Framework

Methods

In consultation with Key Informants and the Topic Nominator, it was decided that the stakeholders would be best served by addressing this topic in part as a narrative review for those diseases in which HSCT in pediatric patients is a well-established treatment option, with the remainder of the diseases approached as a systematic, comparative effectiveness review to guide decision makers as to appropriate indications for the use of HSCT. Tables 1 and 2 display the diseases to be approached as a narrative and systematic review, respectively.

Methods described are generally applicable to all Key Questions, including Methods of the Review, Evidence Tables, Identifying Additional Studies, and Assessing Study Quality. However, certain aspects of the Methods may vary to satisfy disease-specific requirements of each question in the narrative reviews. Specific methods sections for the narrative and systematic review are detailed.

A. Criteria for Inclusion/Exclusion of Studies in the Review

Prior literature review has confirmed that the highest level of evidence available for the systematic reviews is case series of various sizes and case reports. The abstracts of all identified articles will be reviewed and coded for relevance and completeness. On the advice of the Technical Expert Panel, to ensure comparative data from contemporary treatment regimens, articles older than 15 years will not be retrieved, with the exception of natural history data for the inherited metabolic diseases. This cut off may be altered if older articles are deemed sufficiently similar to contemporary practice to be comparable. Studies not published in English will be excluded. For all key questions, all articles published in English will be systematically reviewed for inclusion.

Studies will be included for Key Question 1 and Key Question 2 if they:

• report on an outcome of interest specifically among pediatric patients with malignant solid tumors and;
• involve a uniform intervention of interest.

Studies will be included for Key Question 3 and Key Question 4 if they:

• report on an outcome of interest specifically among pediatric patients with inherited metabolic disease and;
• involve a uniform intervention of interest or;
• report on the natural history of an inherited metabolic disease in the absence of treatment.

Studies will be included for Key Question 5 and Key Question 6 if they:

• report on an outcome of interest specifically among pediatric patients with autoimmune disease and;
• involve a uniform intervention of interest.

B. Searching for the Evidence: Literature Search Strategies for Identification of Relevant Studies to Answer the Key Questions.

Narrative Review

In accordance with the recommendation from the TEP, for each disease in the narrative review existing literature was searched for recent review articles, guidelines and systematic reviews. Electronic databases to be searched include MEDLINE®, EMBASE®, and the Cochrane Controlled Trials Register. These articles were then used to help identify the most relevant clinical studies or case series for inclusion. As HSCT is considered established therapy for the diseases in the narrative review, the TEP believed this to be the best approach.

Initial searches of the MEDLINE® database via PubMed were performed. Table 3 displays the level of evidence available for the narrative review by disease group. However this is not a systematic review; therefore, a reconciled database will not be created.

Systematic Review

Electronic databases to be searched include MEDLINE®, EMBASE®, and the Cochrane Controlled Trials Register. Databases will be searched without restriction on date as many of these diseases are rare and natural history data may be hard to capture in the contemporary literature. The exact search strategy is shown in the Appendix (Section XI).

Literature searches will be updated prior to finalizing a draft to ensure the identification of new literature, which may potentially impact the review. New studies will be evaluated against inclusion/exclusion criteria in the same manner as all other studies. In addition to updating the initial searches, members of the Technical Expert Panel and individuals and organizations providing peer review will be asked to inform the project team of any studies relevant to the key questions addressed in this evidence report that are not included in the list of selected studies. These studies will be reviewed and included if they meet the inclusion criteria set out above and alter the report’s conclusions.

C. Data Abstraction and Data Management

Study Selection:

One (of three) reviewer will screen titles and abstracts of literature identified from the above sources for the eligibility criteria previously stated. For those items where a more detailed screen is needed to determine eligibility, we will obtain a full-text version for assessment. We will assess literature that appears to meet the inclusion criteria in the initial screening for eligibility with an eligibility form containing the following questions:

1. Is the treatment setting for the particular disease indication included in the systematic review? AND
2. Did the patients receive a uniform intervention of interest? AND
3. Did the article report on a relevant outcome of interest? OR
4. Did the article report on a relevant comparator of interest? OR
5. For inherited metabolic diseases does the article report on disease natural history?

To be included, articles must meet criteria 1-3 or criteria 1 and 4 or 5. Any disagreements between reviewers will be resolved through discussion. If a reviewer is uncertain if a study should be selected for inclusion, this will be resolved through discussion at team meetings. Duplicate studies will be identified and data will be extracted from the study with the longest follow-up. Full-text versions of all eligible studies will be obtained for quality assessment and data extraction.

Data Abstraction:

One (of three) reviewer will perform data abstraction for the review using a standardized data extraction form. A second reviewer will fact-check the data. Since the evidence base for this systematic review is comprised primarily of case reports and case series of various sizes, elements that would have been abstracted from RCT and non-randomized controlled trials will be excluded from the data abstraction protocol. Prior to the start of data abstraction, data elements to be abstracted were defined in consultation with the Technical Expert Panel. The following data elements of primary studies will be abstracted from the articles meeting all selection criteria.

• critical features of the study design
  • patient inclusion/exclusion criteria
  • number of participants and flow of participants through steps of study
• patient characteristics, including:
  • age
  • sex
  • race/ethnicity
  • disease and stage
  • disease duration
  • other prognostic characteristics
• treatment characteristics, including
  • stem-cell source
  • chemotherapy vs. chemo-radiotherapy
  • immunosuppressive therapy as prophylaxis for graft versus host disease
  • supportive care
• outcome assessment details
  • identified primary outcome
  • secondary outcomes
  • response criteria
  • use of independent outcome assessor
  • follow-up frequency and duration
• data analysis details
  • statistical analyses (statistical test/estimation results)
    • test used
    • summary measures
    • sample variability measures
    • precision of estimate
    • p values

Evidence Tables:

We will create templates for evidence tables in Microsoft Excel® and Microsoft Word®. One reviewer will perform primary data abstraction of all data elements into the evidence tables, and a second reviewer will perform accuracy checks on the evidence tables.

PRISMA:

A PRISMA diagram will be constructed for each Key Question.¹

Assessment of Applicability:

Applicability of findings in this review will be assessed within the EPICOT framework (Evidence, Population, Intervention, Comparison, Outcome, Time stamp.² Selected studies will be assessed for relevance against target populations, interventions of interest and outcomes of interest.

D. Assessment of Methodological Quality of Individual Studies

In consultation with the AHRQ Task Order Officer and Technical Expert Panel, the general approach to grading evidence developed by the U.S. Preventive Services Task Force³ will be applied to primary studies. The quality of the abstracted studies and the body of evidence will be assessed by two independent reviewers. Discordant quality assessments will be resolved with input from a third reviewer, if necessary.

a. The quality of RCTs will be assessed on the basis of the following criteria:

• Initial assembly of comparable groups: adequate randomization, including concealment and whether potential confounders (e.g., other concomitant care) were distributed equally among groups
• Maintenance of comparable groups (includes attrition, crossovers, adherence, contamination)
• Important differential loss to follow-up or overall high loss to follow-up
• Measurements: equal, reliable, and valid (includes masking of outcome assessment)
• Clear definition of interventions
• All important outcomes considered
• Analysis: adjustment for potential confounders, intention-to-treat analysis

Definition of ratings based on above criteria:

The rating of intervention studies encompasses the three quality categories described here.

Good:

Meets all criteria: Comparable groups are assembled initially and maintained throughout the study (follow-up at least 80 percent); reliable and valid measurement instruments are used and applied equally to the groups; interventions are spelled out clearly; all important outcomes are considered; and appropriate attention is given to confounders in analysis. In addition, for RCTs, intention-to-treat analysis is used.

Fair:

Studies will be graded “fair” if any or all of the following problems occur, without the fatal flaws noted in the “poor” category below: In general, comparable groups are assembled initially but some question remains whether some (although not major) differences occurred with follow-up; measurement instruments are acceptable (although not the best) and generally applied equally; some but not all important outcomes are considered; and some but not all potential confounders are accounted for. Intention-to-treat analysis is done for RCTs.

Poor:

Studies will be graded “poor” if any of the following fatal flaws exists: Groups assembled initially are not close to being comparable or maintained throughout the study; unreliable or invalid measurement instruments are used or not applied at all equally among groups (including not masking outcome assessment); and key confounders are given little or no attention. For RCTs, intention-to-treat analysis is lacking.

b. The quality of included nonrandomized comparative intervention studies will be assessed based on a selection of items proposed by Deeks et al.4 and informed by the Harris-USPSTF approach, as follows:

• Was sample definition and selection prospective or retrospective?
• Were inclusion/exclusion criteria clearly described?
• Were participants selected to be representative?
• Was there an attempt to balance groups by design?
• Were baseline prognostic characteristics clearly described and groups shown to be comparable?
• Were interventions clearly specified?
• Were participants in treatment groups recruited in the same time period?
• Was there an attempt by investigators to allocate participants to treatment groups in an attempt to minimize bias?
• Were concurrent/concomitant treatments clearly specified and given equally to treatment groups?
• Were outcome measures clearly valid, reliable and equally applied to treatment groups?
• Were outcome assessors blind?
• Was the length of follow-up adequate?
• Was attrition below an overall high level (<20%)?
• Was the difference in attrition between treatment groups below a high level (<15%)?
• Did the analysis of outcome data incorporate a method for handling confounders such as statistical adjustment?

c. The quality of included single-arm intervention studies will be assessed based on a set of study characteristics proposed by Carey and Boden,5 as follows:

• Clearly defined question
• Well-described study population
• Well-described intervention
• Use of validated outcome measures
• Appropriate statistical analyses
• Well-described results
• Discussion and conclusion supported by data
• Funding source acknowledged

d. The quality of the assessment of adverse effects data will be assessed by an empirically derived set of questions informed by the McMaster Quality Assessment Scale for Harms6 and CER Draft Methods Manual guidance. Questions are as follows:

• Is there an explanation of how harms were identified?
• Was a standardized or validated instrument or scale used?
• Was ascertainment similar and complete in all study groups?
• Was a measure of severity reported?
• Were harms attributed to the study intervention likely causally associated?
• Were the number and type of harmful events reported separately for study groups?

E. Data Synthesis

Given that a preliminary literature search identified only case series and case reports involving HSCT for the diseases of interest, this evidence review will not incorporate formal data synthesis using meta-analysis. Rather, the synthesis will emphasize the specific patient characteristics, specific outcomes and status relative to the evidence hierarchy/study quality assessment.

F. Grading the Evidence for Each Key Question

We will use the system for rating the strength of the overall body of evidence developed by the GRADE Working Group and modified by AHRQ.^7,8 The system used for rating the strength of the overall body of evidence was developed by AHRQ⁸ for the EPC Methods Guide, based on a system developed by the GRADE Working Group.⁷ This system explicitly addresses the following domains: risk of bias, consistency, directness and precision.

Grade of evidence strength is classified into the following four categories:

High quality:

Further research is very unlikely to change our confidence in the estimate of effect

Moderate quality:

Further research is likely to have an important impact on our confidence in the estimate of effect and may change the estimate

Low quality:

Further research is very likely to have an important impact on our confidence in the estimate of effect and is likely to change the estimate

Insufficient:

Any estimate of effect is very uncertain

If concerns arise with the body of evidence, additional domains will be addressed, such as coherence, dose-response relationship and residual confounding.

References

1. Liberati A, Altman DG, Tetzlaff J et al. The PRISMA statement for reporting systematic reviews and meta-analyses of studies that evaluate health care interventions: explanation and elaboration. Ann Intern Med 2009 Aug 18;151(4):W65-94.
2. Brown P, Brunnhuber K, Chalkidou K et al. How to formulate research recommendations. BMJ 2006 Oct 14;333(7572):804-6.
3. Harris RP, Helfand M, Woolf SH et al. Current methods of the US Preventive Services Task Force: a review of the process. Am J Prev Med 2001 Apr;20(3 Suppl):21-35.
4. Deeks JJ, Dinnes J, D'Amico R et al. Evaluating non-randomised intervention studies. Health Technol Assess 2003;7(27):iii-x, 1-173.
5. Carey TS, Boden SD. A critical guide to case series reports. Spine (Phila Pa 1976) 2003 Aug 1;28(15):1631-4.
6. Santaguida PL Raina P. McMaster Quality Assessment Scale of Harms (McHarm) for primary studies. 2005. Available online at: http://hiru.mcmaster.ca/epc/mcharm.pdf. Last accessed February 2010.
7. Guyatt GH, Oxman AD, Vist GE et al. GRADE: an emerging consensus on rating quality of evidence and strength of recommendations. BMJ 2008 Apr 26;336(7650):924-6.
8. Owens DK, Lohr KN, Atkins D et al. AHRQ series paper 5: grading the strength of a body of evidence when comparing medical interventions--agency for healthcare research and quality and the effective health-care program. J Clin Epidemiol 2010 May;63(5):513-23.

Definition of Terms

Hematopoietic stem-cell transplantation (HSCT) refers to a procedure in which hematopoietic stem cells are infused to restore bone marrow function in patients, and is categorized by the source of the stem cells.
Autologous transplants involve returning the patient’s own stem cells, typically after the patient has received doses of chemotherapy that are myeloablative or lymphoablative (for autoimmune disorders).
Allogeneic HSCT uses stem cells from an HLA-matched donor, either related or unrelated, and, in malignant diseases, exploits a graft-versus-tumor effect. Myeloablative or reduced-intensity (nonmyeloablative) conditioning regimens may be used.
Pediatric in this document refers to patients aged birth to 21 years. While the upper age limit varies, this definition is consistent with the definition found in several sources.

*Berhman RE et al. Nelson Textbook of Pediatrics, 15th Ed. Philadelphia: W.B. Saunders Company; 1996.
Rudolph AM et al. Rudolph’s Pediatrics, 21st Ed. New York: McGraw-Hill; 2002.
Avery MD, First LR. Pediatric Medicine, 2nd Ed. Baltimore: Williams & Wilkins; 1994.

Summary of Protocol Amendments

Appendix

All Child: 0-18 years=3709

Additional searching was done to obtain information on comparators in the following diseases: