Critical Reviews in Oncology / Hematology
Volume 73, Issue 1 , Pages 47-67 , January 2010

Development of symptom assessments utilising item response theory and computer-adaptive testing—A practical method based on a systematic review

  • Jochen Walker

      Affiliations

    • Oncological Palliative Medicine, Section Oncology/Haematology, Department of Internal Medicine and Palliative Care Center, Cantonal Hospital, Rorschacherstrasse 95, 9007 St. Gallen, Switzerland
    • ,
  • Jan R. Böhnke

      Affiliations

    • Clinical Psychology and Psychotherapy, University of Trier, 54286 Trier, Germany
    • ,
  • Thomas Cerny

      Affiliations

    • Oncological Palliative Medicine, Section Oncology/Haematology, Department of Internal Medicine and Palliative Care Center, Cantonal Hospital, Rorschacherstrasse 95, 9007 St. Gallen, Switzerland
    • ,
  • Florian Strasser

      Affiliations

    • Oncological Palliative Medicine, Section Oncology/Haematology, Department of Internal Medicine and Palliative Care Center, Cantonal Hospital, Rorschacherstrasse 95, 9007 St. Gallen, Switzerland
    • Corresponding Author InformationCorresponding author. Tel.: +41 71 494 1179; fax: +41 71 494 6325.

,Accepted 6 March 2009.

References 

  1. Ingham J, Portenoy R. Cachexia-anorexia in cancer patients. Oxford University Press; 1996;p. 158–71
  2. Radbruch L, Sabatowski R, Elsner F, Everts J, Mendoza T, Cleeland C. Validation of the German version of the brief fatigue inventory. J Pain Symptom Manage. 2003;25:449–458
  3. Johnston M, Pollard B, Hennessey P. Construct validation of the hospital anxiety and depression scale with clinical populations. J Psychosom Res. 2000;48:579–584
  4. Stroemgren AS, Goldschmidt D, Groenvold M, et al. Self-assessment in cancer patients referred to palliative care: a study of feasibility and symptom epidemiology. Cancer. 2002;94:512–520
  5. Nekolaichuk CL, Bruera E. Assessing hope at the end of life: validation of an experience of hope scale in advanced cancer patients. Palliat Support Care. 2004;2:243–253
  6. Rosenfeld B, Breitbart W, Galietta M, et al. The schedule of attitudes toward hastened death: measuring desire for death in terminally ill cancer patients. Cancer. 2000;88:2868–2875
  7. Groenvold M, Petersen MA, Aaronson NK, et al. The development of the EORTC QLQ-C15-PAL: a shortened questionnaire for cancer patients in palliative care. Eur J Cancer. 2006;42:55–64
  8. Echteld MA, Deliens L, Onwuteaka-Philipsen B, Klein M, van der Wal G. EORTC QLQ-C15-PAL: the new standard in the assessment of health-related quality of life in advanced cancer?. Palliat Med. 2006;20:1–2
  9. Lai JS, Cella D, Dineen K, et al. An item bank was created to improve the measurement of cancer-related fatigue. J Clin Epidemiol. 2005;58:190–197
  10. Lai JS, Dineen K, Reeve BB, et al. An item response theory-based pain item bank can enhance measurement precision. J Pain Symptom Manage. 2005;30:278–288
  11. Gardner W, Shear K, Kelleher KJ, et al. Computerized adaptive measurement of depression: a simulation study. BMC Psychiatry. 2004;4:13
  12. Jette AM, Haley SM, Tao W, et al. Prospective evaluation of the AM-PAC-CAT in outpatient rehabilitation settings. Phys Ther. 2007;87:385–398
  13. Cella D, Yount S, Rothrock N, et al. The Patient-Reported Outcomes Measurement Information System (PROMIS): progress of an NIH Roadmap cooperative group during its first two years. Med Care. 2007;45:S3–S11
  14. Reeve BB, Hays RD, Bjorner JB, et al. Psychometric evaluation and calibration of health-related quality of life item banks: plans for the Patient-Reported Outcomes Measurement Information System (PROMIS). Med Care. 2007;45:S22–S31
  15. Strasser F, Binswanger J, Cerny T, Kesselring A. Fighting a losing battle: eating-related distress of men with advanced cancer and their female partners. A mixed-methods study. Palliat Med. 2007;21:129–137
  16. Greenhalgh T, Peacock R. Effectiveness and efficiency of search methods in systematic reviews of complex evidence: audit of primary sources. Br Med J. 2005;331:1064–1065
  17. Ware JE, Gandek B, Sinclair SJ, Bjorner JB. Item response theory and computerized adaptive testing: implications for outcomes measurement in rehabilitation. Rehabil Psychology. 2005;50:71–78
  18. Haley SM, Andres PL, Coster WJ, Kosinski M, Ni P, Jette AM. Short-form activity measure for post-acute care. Arch Phys Med Rehabil. 2004;85:649–660
  19. Haley SM, Coster WJ, Andres PL, Kosinski M, Ni P. Score comparability of short forms and computerized adaptive testing: simulation study with the activity measure for post-acute care. Arch Phys Med Rehabil. 2004;85:661–666
  20. Schwartz C, Welch G, Santiago-Kelley P, Bode R, Sun X. Computerized adaptive testing of diabetes impact: a feasibility study of Hispanics and non-Hispanics in an active clinic population. Qual Life Res. 2006;15:1503–1518
  21. Siebens H, Andres PL, Pengsheng N, Coster WJ, Haley SM. Measuring physical function in patients with complex medical and postsurgical conditions: a computer adaptive approach. Am J Phys Med Rehabil. 2005;84:741–748
  22. Haley SM, Ni P, Fragala-Pinkham MA, Skrinar AM, Corzo D. A computer adaptive testing approach for assessing physical functioning in children and adolescents. Dev Med Child Neurol. 2005;47:113–120
  23. Hart DL, Cook KF, Mioduski JE, Teal CR, Crane PK. Simulated computerized adaptive test for patients with shoulder impairments was efficient and produced valid measures of function. J Clin Epidemiol. 2006;59:290–298
  24. Crane PK, Hart DL, Gibbons LE, Cook KF. A 37-item shoulder functional status item pool had negligible differential item functioning. J Clin Epidemiol. 2006;59:478–484
  25. Cook KF, Roddey TS, Gartsman GM, Olson SL. Development and psychometric evaluation of the Flexilevel Scale of Shoulder Function. Med Care. 2003;41:823–835
  26. Gardner W, Kelleher KJ, Pajer KA. Multidimensional adaptive testing for mental health problems in primary care. Med Care. 2002;40:812–823
  27. Ware JE, Kosinski M, Bjorner JB, et al. Applications of computerized adaptive testing (CAT) to the assessment of headache impact. Qual Life Res. 2003;12:935–952
  28. Bjorner JB, Kosinski M, Ware JE. Calibration of an item pool for assessing the burden of headaches: an application of item response theory to the headache impact test (HIT). Qual Life Res. 2003;12:913–933
  29. Andres PL, Black-Schaffer RM, Ni P, Haley SM. Computer adaptive testing: a strategy for monitoring stroke rehabilitation across settings. Top Stroke Rehabil. 2004;11:33–39
  30. Haley SM, Coster WJ, Andres PL, et al. Activity outcome measurement for postacute care. Med Care. 2004;42:I49–I61
  31. Petersen MA, Groenvold M, Aaronson N, Fayers P, Sprangers M, Bjorner JB. Multidimensional computerized adaptive testing of the EORTC QLQ-C30: basic developments and evaluations. Qual Life Res. 2006;15:315–329
  32. Haley SM, Ni P, Hambleton RK, Slavin MD, Jette AM. Computer adaptive testing improved accuracy and precision of scores over random item selection in a physical functioning item bank. J Clin Epidemiol. 2006;59:1174–1182
  33. Haley SM, Raczek AE, Coster WJ, Dumas HM, Fragala-Pinkham MA. Assessing mobility in children using a computer adaptive testing version of the pediatric evaluation of disability inventory. Arch Phys Med Rehabil. 2005;86:932–939
  34. Dijkers MP. A computer adaptive testing simulation applied to the FIM instrument motor component. Arch Phys Med Rehabil. 2003;84:384–393
  35. Cook KF, Roddey TS, O’Malley KJ, Gartsman GM. Development of a Flexilevel Scale for use with computer-adaptive testing for assessing shoulder function. J Shoulder Elbow Surg. 2005;14:90S–94S
  36. Fliege H, Becker J, Walter OB, Bjorner JB, Klapp BF, Rose M. Development of a computer-adaptive test for depression (D-CAT). Qual Life Res. 2005;14:2277–2291
  37. Hart DL, Mioduski JE, Stratford PW. Simulated computerized adaptive tests for measuring functional status were efficient with good discriminant validity in patients with hip, knee, or foot/ankle impairments. J Clin Epidemiol. 2005;58:629–638
  38. Hart DL, Mioduski JE, Werneke MW, Stratford PW. Simulated computerized adaptive test for patients with lumbar spine impairments was efficient and produced valid measures of function. J Clin Epidemiol. 2006;59:947–956
  39. Kosinski M, Bjorner JB, Ware JE, Sullivan E, Straus WL. An evaluation of a patient-reported outcomes found computerized adaptive testing was efficient in assessing osteoarthritis impact. J Clin Epidemiol. 2006;59:715–723
  40. Lai JS, Cella D, Chang CH, Bode RK, Heinemann AW. Item banking to improve, shorten and computerize self-reported fatigue: an illustration of steps to create a core item bank from the FACIT-Fatigue Scale. Qual Life Res. 2003;12:485–501
  41. Walter OB, Becker J, Bjorner JB, Fliege H, Klapp BF, Rose M. Development and evaluation of a computer adaptive test for ‘Anxiety’ (Anxiety-CAT). Qual Life Res. 2007;16(Suppl. 1):143–155
  42. Walter OB, Becker J, Fliege H, et al. Entwicklungsschritte für einen computeradaptiven Test zur Erfassung von Angst (A-CAT1). Diagnostica. 2005;51:88–100
  43. Rose M, Bjorner JB, Becker J, Fries JF, Ware JE. Evaluation of a preliminary physical function item bank supported the expected advantages of the Patient-Reported Outcomes Measurement Information System (PROMIS). J Clin Epidemiol. 2008;61:17–33
  44. Haley SM, Siebens H, Coster WJ, et al. Computerized adaptive testing for follow-up after discharge from inpatient rehabilitation: I. Activity outcomes. Arch Phys Med Rehabil. 2006;87:1033–1042
  45. Cook KF, Teal CR, Bjorner JB, et al. IRT health outcomes data analysis project: an overview and summary. Qual Life Res. 2007;16(Suppl. 1):121–132
  46. Cook KF, O’Malley KJ, Roddey TS. Dynamic assessment of health outcomes: time to let the CAT out of the bag?. Health Serv Res. 2005;40:1694–1711
  47. FACIT Development+Validation References. http://www.facit.org/validity/validation_articles.aspx. Ref Type: Electronic Citation; 2007.
  48. Smith AB, Rush R, Velikova G, et al. The initial development of an item bank to assess and screen for psychological distress in cancer patients. Psychooncology. 2007;16:724–732
  49. Likert R. A Technique for the measurement of attitudes. Arch Psychology. 1932;140:1–55
  50. Linacre J. Sample size and item calibration stability. Rasch Meas Trans. 1994;7:328
  51. Fayers PM. Applying item response theory and computer adaptive testing: the challenges for health outcomes assessment. Qual Life Res. 2007;16(Suppl. 1):187–194
  52. Holman R, Weisscher N, Glas CAW, Dijkgraaf MGW, Vermeulen M, de Haan RJ. The Academic Medical Center Linear Disability Score (ALDS) item bank: item response theory analysis in a mixed patient population. Health Qual Life Outcomes. 2005;3:83
  53. Reeve BB, Fayers P. Applying item response theory modelling for evaluating questionnaire item and scale properties. In:  Fayers Q,  Hays RD editor. Assessing quality of life in clinical trials: methods and practice. Oxford University Press; 2005;p. 55–73
  54. Holman R, Lindeboom R, Glas CAW, Vermeulen R. Constructing an item bank using item response theory: the AMC linear disability score project. Health Serv Outcomes Res Method. 2003;4:19–33
  55. van der Linden W, Hambleton RK. Handbook of modern item response theory. Springer; 1997;
  56. Hambleton RK, Swaminathan H, Rogers HJ. Fundamentals of item response theory. Sage; 1991;
  57. McCutcheon AL. Latent class analysis. Sage; 1987;
  58. Reise SP, Morizot J, Hays RD. The role of the bifactor model in resolving dimensionality issues in health outcomes measures. Qual Life Res. 2007;16(Suppl. 1):19–31
  59. Rasch G. Probabilistic models for some intelligence and attainment tests. University of Chicago Press; 1980;
  60. Birnbaum A. Some latent trait models and their use in inferring an examinee's ability. In:  Lord FM,  Novick MR editor. Statistical theories of mental test scores. Reading, MA: Addison-Wesely; 1968;p. 397–479
  61. Baker FB. The basics of item response theory. ERIC Clearinghouse on Assessment and Evaluation; 2001;
  62. Akaike H. Information theory and an extension of the maximum likelihood principle. In:  Petrov B,  Csaki F editor. 2nd International symposium on information theory. Akademiai Kiado. 1973;p. 267–381
  63. Schwarz G. Estimating the dimension of a model. Ann Stat. 1978;6:461–464
  64. Glas CAW. Tests of fit for polytomous Rasch models. In:  Fischer GH editors. Rasch models: foundations, recent developments, and applications. Springer; 1995;p. 325–352
  65. von Davier M. Bootstrapping goodness-of-fit statistics for sparse categorical data: results of a Monte Carlo study. Methods Psychol Res Online. 1997;29–48
  66. Glas CAW. Differential item functioning depending on general covariates. In:  Boomsma A,  van Duijn M,  Snijders T editor. Essays on item response theory. Springer; 2001;p. 131–148
  67. Crane PK, Gibbons LE, Narasimhalu K, Lai JS, Cella D. Rapid detection of differential item functioning in assessments of health-related quality of life: the functional assessment of cancer therapy. Qual Life Res. 2007;16:101–114
  68. Wainer H. Computerized adaptive testing: a primer, 2nd ed.; 2000 [Ref Type: Generic].
  69. Glas CAW. Item calibration and parameter drift. In:  van der Linden W,  Glas CAW editor. Computerized adaptive testing: theory and practice. Kluwer; 2003;p. 183–200
  70. Thissen D, Reeve BB, Bjorner JB, Chang CH. Methodological issues for building item banks and computerized adaptive scales. Qual Life Res. 2007;16(Suppl. 1):109–119
  71. van der Linden W, Glas CAW. Cross-validating item parameter estimation in adaptive testing. In:  Boomsma A,  van Duijn M,  Snijders T editor. Essays on item response theory. Springer; 2001;p. 205–219
  72. General overview and comparison of programs (http://www.assess.com/). http://www.assess.com/. Ref Type: Electronic Citation; 13-1-2007.
  73. Mislevy R, Stocking M. A consumer's guide to LOGIST and BILOG. Applied Psychological Measurement. 1989;13:57–75
  74. Mislevy RJ, Bock RD. BILOG: maximum likelihood item analysis and test scoring with logistic models for binary items [Computer program]. Chicago: Scientific Software International; 1990;Ref Type: Computer Program
  75. Thissen D. MULTILOG user's guide – version 6 [Computer software and manual], Chicago: Scientific Software International; 1991;Ref Type: Computer Program
  76. du Toit M. IRT from SSI. Lincoln, IL: Scientific Software International Inc.; 2003;
  77. Petersen MA, Groenvold M, Aaronson N, et al. Item response theory was used to shorten EORTC QLQ-C30 scales for use in palliative care. J Clin Epidemiol. 2006;59:36–44

PII: S1040-8428(09)00057-2

doi: 10.1016/j.critrevonc.2009.03.007

Critical Reviews in Oncology / Hematology
Volume 73, Issue 1 , Pages 47-67 , January 2010

Article Tools

* Image Usage & Resolution