Observer variability in mesothelioma tumor thickness measurements: Defining minimally measurable lesions

S.G. Armato, Anna Nowak, Roslyn Francis, M. Kocherginsky, Michael Byrne

Research output: Contribution to journalArticle

15 Citations (Scopus)

Abstract

Introduction: Single time-point unidimensional tumor thickness measurements define measurable disease for clinical trial inclusion and also constitute a field in the International Association for the Study of Lung Cancer prospective mesothelioma staging database. The modified Response Evaluation Criteria in Solid Tumors (RECIST) guidelines for mesothelioma did not alter the 10-mm minimum tumor measurement recommendation. However, as computed tomography technology has advanced, we sought to examine whether interobserver agreement was acceptable at smaller tumor thickness in mesothelioma. Methods: The primary observer selected 170 discrete measurement sites from 105 thoracic computed tomography scans from 50 consenting patients with pleural mesothelioma. Sites represented a range of tumor thickness, lesion morphology, and location. The outer (parietal) tumor margin was marked at each site and presented to five additional observers, who then selected the visceral margin of the tumor to create a line segment that captured tumor thickness. Relative differences among the observer measurements were estimated using a random-effects analysis of variance model to identify the smallest tumor thickness at which linear measurements could be made reliably. Results: Systematic bias was observed, with some observers consistently measuring larger or smaller thicknesses than the thickness measured by others. The mean range across all 170 sites was 15.1% of the mean per-site measurement (SD = 9.1%; median range, 13.1%). Measurements acquired at sites with mean tumor thickness less than 7.5 mm demonstrated interobserver variability with a 75th percentile included 20% of the tumor thickness. The 95% confidence interval for relative interobserver measurement differences obtained for mean measurement lengths in the range 5 to 7.5 mm does not exceed the RECIST thresholds. Conclusions: This study has implications for the definition of minimally measurable tumor adopted by clinical trial and staging protocols. Although the statistical findings suggest that a reduction in "minimally measurable disease" from 10 mm to 5 or 7.5 mm might be warranted, clinical factors may ultimately dictate the most appropriate definition. Copyright © 2014 by the International Association for the Study of Lung.
Original languageEnglish
Pages (from-to)1187-1194
JournalJournal of Thoracic Oncology
Volume9
Issue number8
DOIs
Publication statusPublished - 2014

Fingerprint

Mesothelioma
Neoplasms
Tomography
Clinical Trials
Observer Variation
Clinical Protocols
Analysis of Variance
Thorax
Databases
Guidelines
Confidence Intervals
Technology
Lung

Cite this

@article{21a35ea113a241af884703d83ae13697,
title = "Observer variability in mesothelioma tumor thickness measurements: Defining minimally measurable lesions",
abstract = "Introduction: Single time-point unidimensional tumor thickness measurements define measurable disease for clinical trial inclusion and also constitute a field in the International Association for the Study of Lung Cancer prospective mesothelioma staging database. The modified Response Evaluation Criteria in Solid Tumors (RECIST) guidelines for mesothelioma did not alter the 10-mm minimum tumor measurement recommendation. However, as computed tomography technology has advanced, we sought to examine whether interobserver agreement was acceptable at smaller tumor thickness in mesothelioma. Methods: The primary observer selected 170 discrete measurement sites from 105 thoracic computed tomography scans from 50 consenting patients with pleural mesothelioma. Sites represented a range of tumor thickness, lesion morphology, and location. The outer (parietal) tumor margin was marked at each site and presented to five additional observers, who then selected the visceral margin of the tumor to create a line segment that captured tumor thickness. Relative differences among the observer measurements were estimated using a random-effects analysis of variance model to identify the smallest tumor thickness at which linear measurements could be made reliably. Results: Systematic bias was observed, with some observers consistently measuring larger or smaller thicknesses than the thickness measured by others. The mean range across all 170 sites was 15.1{\%} of the mean per-site measurement (SD = 9.1{\%}; median range, 13.1{\%}). Measurements acquired at sites with mean tumor thickness less than 7.5 mm demonstrated interobserver variability with a 75th percentile included 20{\%} of the tumor thickness. The 95{\%} confidence interval for relative interobserver measurement differences obtained for mean measurement lengths in the range 5 to 7.5 mm does not exceed the RECIST thresholds. Conclusions: This study has implications for the definition of minimally measurable tumor adopted by clinical trial and staging protocols. Although the statistical findings suggest that a reduction in {"}minimally measurable disease{"} from 10 mm to 5 or 7.5 mm might be warranted, clinical factors may ultimately dictate the most appropriate definition. Copyright {\circledC} 2014 by the International Association for the Study of Lung.",
author = "S.G. Armato and Anna Nowak and Roslyn Francis and M. Kocherginsky and Michael Byrne",
year = "2014",
doi = "10.1097/JTO.0000000000000211",
language = "English",
volume = "9",
pages = "1187--1194",
journal = "Journal of Thoracic Oncology",
issn = "1556-0864",
publisher = "Academic Press",
number = "8",

}

Observer variability in mesothelioma tumor thickness measurements: Defining minimally measurable lesions. / Armato, S.G.; Nowak, Anna; Francis, Roslyn; Kocherginsky, M.; Byrne, Michael.

In: Journal of Thoracic Oncology, Vol. 9, No. 8, 2014, p. 1187-1194.

Research output: Contribution to journalArticle

TY - JOUR

T1 - Observer variability in mesothelioma tumor thickness measurements: Defining minimally measurable lesions

AU - Armato, S.G.

AU - Nowak, Anna

AU - Francis, Roslyn

AU - Kocherginsky, M.

AU - Byrne, Michael

PY - 2014

Y1 - 2014

N2 - Introduction: Single time-point unidimensional tumor thickness measurements define measurable disease for clinical trial inclusion and also constitute a field in the International Association for the Study of Lung Cancer prospective mesothelioma staging database. The modified Response Evaluation Criteria in Solid Tumors (RECIST) guidelines for mesothelioma did not alter the 10-mm minimum tumor measurement recommendation. However, as computed tomography technology has advanced, we sought to examine whether interobserver agreement was acceptable at smaller tumor thickness in mesothelioma. Methods: The primary observer selected 170 discrete measurement sites from 105 thoracic computed tomography scans from 50 consenting patients with pleural mesothelioma. Sites represented a range of tumor thickness, lesion morphology, and location. The outer (parietal) tumor margin was marked at each site and presented to five additional observers, who then selected the visceral margin of the tumor to create a line segment that captured tumor thickness. Relative differences among the observer measurements were estimated using a random-effects analysis of variance model to identify the smallest tumor thickness at which linear measurements could be made reliably. Results: Systematic bias was observed, with some observers consistently measuring larger or smaller thicknesses than the thickness measured by others. The mean range across all 170 sites was 15.1% of the mean per-site measurement (SD = 9.1%; median range, 13.1%). Measurements acquired at sites with mean tumor thickness less than 7.5 mm demonstrated interobserver variability with a 75th percentile included 20% of the tumor thickness. The 95% confidence interval for relative interobserver measurement differences obtained for mean measurement lengths in the range 5 to 7.5 mm does not exceed the RECIST thresholds. Conclusions: This study has implications for the definition of minimally measurable tumor adopted by clinical trial and staging protocols. Although the statistical findings suggest that a reduction in "minimally measurable disease" from 10 mm to 5 or 7.5 mm might be warranted, clinical factors may ultimately dictate the most appropriate definition. Copyright © 2014 by the International Association for the Study of Lung.

AB - Introduction: Single time-point unidimensional tumor thickness measurements define measurable disease for clinical trial inclusion and also constitute a field in the International Association for the Study of Lung Cancer prospective mesothelioma staging database. The modified Response Evaluation Criteria in Solid Tumors (RECIST) guidelines for mesothelioma did not alter the 10-mm minimum tumor measurement recommendation. However, as computed tomography technology has advanced, we sought to examine whether interobserver agreement was acceptable at smaller tumor thickness in mesothelioma. Methods: The primary observer selected 170 discrete measurement sites from 105 thoracic computed tomography scans from 50 consenting patients with pleural mesothelioma. Sites represented a range of tumor thickness, lesion morphology, and location. The outer (parietal) tumor margin was marked at each site and presented to five additional observers, who then selected the visceral margin of the tumor to create a line segment that captured tumor thickness. Relative differences among the observer measurements were estimated using a random-effects analysis of variance model to identify the smallest tumor thickness at which linear measurements could be made reliably. Results: Systematic bias was observed, with some observers consistently measuring larger or smaller thicknesses than the thickness measured by others. The mean range across all 170 sites was 15.1% of the mean per-site measurement (SD = 9.1%; median range, 13.1%). Measurements acquired at sites with mean tumor thickness less than 7.5 mm demonstrated interobserver variability with a 75th percentile included 20% of the tumor thickness. The 95% confidence interval for relative interobserver measurement differences obtained for mean measurement lengths in the range 5 to 7.5 mm does not exceed the RECIST thresholds. Conclusions: This study has implications for the definition of minimally measurable tumor adopted by clinical trial and staging protocols. Although the statistical findings suggest that a reduction in "minimally measurable disease" from 10 mm to 5 or 7.5 mm might be warranted, clinical factors may ultimately dictate the most appropriate definition. Copyright © 2014 by the International Association for the Study of Lung.

U2 - 10.1097/JTO.0000000000000211

DO - 10.1097/JTO.0000000000000211

M3 - Article

VL - 9

SP - 1187

EP - 1194

JO - Journal of Thoracic Oncology

JF - Journal of Thoracic Oncology

SN - 1556-0864

IS - 8

ER -