An interobserver reproducibility study on invasiveness of bladder cancer using virtual microscopy and heatmaps

Eva Compérat, Lars Egevad, Antonio Lopez-Beltran, Philippe Camparo, Ferran Algaba, Mahul Amin, Jonathan I. Epstein, Hans Hamberg, Christina Hulsbergen-van de Kaa, Glen Kristiansen, Rodolfo Montironi, Chin Chen Pan, Fabrice Heloir, Kilian Treurniet, Jenna Sykes, Theo H. Van der Kwast

Research output: Contribution to journalArticle

24 Citations (Scopus)

Abstract

Aims: The distinction between non-invasive (pTa) and invasive (pT1) non-muscle invasive bladder cancer (NMIBC) is subject to considerable interobserver variation. We aimed to generate a teaching set of images based on the diagnostic opinions of a panel of expert genitourinary pathologists. Methods and results: Twenty-five transurethral resection specimens initially reported as pT1 NMIBC from two university hospitals were selected on the basis of potential uncertainty of stromal invasion. Digitized slides were reviewed independently by a panel of eight genitourinary pathologists, who annotated any invasive area if present. Annotations were reviewed by the lead panel, and heatmaps of annotated areas were constructed. Reasons for discrepancies were analysed, and kappa scores were calculated to determine agreement among the eight panellists. Full agreement by the eight panellists was obtained in 11 of 25 cases (44%), with a multi-rater (Fleiss) kappa of 0.47 (P < 0.0001). After joint review of the seven discordant (agreement <75% of panellists) cases, consensus was obtained for six cases, and a teaching set of images was generated. Conclusions: Interobserver agreement among the panellists in the selected cases was moderate, but consensus could be reached in almost all cases. Heatmaps proved to be instrumental in generating a teaching set of images for standardization of histological criteria for NMIBC invasion.

Original languageEnglish (US)
Pages (from-to)756-766
Number of pages11
JournalHistopathology
Volume63
Issue number6
DOIs
StatePublished - Dec 1 2013
Externally publishedYes

Fingerprint

Urinary Bladder Neoplasms
Microscopy
Teaching
Observer Variation
Uncertainty
Pathologists

All Science Journal Classification (ASJC) codes

  • Pathology and Forensic Medicine
  • Histology

Cite this

Compérat, E., Egevad, L., Lopez-Beltran, A., Camparo, P., Algaba, F., Amin, M., ... Van der Kwast, T. H. (2013). An interobserver reproducibility study on invasiveness of bladder cancer using virtual microscopy and heatmaps. Histopathology, 63(6), 756-766. https://doi.org/10.1111/his.12214

An interobserver reproducibility study on invasiveness of bladder cancer using virtual microscopy and heatmaps. / Compérat, Eva; Egevad, Lars; Lopez-Beltran, Antonio; Camparo, Philippe; Algaba, Ferran; Amin, Mahul; Epstein, Jonathan I.; Hamberg, Hans; Hulsbergen-van de Kaa, Christina; Kristiansen, Glen; Montironi, Rodolfo; Pan, Chin Chen; Heloir, Fabrice; Treurniet, Kilian; Sykes, Jenna; Van der Kwast, Theo H.

In: Histopathology, Vol. 63, No. 6, 01.12.2013, p. 756-766.

Research output: Contribution to journalArticle

Compérat, E, Egevad, L, Lopez-Beltran, A, Camparo, P, Algaba, F, Amin, M, Epstein, JI, Hamberg, H, Hulsbergen-van de Kaa, C, Kristiansen, G, Montironi, R, Pan, CC, Heloir, F, Treurniet, K, Sykes, J & Van der Kwast, TH 2013, 'An interobserver reproducibility study on invasiveness of bladder cancer using virtual microscopy and heatmaps', Histopathology, vol. 63, no. 6, pp. 756-766. https://doi.org/10.1111/his.12214
Compérat, Eva ; Egevad, Lars ; Lopez-Beltran, Antonio ; Camparo, Philippe ; Algaba, Ferran ; Amin, Mahul ; Epstein, Jonathan I. ; Hamberg, Hans ; Hulsbergen-van de Kaa, Christina ; Kristiansen, Glen ; Montironi, Rodolfo ; Pan, Chin Chen ; Heloir, Fabrice ; Treurniet, Kilian ; Sykes, Jenna ; Van der Kwast, Theo H. / An interobserver reproducibility study on invasiveness of bladder cancer using virtual microscopy and heatmaps. In: Histopathology. 2013 ; Vol. 63, No. 6. pp. 756-766.
@article{cd7a8b2afb2943fe875e2b50ad7efac3,
title = "An interobserver reproducibility study on invasiveness of bladder cancer using virtual microscopy and heatmaps",
abstract = "Aims: The distinction between non-invasive (pTa) and invasive (pT1) non-muscle invasive bladder cancer (NMIBC) is subject to considerable interobserver variation. We aimed to generate a teaching set of images based on the diagnostic opinions of a panel of expert genitourinary pathologists. Methods and results: Twenty-five transurethral resection specimens initially reported as pT1 NMIBC from two university hospitals were selected on the basis of potential uncertainty of stromal invasion. Digitized slides were reviewed independently by a panel of eight genitourinary pathologists, who annotated any invasive area if present. Annotations were reviewed by the lead panel, and heatmaps of annotated areas were constructed. Reasons for discrepancies were analysed, and kappa scores were calculated to determine agreement among the eight panellists. Full agreement by the eight panellists was obtained in 11 of 25 cases (44{\%}), with a multi-rater (Fleiss) kappa of 0.47 (P < 0.0001). After joint review of the seven discordant (agreement <75{\%} of panellists) cases, consensus was obtained for six cases, and a teaching set of images was generated. Conclusions: Interobserver agreement among the panellists in the selected cases was moderate, but consensus could be reached in almost all cases. Heatmaps proved to be instrumental in generating a teaching set of images for standardization of histological criteria for NMIBC invasion.",
author = "Eva Comp{\'e}rat and Lars Egevad and Antonio Lopez-Beltran and Philippe Camparo and Ferran Algaba and Mahul Amin and Epstein, {Jonathan I.} and Hans Hamberg and {Hulsbergen-van de Kaa}, Christina and Glen Kristiansen and Rodolfo Montironi and Pan, {Chin Chen} and Fabrice Heloir and Kilian Treurniet and Jenna Sykes and {Van der Kwast}, {Theo H.}",
year = "2013",
month = "12",
day = "1",
doi = "10.1111/his.12214",
language = "English (US)",
volume = "63",
pages = "756--766",
journal = "Histopathology",
issn = "0309-0167",
publisher = "Wiley-Blackwell",
number = "6",

}

TY - JOUR

T1 - An interobserver reproducibility study on invasiveness of bladder cancer using virtual microscopy and heatmaps

AU - Compérat, Eva

AU - Egevad, Lars

AU - Lopez-Beltran, Antonio

AU - Camparo, Philippe

AU - Algaba, Ferran

AU - Amin, Mahul

AU - Epstein, Jonathan I.

AU - Hamberg, Hans

AU - Hulsbergen-van de Kaa, Christina

AU - Kristiansen, Glen

AU - Montironi, Rodolfo

AU - Pan, Chin Chen

AU - Heloir, Fabrice

AU - Treurniet, Kilian

AU - Sykes, Jenna

AU - Van der Kwast, Theo H.

PY - 2013/12/1

Y1 - 2013/12/1

N2 - Aims: The distinction between non-invasive (pTa) and invasive (pT1) non-muscle invasive bladder cancer (NMIBC) is subject to considerable interobserver variation. We aimed to generate a teaching set of images based on the diagnostic opinions of a panel of expert genitourinary pathologists. Methods and results: Twenty-five transurethral resection specimens initially reported as pT1 NMIBC from two university hospitals were selected on the basis of potential uncertainty of stromal invasion. Digitized slides were reviewed independently by a panel of eight genitourinary pathologists, who annotated any invasive area if present. Annotations were reviewed by the lead panel, and heatmaps of annotated areas were constructed. Reasons for discrepancies were analysed, and kappa scores were calculated to determine agreement among the eight panellists. Full agreement by the eight panellists was obtained in 11 of 25 cases (44%), with a multi-rater (Fleiss) kappa of 0.47 (P < 0.0001). After joint review of the seven discordant (agreement <75% of panellists) cases, consensus was obtained for six cases, and a teaching set of images was generated. Conclusions: Interobserver agreement among the panellists in the selected cases was moderate, but consensus could be reached in almost all cases. Heatmaps proved to be instrumental in generating a teaching set of images for standardization of histological criteria for NMIBC invasion.

AB - Aims: The distinction between non-invasive (pTa) and invasive (pT1) non-muscle invasive bladder cancer (NMIBC) is subject to considerable interobserver variation. We aimed to generate a teaching set of images based on the diagnostic opinions of a panel of expert genitourinary pathologists. Methods and results: Twenty-five transurethral resection specimens initially reported as pT1 NMIBC from two university hospitals were selected on the basis of potential uncertainty of stromal invasion. Digitized slides were reviewed independently by a panel of eight genitourinary pathologists, who annotated any invasive area if present. Annotations were reviewed by the lead panel, and heatmaps of annotated areas were constructed. Reasons for discrepancies were analysed, and kappa scores were calculated to determine agreement among the eight panellists. Full agreement by the eight panellists was obtained in 11 of 25 cases (44%), with a multi-rater (Fleiss) kappa of 0.47 (P < 0.0001). After joint review of the seven discordant (agreement <75% of panellists) cases, consensus was obtained for six cases, and a teaching set of images was generated. Conclusions: Interobserver agreement among the panellists in the selected cases was moderate, but consensus could be reached in almost all cases. Heatmaps proved to be instrumental in generating a teaching set of images for standardization of histological criteria for NMIBC invasion.

UR - http://www.scopus.com/inward/record.url?scp=84887823804&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84887823804&partnerID=8YFLogxK

U2 - 10.1111/his.12214

DO - 10.1111/his.12214

M3 - Article

VL - 63

SP - 756

EP - 766

JO - Histopathology

JF - Histopathology

SN - 0309-0167

IS - 6

ER -