Importance of the reference volume in assessing outlining performance for the purpose of training and revalidation


Session type:

Sarah Gwynne1,2, Emiliano Spezi2, Chris Hurt3, Stephen Falk4, Simon Gollins5, George Joseph2, John Staffurth2, Tom Crosby2
1Singleton Hospital, Swansea, UK, 2Velindre Cancer Centre, Cardiff, UK, 3Wales Cancer Trials Unit, Cardiff, UK, 4Bristol Haematology and Oncology Centre, Bristol, UK, 5North Wales Cancer Treatment Centre, Rhyl, UK


Assessment of outlining performance will form an increasingly important part of future training and revalidation for clinical oncologists, measured by comparison to a pre-defined reference volume. Definition of the latter can be informed by experience gained within the SCOPE 1 trial.


The pre-trial RTTQA programme of SCOPE 1 (UK phase 2/3 randomised controlled trial of chemoradiotherapy for oesophageal cancer) included an outlining assessment of a mid oesophageal tumour. 50 investigator GTV (i-GTV) outlines were compared both to the original pre-defined gold standard (gs-GTV) (defined by the chief investigator and an upper GI radiologist) and a consensus volume (c-GTV) (included the gs-GTV plus i-GTVs of three clinical oncology members of the Trial Management Group using the STAPLE algorithm).

CERR was used to calculate the conformity of each i-GTV against the gs-GTV and c-GTV using the Jaccard Conformity Index (JCI). JCI values of <0.5 and ≥0.7 were taken to represent poor and excellent conformity respectively. All 50 i-GTVs were considered acceptable by the RTTQA team.


Length and volume of gs-GTV and c-GTV were 7.8cm and 8.1cm and 39.15cc and 46.77cc respectively. Median JCI and the percentage of i-GTVs achieving a JCI of ≥0.7 against the gs-GTV and c-GTV was 0.67 and 0.74 (Wilcoxon signed rank test p=<0.0001) and 28% and 81% respectively. Only one i-GTV achieved a JCI <0.5 against the gs-GTV but none against the c-GTV.


In the SCOPE 1 pre-trial test case the method of creation of the reference volume increased the number of i-GTVs meeting the criteria for excellent conformity by 53%. In the context of training and revalidation, correctly identifying individuals with excellent and poor conformity is critical. This will require careful definition of a consensus volume to act as a reference volume and development of robust criteria for acceptable JCI values.