A history of lung metastasectomy may be told in narratives but we need randomized controlled trials to know what treatments should be used in the future

“Large randomized controlled trials are needed to guide our future management of colorectal lung metastases.” This is the final sentence of the narrative review by Ka Lun Mak and Michael Kuan Yew Hsin (1). We fully agree with them and we use that as our starting point, encouraging them and our many colleagues around the world to rise to that challenge.

We are grateful to them for acknowledging the Pulmonary Metastasectomy in Colorectal Cancer (PulMiCC) study which we believe to be the largest prospective cohort of the practice, with 512 patients recruited. It provides baseline data to randomized controlled trial (RCT) standards (2). Nested within it is the only RCT that has ever addressed the question (3). While we appreciated PulMiCC being mentioned, we do not think it is adequately represented in their narrative. But we will return to that in due course. First we would like to parse their concluding sentence to tease out some points.

They specify that a trial must be “large”. The size of the trial required to produce trustworthy evidence is inversely proportional to the effect size. The smaller the effect size the larger is the trial required to detect the difference. Conversely if there is a large effect relatively few patients are required to show it. For the question of lung metastasectomy the US Society of Thoracic Surgeons (STS)—which might be expected to be one of the better-informed bodies of opinion on the matter—concluded that without operation, five-year survival can be assumed be zero among patients with lung metastases (4). An editorial in the European Journal of Cardio-Thoracic Surgery (EJCTS) claimed that survival with metastasectomy is as high as 60% (5). Neither statement was supported by a systematic review of numerical evidence. Both teams of authors knew that the question was being tested in the PulMiCC trial, and they mention the trial. But in their narratives, they chose to dismiss it before it had even been published.

If both estimates had been correct, it would indicate a really large effect size of 60%. That means that for two groups of 10 patients expected survival might be, 6/10 versus 0/10 which by Fisher’s exact test would be highly significant (P=0.018). A difference of anywhere close to that magnitude would have signalled in the PulMiCC RCT of 93 patients but it was nowhere to be seen (lower panel of the figure). By specifying that the trial must be “large” Mak and Hsin inadvertently accept that the benefit attributable to localise treatment of lung metastases from colorectal cancer must be small, and much less than is widely claimed.

Our research in planning the RCT included mathematical modelling (6). We were convinced then by the data that if there was any benefit it would be much smaller than that believed. The statistician Francesca Fiorentino, who did the systematic review and quantitative synthesis (7) prior to the PulMiCC trial, has explained the matter of sample size calculations in the journal Colorectal Disease (8). So now that PulMiCC has shown that any benefit cannot be as great as is believed, have practitioners accepted that “Large randomized controlled trials are needed to guide our future management of colorectal lung metastases”? It seems not. Despite failing to see a difference in a controlled trial, those who seek to expand treatment by less invasive means with image guided thermal ablation (IGTA) and stereotactic radiotherapy (SBRT/SABR) want to hold to their belief based on small trials flawed by imbalance (9,10).

The full PulMiCC study recruited well, with 512 patient giving fully informed consent. In the course of running PulMiCC the proportion randomised fell. It was difficult for the trialists to resist the pressure to operate engendered by powerful optimism bias and the misleading evidence available (4,5). Seeing this happening the Independent Data Monitoring Committee asked for an investigation and the three largest recruiting teams provided reasons for not randomising 155 patients. Of them 41 patients wanted to make their own decision and 19 (nearly half) chose not to be randomised. When the cancer team made the decision 77/78 (99%) were operated overriding the patients’ signed consent to be in an RCT and undermining an ethically approved and charitably funded trial (11). To gain trustworthy evidence a trial must be controlled and the control and treatment arms should be selected from a pool of similarly eligible patients by randomization as Mak and Hsin write—but those critical words are being resisted by contemporary practitioners.

Mak and Hsin go on to write “Treasure analyzed the data from the incomplete randomized controlled trial PulMiCC”. The word incomplete is part of their narrative. In fact, the PulMiCC team honoured the commitment to the funders, and the participating trialists, and of course the patients. We have analyzed and reported all the data. Of the 512 consented patients—a larger number than in any comparable study of which we are aware—93 were randomised and only 28 were excluded because they were found have benign nodules or other malignancies. This provided a cohort of 391 patients of whom 263 went on to have metastasectomy and 128 did not. The survival over five years of operated patients is shown in the upper panel of Figure 1. Their survival is comparable with the best of the follow-up studies in the Gonzalez landmark meta-analysis (12) and the EJCTS editorial (5). The results are therefore in line with “real world results” indicating face validity for the study as a whole.

Figure 1 Kaplan-Meier analysis of above 391 patients in the PulMiCC non-randomised cohort and below 93 patients in the randomised controlled trial. Interpretation is in the text. PulMiCC, Pulmonary Metastasectomy in Colorectal Cancer.

The teams recruiting patients knew the prognostic factors. Patients selected for metastasectomy had predominately solitary metastases, fewer had liver involvement or elevation of their carcinoembryonic antigen assay, they had better lung function, better Eastern Cooperative Oncology Group (ECOG) performance scores and they were younger. The teams had quite reasonably selected for surgery patients with the best prognosis but mistakenly attributed better survival to the operation. In the nested RCT the two arms were well balanced for all baseline factors. The patients turned down for surgery did not have zero survival as assumed in the Expert Consensus Document on Pulmonary Metastasectomy of the STS. The result of the RCT can be seen in the lower graph (Figure 1). The survival curves weave in and out of each other—median survival was in fact longer in the control group at 3.8 years compared with 3.5 years after metastasectomy—but there was no significant difference at any time point. The confidence intervals broadly overlap.

The cohort study also refutes zero mortality showing it to be a myth; it was always was implausible, that being a feature of myths, which are part of narrative story telling (13). The RCT is adequately sized to show that any difference attributable to metastasectomy is only a fraction of what is believed—and what patients are misleadingly told. No larger trial is required to prove that any benefit from lung metastasectomy is smaller than generally believed but it would take a very much larger trial to define just how small that difference might be—and to show whether or not intervention is worthwhile.

PulMiCC’s large prospective cohort provided another important finding. In the full study there were four deaths on the day of operation, three were among the elective metastasectomy patients with an additional patient randomly assigned to control who asked to cross over to operation and who died in the operating room. A further five deaths occurred in the first six months after metastasectomy. In contrast, among the 128 patients not operated on there was only one death during the first six months. In general the process of recruiting and assessing patients excludes those likely to die soon creating a relatively death-free period due to guarantee time bias (14). These nine peri- and postoperative deaths were either caused directly by metastasectomy or related to other detrimental effects of thoracic surgery.

A subset of 220 patients PulMiCC provides a record of the burden of further local treatments and chemotherapy which was great (15). This is contrary to the belief that metastasectomy spares patients chemotherapy. In the RCT there was no health utility gain attributable to metastasectomy (16). And there was a detrimental effect on health related quality of life and lung function in the patients assigned to metastasectomy in the six-month after operation (11).

The STS stated that there are over 1,000 papers on lung metastasectomy (4). In the “narrative review” only 110 papers are cited based on the search terms listed (1). We are not told how these were selected, whether they were assessed for quality and there is no explicit data synthesis. The “narrative” has confirmed important information which is that “Lung metastases are usually indolent and asymptomatic” and that “The goal of pulmonary metastasectomy is to improve patient survival.” Prolonging survival has always been the stated or implicit goal since the earliest reports such as that from Memorial Sloan-Kettering (17). The authors go on to write “It is paramount to balance the risks and benefits of pulmonary metastasectomy, and judicious patient selection can improve the outcome of pulmonary metastasectomy.” But “selection” allows clinicians to choose those whom by their nature are likely to do well and then misleadingly attribute that survival to the metastasectomy rather than to the selection.

Separating the effect of treatment and selection for treatment is not easy. Considering the effects of climatic treatment in tuberculosis in 1899 D. W. Samways wrote “Neither Switzerland, the Riviera, Egypt, the sea, or an English verandah, can justly claim patent right for the treatment of phthisis. Any of them may be statistically shown to be the best if the cases they treat are selected with sufficient care, and especially if their failures are quietly sent elsewhere.” (18). A century later the Evidence Based Medicine movement sought to move to scientifically collected data, rather than anecdotal observations, to support adoption of new treatments. Has “narrative” now emerged to push back against hard won randomized trial evidence and statistical analysis? Recently this statement has been made “If patients do not receive timely and effective treatment, they may die as a result of respiratory failure.” (19), cited in support of ablative therapy for lung metastases. It is unsubstantiated and we know that respiratory failure is a rare consequence of lung metastases. But it is a compelling image and illustrates the potentially misleading nature of narrative.

We offer two examples where RCTs have reversed a clinical practice in cancer surgery. Throughout the time Samways was practising and writing about medicine (20) the Halsted mastectomy was promoted with ever more radical extensions to eradicate lymphatic metastases (21). After 90 years two randomised trials showed that it was no more effective than limited surgery to control the primary cancer (22,23). These trials spared millions of women the mutilation and morbidity of lymphadenectomy. A hundred years after Halsted enthusiastic surgeons promoted extrapleural pneumonectomy (EPP) for mesothelioma. In the Mesothelioma and Radical Surgery (MARS) trial those assigned to operation had similar survival to the reported series while those assigned to not have EPP lived longer and better (Figure 2) (24). Patients should be spared surgery unless it is of proven benefit.

Figure 2 Kaplan-Meier analysis of the MARS randomised controlled trial. The vertical markers are the median survivals in follow-up reports from MD Anderson, a combined New York series, Brigham and Women’s and MSK-CC. MARS, Mesothelioma and Radical Surgery; EPP, extrapleural pneumonectomy.

And so to return to that opening quotation, can we expect controlled trials to be done to guide future treatments? An interesting sentence in the methods of this narrative review is “Non-English written articles and review articles are excluded” (1). While understanding the appeal of this limitation for authors, reviewers and editors it does raise a question. Writing in Lancet Global Health, Thirusha Naidu makes a powerful plea for entitlement of those like herself working in low and middle-income countries (LMICs) to have their own voices heard (25). Rather than adopting high cost practices, promulgated by surgeons in the US and Europe (4,5), would it not be appropriate for health care workers in LMICS to put such interventions to the fair test of randomized trials in their own clinical context? Her plea is even more compelling when some treatments are based, as we have shown, on exaggerated claims and, in this case, absent evidence. PulMiCC was run in in China but in only one centre (2). We would have liked it to be more. The COVID-19 pandemic has shown that, when there is a will, trials can be designed and implemented at speed and that patients will want to enter them. There are many narrative accounts from the pandemic, some heroic and some tragic, but it is good science, hard data and randomised trials to test vaccines and treatments that are making a difference.

Acknowledgments

Funding: This work was supported by the Cancer Research UK (No. C7678/A1139).

Footnote

Provenance and Peer Review: This article was commissioned by the editorial office, AME Surgical Journal. The article did not undergo external peer review.

Conflicts of Interest: All authors have completed the ICMJE uniform disclosure form (available at https://asj.amegroups.com/article/view/10.21037/asj-21-91/coif). The authors have no conflicts of interest to declare.

Ethical Statement: The authors are accountable for all aspects of the work in ensuring that questions related to the accuracy or integrity of any part of the work are appropriately investigated and resolved.

Open Access Statement: This is an Open Access article distributed in accordance with the Creative Commons Attribution-NonCommercial-NoDerivs 4.0 International License (CC BY-NC-ND 4.0), which permits the non-commercial replication and distribution of the article with the strict proviso that no changes or edits are made and the original work is properly cited (including links to both the formal publication through the relevant DOI and the license). See: https://creativecommons.org/licenses/by-nc-nd/4.0/.

References

1. Mak KL, Hsin MKY. Lung metastases from colorectal cancer: a multidisciplinary approach and role of local treatment: a narrative review. AME Surg J 2022;2:23. [Crossref]
2. Treasure T, Farewell V, Macbeth F, et al. The Pulmonary Metastasectomy in Colorectal Cancer cohort study: Analysis of case selection, risk factors and survival in a prospective observational study of 512 patients. Colorectal Dis 2021;23:1793-803. [Crossref] [PubMed]
3. Milosevic M, Edwards J, Dunning J, et al. Five-year survival of patients in control groups of randomized controlled trials is much higher than that assumed in observational study reports. Int J Colorectal Dis 2020;35:941-2. [Crossref] [PubMed]
4. Handy JR, Bremner RM, Crocenzi TS, et al. Expert Consensus Document on Pulmonary Metastasectomy. Ann Thorac Surg 2019;107:631-49. [Crossref] [PubMed]
5. Schirren J, Schirren M, Lampl L, et al. Surgery for pulmonary metastases: quo vadis? Eur J Cardiothorac Surg 2017;51:408-10. [Crossref] [PubMed]
6. Utley M, Treasure T. Interpreting data from surgical follow-up studies: the role of modeling. J Thorac Oncol 2010;5:S200-2. [Crossref] [PubMed]
7. Fiorentino F, Hunt I, Teoh K, et al. Pulmonary metastasectomy in colorectal cancer: a systematic review and quantitative synthesis. J R Soc Med 2010;103:60-6. [Crossref] [PubMed]
8. Fiorentino F, Treasure T. Sample size calculations for randomized controlled trials and for prediction models. Colorectal Dis 2021;23:316-9. [Crossref] [PubMed]
9. Macbeth FR, Abratt RP. Stereotactic Body Radiotherapy for Oligometastatic Prostate Cancer: Where is the Evidence? Clin Oncol (R Coll Radiol) 2021;33:e94. [Crossref] [PubMed]
10. Macbeth F, Hughes-Davies L. Should We be Offering our Patients with Oligometastases Stereotactic Ablative Body Radiotherapy – No. Clin Oncol (R Coll Radiol) 2021;33:747-8. [Crossref] [PubMed]
11. Treasure T, Farewell V, Macbeth F, et al. Pulmonary Metastasectomy versus Continued Active Monitoring in Colorectal Cancer (PulMiCC): a multicentre randomised clinical trial. Trials 2019;20:718. [Crossref] [PubMed]
12. Gonzalez M, Poncet A, Combescure C, et al. Risk factors for survival after lung metastasectomy in colorectal cancer patients: a systematic review and meta-analysis. Ann Surg Oncol 2013;20:572-9. [Crossref] [PubMed]
13. Macbeth F, Fallowfield L. The myth of pulmonary metastasectomy. Br J Cancer 2020;123:499-500. [Crossref] [PubMed]
14. Giobbie-Hurder A, Gelber RD, Regan MM. Challenges of guarantee-time bias. J Clin Oncol 2013;31:2963-9. [Crossref] [PubMed]
15. Treasure T, Farewell V, Macbeth F, et al. The Pulmonary Metastasectomy in Colorectal Cancer (PulMiCC) burden of care study: Analysis of local treatments for lung metastases and systemic chemotherapy in 220 patients in the PulMiCC cohort. Colorectal Dis 2021; Epub ahead of print. [Crossref] [PubMed]
16. Brew-Graves C, Farewell V, Monson K, et al. Pulmonary metastasectomy in colorectal cancer: health utility scores by EQ-5D-3L in a randomized controlled trial show no benefit from lung metastasectomy. Colorectal Dis 2021;23:200-5. [Crossref] [PubMed]
17. McCormack PM, Burt ME, Bains MS, et al. Lung resection for colorectal metastases. 10-year results. Arch Surg 1992;127:1403-6. [Crossref] [PubMed]
18. Samways DW. Ocean voyages in pulmonary phthisis. Br Med J 1899;2:1817. [Crossref]
19. Qi H, Fan W. Value of ablation therapy in the treatment of lung metastases. Thorac Cancer 2018;9:199-207. [Crossref] [PubMed]
20. Treasure T. Dr Samways writes to the editor: The Life and Times of an Exceptional Physician (1857-1931). Newcastle: Cambridge Scholars Publishing; 2021.
21. Mukherjee S. A Radical Idea in "The Emperor of All Maladies". The Emperor of the Maladies. New York: Scribner, 2010:60-72.
22. Veronesi U, Saccozzi R, Del Vecchio M, et al. Comparing radical mastectomy with quadrantectomy, axillary dissection, and radiotherapy in patients with small cancers of the breast. N Engl J Med 1981;305:6-11. [Crossref] [PubMed]
23. Fisher B, Bauer M, Margolese R, et al. Five-year results of a randomized clinical trial comparing total mastectomy and segmental mastectomy with or without radiation in the treatment of breast cancer. N Engl J Med 1985;312:665-73. [Crossref] [PubMed]
24. Treasure T, Lang-Lazdunski L, Waller D, et al. Extra-pleural pneumonectomy versus no extra-pleural pneumonectomy for patients with malignant pleural mesothelioma: clinical outcomes of the Mesothelioma and Radical Surgery (MARS) randomised feasibility study. Lancet Oncol 2011;12:763-72. [Crossref] [PubMed]
25. Naidu T. Says who? Northern ventriloquism, or epistemic disobedience in global health scholarship. Lancet Glob Health 2021;9:e1332-5. [Crossref] [PubMed]