Letter To The Editor of The Journal of Trauma® Injury, Infection, and Critical Care

BASIL A. PRUITT, JR., MD

Editor

7330 San Pedro Ave., Suite 654

San Antonio, TX 78216

Telephone: 210-342-7903

Fax: 210-342-2966

E-mail: jtrauma@uthscsa.edu

S. Frankema

Trauma Center ZWN

Erasmus MC

PO BOX: 2040

3000 CA ROTTERDAM

The Netherlands

Tel: +31-10-46335034

Fax: +31-10-46335039

E-mail: s.frankema@erasmusmc.nl

Rotterdam, November - 18 - 2003

Dear editor,

With great interest have we read the original article by Dr. Kilgo et al, as published in the October issue of your journal. Accompanied by Dr Osler and Dr Turner they have provided us with a detailed analysis of the performance of current injury scales and their authority is highly regarded by us.

This article is of great importance, as using the right injury scale will be of the utmost importance in Quality Of Care studies evaluating the performance of trauma-systems.

However, we would like to be granted the chance to comment on this article in the next issue of the Journal of Trauma. The overall impression we get from this study is one of overly complex statistics to deduce an answer from a suboptimal study population.

Attached is the proposed Letter To The Editor.

Thank you for your consideration,

Sander Frankema

When Less seems to be More

We would like to congratulate Dr. Kilgo and his colleagues on their detailed analysis[1] of the use of a single or multiple injury descriptions in the assessment of injury severity. The best method to indicate injury severity for survival chance estimation is still unclear [2, 3]. This study based on an enormous sample from the National Trauma Databank (NTDB, n=160.000), should bring some of the answers closer.

We would, however, like to make some comments concerning the methods used in this study. Also, as a modest contribution to the requirement for external validation of these findings, as called for by Dr. Champion, we conducted a similar analysis in a registry of trauma patients (n=2.942) seen here in Rotterdam, the Netherlands.

The Rotterdam cohort seems to resemble the demographics of the NTDB group rather well (mean age: 34.9 vs 36.7, mortality: 6.4% vs 5.2%, NISS quartiles and median: 3, 8, 18 vs 4, 9, 17 respectively). The Rotterdam cohort is coded by AIS-90 only, so we can only provide performance data for models that use Survival Risk Ratios based on AIS-90 codes.

Several differences stand out when comparing the original article with the results from our analysis. First, all injury scales, when fitted to the Rotterdam cohort, were notably better discriminators compared to the results form the NTDB cohort. For instance NISS and MAXAIS achieved C-indices (area under ROC) of 0.96 and 0.94 and R^2 values of 0.58 and 0.53 respectively versus the NTBD-cohort: 0.87/0.88 and 0.34/0.38.

Second, the models’ performance in the NTDB seems to improve when limiting to multiple injuries only (compare tables 3 and 5). In the Rotterdam cohort the average injury severity increased when limited to multiple injuries only and, as expected, C-indices and R^2 decreased. These two findings could reflect some inconsistencies in the NTDB data set, for example that the models poorly fit to the single injury patients.

To validate the studies findings we made a performance comparison between the single injury models (MAXAIS and AISSRR1) to the multiple injuries models (NISS and TRAIS) in a number of sub-populations (total population, patients with multiple injuries, MAXAIS>2 and ISS>15).

In the AIS-90 models, NISS was able to outperform MAXAIS in all instances. Actually, the advantage of using multiple injuries increased considerably as injury severity increased (general population: ROC difference: 0.02 / R^2 difference: 0.049; multiple injuries: 0.03 / 0.073; AIS>2: 0.051 / 0.079; ISS>15: 0.074 / 0.114). These differences are approximately twice as large as the reported benefits of using single injuries in the original article.

A similar, but less profound effect was seen in the SRR-based models. The differences in C-indices between AISSSR1 and TRAIS increased along with the injury severity from 0.005 (total population) to 0.021 (ISS>15) in favor of the use of multiple injury codes. Finally, in our cohort the ISS was by far the worst indicator for injury severity. So, the authors rightfully exclude the ISS from their evaluation.

Adding vital parameters, age and injury mechanism to the equations led to performance improvements in all models, but did not chance the clear benefit of using multiple injuries over the single most severe injury.

A general concern with this study is the use of unnecessarily complex techniques. For this kind of comparison of scoring systems the assessment of discrimination ability (C-index) is sufficient to indicate performance. Any scoring system can be made to calibrate perfectly with the appropriate monotonic transformation, and this does not solve the problem of having a highly skewed underlying risk distribution. Thus fitting transformations and assessing lack of fit are an overkill. In addition, there is no need to worry about internal validation in huge datasets with relatively small number of predictor degrees of freedom[4], though we think in general that external validation is performed far too infrequently.

We would like to provoke the authors to conduct their study again in a more restrictive population with only multiple and more severe injuries. We wonder if the suggestion that ”patients are not dying from a litany of less severe secondary injuries [..] they are dying from single serious injuries” would remain standing.

Our relatively limited analysis confirms that the single most severe injury has substantial predictive performance. However, additional injury information may provide a considerable performance enhancement, particularly in patients with multiple or severe injuries.

Sander P.G. Frankema (1), Ewout W. Steyerberg (2), Frank E. Harrell, Jr. (3)

Department of General Surgery and Traumatology
Erasmus MC, Rotterdam
The Netherlands

Department of Public Health
Erasmus MC, Rotterdam
The Netherlands

Department of Biostatistics
Vanderbilt University School of Medicine
Nashville Tennessee, USA

1.	Kilgo PD, Osler TM, Meredith W. The worst injury predicts mortality outcome the best: rethinking the role of multiple injuries in trauma outcome scoring. J Trauma, 2003. 55(4): p. 599-606; discussion 606-7.

2.	Meredith JW, Evans G, Kilgo PD, et al. A comparison of the abilities of nine scoring algorithms in predicting mortality. J Trauma, 2002. 53(4): p. 621-8; discussion 628-9.

3.	Steyerberg EW, Frankema SP, Harrell FE, Jr. Statistical analyses of trauma outcome: when is more too much? J Trauma, 2003. 54(6): p. 1256-7; author reply 1257-8.

4.	Harrell FE, REGRESSION MODELING STRATEGIES. 2001, New York: Springer.

Opzet LTTE

MASTER: AZR9801.IMG

2003-11-18T02:12:00