Clinical Trialist
- This is tougher than most because many people rank stroke with severe disability as worse than death whereas ‘mild stroke’ is much better than death.
- I wonder how the two choices in question 2 affect the power and sample size of the study design
- The endpoint should capture all levels of severity without losing information and by allowing clinicians the opportunity to estimate chances of each or combinations of each. Any other approach imposes one’s own judgement on what is clinically important while ignoring other important thresholds.
- longitudinal, ordinal, and bayesian. and give patients (outside the trial) the chance to personally assign how they value different health states.
- It depends what you want to assess; if you want to know about efficacy of A vs B for stroke, then - in my opinion - all strokes is the metric of choice (assuming there are no deaths caused by strokes were stroke is unascertained because the person died). In that setting the deaths are ‘getting in the way’ of observing a stroke and I’d be tempted to use something like a Fine/Grey subdistrubution hazards or a straightforward time-to-event censoring at death. I’ve thought about trying / done some simulations with SDHR but never actually used it ‘in anger’ for a trial. On the other hand, if you want to have something truly patient-relavent (effectiveness), and you want an outcome encompassing disability and death whether to help with sample size / because patients think a disability or functional outcome important, then - again in my opinion - you need a death-or-new-significant-disability metric that is cause-of-disability agnostic (i.e. not just functional impairment caused the stroke). This is obviously tricky to measure and functional impairments are subjective and what do you do with people with significant disability/functional impairment at baseline. But, it makes more conceptual sense. Either evaluate whether drug A vs B prevents stroke or not (in which case death irrelevant other than as something with obscures stroke observation) or evaluate whether A vs B leads to the population being more likely to be alive and not functionally impaired (all-cause mortality + function). CoI junior clinician trialist with a public health angle, not a statistician.
- There is a signal (? Chance or real) the the treatment causes lower risk of stroke but higher right of death. If real, this should be communicated to clinicians.
- Outcome G, with The traditional comparison of two proportions is the least ‘gameable’ outcome.
- In this case, it’s specifically stated that the purpose of the study is to reduce risk of stroke (in general); then I’m not quite sure it makes sense to count death amongst the events (although I can see the argument for it). On the other hand, if we are to do an ordinal analysis that effectively grades the outcome of stroke, and weights some (in this case death) worse than others, why stop at a two-category approach - shouldn’t we grade the non-fatal strokes as well, as they’ll presumably be on some continuum of severity?
Clinician Not Engaged in Clinical Trial Design
- We object to ‘spin’ but the truth is that we need ‘stories’ to help to explain clinical trial results, just like in other areas of medicine.
- ordinal system appealing for stroke, here you state that the stokes included were ‘did not result in disability severe enough to be deemed worse than death’ - this is probably hard to quantify. I mostly see all strokes lumped together regardless of severity which can be misleading given the extreme heterogeneity of these patients.
- I would favor a time to event analysis of stroke accounting for death as a competing risk. Alternatively, a multi state model where stroke can occur multiple times and death is the consuming state.
- Stroke is a serious event, because it deprives us of healthy active daily life even if it’s not fatal. Maybe it’s better to extract ‘disabled stroke’ and set the outcome measure as composite of death or disabled stroke.
- I ignored the frequencies when answering the question because one will not know the results when prespecifying the analysis plan!
- The composite outcome to address competing risks is a major issue in clinical trials in neonatology, which has two unique features: 1) rates of death (usually early) are greater than 20% in some populations; and 2) there is broad interest in outcomes that only manifest later in life (e.g., blindness, developmental disability, lung disease). The standard in our field to use a composite outcome to address the competing risks is clearly problematic. The combined outcome often has qualitative heterogeneity (death is worse than lung disease to many patients); quantitative heterogeneity (the effect of intervention goes in different directions); loss of power (related to the prior issue). It’s led to some poor trial interpretations. See, e.g.: https://jamanetwork.com/journals/jama/fullarticle/2722773 https://www.nejm.org/doi/full/10.1056/NEJMoa0911781 https://jamanetwork.com/journals/jama/fullarticle/1866096 It would be great to change the paradigm. There has been some interest in the win-ratio approach among some trialist friends, although I’m not aware of any major neonatal trials taking this yet.
- Report both Stroke Death A drug might decrease strokes but have no difference in mortality overall (due to increasing). If you see a decrease in stroke or no difference/increase in death it can facilitate further research into why - obviously it helped stop strokes, but maybe dose was too high and it caused bleeds etc. This gives us every chance to see a potentially effective treatment and fine tune it. Maybe with said tuning we might see the decrease in strokes and decrease in mortality (decreasing dose for example might stop the bleeds that caused the death issue)
- I object to the methods of this survey, because none of these questions can truly be answered without knowing the patient selection, study intervention, or cause of death in those 5 non stroke deaths.
- None
Statistician
- why not relative weights instead of ordinal analysis only?
- In discussing design and reviewing background information about the drug’s mechanism of action from pre-clinical studies, what does the DAG look like? Is there a plausible direct effect of drug on mortality (not mediated by stroke) based on the known biology? Without detail like that, there may not be a single answer. For example, I respond ‘e’ under the assumption of no direct effect of drug on mortality. Consider counterfactuals where there’s a subpopulation whose counterfactual outcomes are no stroke if given drug A and stroke if given B. Maybe people ‘prevented’ from having stroke because of drug A therefore didn’t get re-admitted, didn’t get close contact with the health system, etc - and it’s that downstream (multiple layers of mediation) that causes higher mortality in A. So under that assumption of no direct effect, I analyze (e), and if I show benefit then it’s up to later studies to develop interventions to reduce death in that subpopulation based on how to intervene on those mediators.
- Pre-definition matters. Also, death needs always be considered by itself in addition to whatever else. Depending on purpose and setting, importance of clinical importance vs hypothesised mechanism for outcomes may differ. Also, usually drug regulatory recommendations are quite good.
- There are numerous similar situations which are often handled poorly. An example is a recent ICU trial where the main outcome is a biochemical measurement ( repeated measures through time) but this is truncated by death or by recovery and discharge from hospital.
- Assume stroke diagnosis is perfect; and no loss to follow-up
- time to event (stroke or death) analysis with potentially more follow-up time seems better than either option in #2
- These sorts of dependent variables are difficult because in my experience there’s little evidence that efforts to disentangle the natural contamination of, say stroke rate with death, is helped by the construction of some sort of composite.I tend to see on the side of multivariate analyses with all their interpretation issues.
- I picked G to Q1, but of course I wouldn’t want to have to choose a single event frequency. I think reporting on most of the categories is interesting, with primaries being E, F, and G. Interesting questions!
- should collect time of stroke/death and use survival analysis.
- I think I would analyze as time to event. Look at stroke free survival and OS. Seems more deaths in armA but less stroke. Then would looks at causes of death and as profile to try to complete the story. Perhaps drug A prevents some stroke but it is too toxic making the benefit risk profile negative
- Days alive free of death and stroke as the outcome should be considered Competing risk of death could be analysed using time to event models
- Multi state
- Non-fatal stroke is not a yes/no thing so that’s why I combine it with death. Stroke followed by years of being in a locked-in state might be worse than death.
- Would also consider analyzing stroke and considering hypothetical as if death from unrelated causes had not happened, if such death were (known a-priori to be) rare (but they are not). Also worry that some deaths with unknown cause will be strokes were we just did not find out that they were strokes, similarly, people that die may also have been at much higher risk of subsequent stroke if the event leading to death had not happened.
- competing risks analysis, with non-stroke related deaths as competing event
- KM + cox seems apt. The death of the patient is of little interest, as any patient who dies after, say, 20 days without having a stroke would be indistinguishible from a patient who had the treatment 20 days ago and have yet to either die or have a stroke. The people who die after the stroke are of little interest IF we are only looking at the treatments effect on having a stroke. The patients who die are accounted for in the KM by raising the uncertainty of the KM-estimate.
- Survival analysis with competing risks
- A would choose traditional analysis in 2) since I wan tto focus on stroke/no stroke. One could perform an analysis taking into account possible competing risks (dying from something else than stroke). It is important to state the main outcome and do the proper analysis according to that (as always:))
Epidemiologist
- The treatment-decision-relevant outcome is ‘stroke or death,’ because what is the point of stroke prevention at the cost of more deaths? But if you’re interested in something other than a treat/don’t treat decision then some other outcome may be more useful; it is hard to evaluate the question when the ultimate scientific purpose has not been specified. If you wanted to understand the risk of stroke while treating death as a competing risk, you could do so; but it is not clear what decision-utility the result of this analysis would have. Death is obviously worse than stroke, and the ordinal analysis is helpful in acknowledging this – but no one will understand what policy decisions should be made on the basis of an ordinal analysis.
- I’m a proponent of weighting outcomes using disutilities, BUT current reality is that this is not yet well accepted. Therefore, given current conventions, I prefer removing minor events from the composite instead of weighting outcomes (such as do NOT include minor strokes w/o functionally substantive morbidity).
- For a, I had trouble. Decided stroke bc that was the main hypothesis, that the drug prevents stroke
- I feel the scenario is quite similar for what we have to do to interpret most trials: most drugs that are effective also come with adverse effects that are not reflected in the primary outcome of trials. In some ways this example is easier because it is clear that death is worse than stroke and the probabilities of both are in the same range. That IMO makes it an easier trade off than trying to decide whether a 0.1% chance of a life threatening adverse effect is more important than a 40% of good response to treatment for a serious but not immediately life threatening illness.
Other
- Common sense approach. If treatment kills and you don’t count it - that is terrible. Better to scale death versus stroke and try and understand both if you can.
- Without knowing the answer, the optimal approach (which may be infeasible in practice) would be one which provides outcome summaries in such a way that it could be interpreted by clinicians/patients with a range of utility functions regarding level of stroke disability and the desire to continue to live. Ie. Some patients may view any survival as preferable to death whereas others may view being physically dependent as status not worth surviving to
- I work in health technology assessment. Small event numbers- such as these- are always hard to assess regardless of how we present and analyse them.
- Death is always the worst outcome.