% template.tex for Biometrics papers
% See https://onlinelibrary.wiley.com/page/journal/15410420/homepage/forauthors.html
%
%  This file provides a template for Biometrics authors.  Use this
%  template as the starting point for creating your manuscript document.
%  See the file biomsample.tex for an example of a full-blown manuscript.

%  ALWAYS USE THE referee OPTION WITH PAPERS SUBMITTED TO BIOMETRICS!!!
%  You can see what your paper would look like typeset by removing
%  the referee option.  Because the typeset version will be in two
%  columns, however, some of your equations may be too long. DO NOT
%  use the \longequation option discussed in the user guide!!!  This option
%  is reserved ONLY for equations that are impossible to split across 
%  multiple lines; e.g., a very wide matrix.  Instead, type your equations 
%  so that they stay in one column and are split across several lines, 
%  as are almost all equations in the journal.  Use a recent version of the
%  journal as a guide. 
%
% FH: usegraphicx could not find bounding box info
\documentclass[useAMS,referee]{biom}
\usepackage{graphicx}
%
%  If your system does not have the AMS fonts version 2.0 installed, then
%  remove the useAMS option.
%
%  useAMS allows you to obtain upright Greek characters.
%  e.g. \umu, \upi etc.  See the section on "Upright Greek characters" in
%  this guide for further information.
%
%  If you are using AMS 2.0 fonts, bold math letters/symbols are available
%  at a larger range of sizes for NFSS release 1 and 2 (using \boldmath or
%  preferably \bmath).
% 
%  Other options are described in the user guide. Here are a few:
% 
%  -  If you use Patrick Daly's natbib  to cross-reference your 
%     bibliography entries, use the usenatbib option
%
%  -  If you use \includegraphics (graphicx package) for importing graphics
%     into your figures, use the usegraphicx option
% 
%  If you wish to typeset the paper in Times font (if you do not have the
%  PostScript Type 1 Computer Modern fonts you will need to do this to get
%  smoother fonts in a PDF file) then uncomment the next line
%  \usepackage{Times}

%%%%% PLACE YOUR OWN MACROS HERE %%%%%

\def\bSig\mathbf{\Sigma}
\newcommand{\VS}{V\&S}
\newcommand{\tr}{\mbox{tr}}
\def\etal{\emph{et al.}}

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

\title[Commentary]{Commentary on Improving Precison and Power in
  Randomized Trials for COVID-19 Treatments Using Covariate
  Adjustment, for Binary, Ordinal, and Time-to-Event Outcomes}


\author{Frank E Harrell Jr\emailx{f.harrell@vumc.org} \\
Department of Biostatistics, Vanderbilt University School of Medicine,
Nashville TN, U.S.A.}

\begin{document}

%  This will produce the submission and review information that appears
%  right after the reference section.  Of course, it will be unknown when
%  you submit your paper, so you can either leave this out or put in 
%  sample dates (these will have no effect on the fate of your paper in the
%  review process!)

\date{{\it Received November} 2020. {\it Revised } 2020.  {\it
Accepted } 2020.}

%  These options will count the number of pages and provide volume
%  and date information in the upper left hand corner of the top of the 
%  first page as in published papers.  The \pagerange command will only
%  work if you place the command \label{firstpage} near the beginning
%  of the document and \label{lastpage} at the end of the document, as we
%  have done in this template.

%  Again, putting a volume number and date is for your own amusement and
%  has no bearing on what actually happens to your paper!  

\pagerange{\pageref{firstpage}--\pageref{lastpage}} 
\volume{99}
\pubyear{2020}
\artmonth{December}

%  The \doi command is where the DOI for your paper would be placed should it
%  be published.  Again, if you make one up and stick it here, it means 
%  nothing!

\doi{10.1111/j.1541-0420.2005.00454.x}

%  This label and the label ``lastpage'' are used by the \pagerange
%  command above to give the page range for the article.  You may have 
%  to process the document twice to get this to match up with what you 
%  expect.  When using the referee option, this will not count the pages
%  with tables and figures.  

\label{firstpage}

%  put the summary for your paper here

\begin{abstract}
Standard covariate adjustment as commonly used in randomized clinical trials recognizes which quantities are likely to be constant (relative treatment effects) and which quantities are likely to vary (within-treatment-grouop outcomes and absolute treatment effects).  Modern statistial modeling tools such as regression splines, penalized maximum likelihood estimation, Bayesian shrinkage priors, semiparametric models, and hierarchical or serial correlation models allow for great flexibility in the covariate adjustment context.  A large number of parametric model adaptations are available without changing the meaning of model parameters or adding complexity to interpretation.  Standard covariate adjustment provides an ideal basis for capturing evidence about heterogeneity of treatment effects through formal interaction assessment.  It is argued that absolute treatment effects (e.g., absolute risk reduction estimates) should be computed only on an individual patient basis and should not be averaged, because of the predictably large variation in risk reduction across patient types.  This is demonstrated with a large randomized trial.  Quantifying treatment effects through average absolute risk reductions hides many interesting phenomena, is inconsistent with individual patient decision making, is not demonstrated to add value, and provides less insights than standard regression modeling.  And since it is not likelihood based, focusing on average absolute treatment effects does not build a bridge to Bayesian or longitudinal frequentist models that are required to take 
external information and various design complexities into account.
\end{abstract}

\begin{keywords}
Estimand; covariate adjustment; likelihood; penalized maximum
likelihood estimation; logistic regression; proportional odds model;
randomized clinical trial; efficiency; odds ratio; absolute risk
reduction; outcome variable; regression splines; precision medicine;
Bayesian methods. 
\end{keywords}

%  As usual, the \maketitle command creates the title and author/affiliations
%  display 

\maketitle

%  If you are using the referee option, a new page, numbered page 1, will
%  start after the summary and keywords.  The page numbers thus count the
%  number of pages of your manuscript in the preferred submission style.
%  Remember, ``Normally, regular papers exceeding 25 pages and Reader Reaction 
%  papers exceeding 12 pages in (the preferred style) will be returned to 
%  the authors without review. The page limit includes acknowledgements, 
%  references, and appendices, but not tables and figures. The page count does 
%  not include the title page and abstract. A maximum of six (6) tables or 
%  figures combined is often required.''

%  You may now place the substance of your manuscript here.  Please use
%  the \section, \subsection, etc commands as described in the user guide.
%  Please use \label and \ref commands to cross-reference sections, equations,
%  tables, figures, etc.
%
%  Please DO NOT attempt to reformat the style of equation numbering!
%  For that matter, please do not attempt to redefine anything!

\section{Introduction}
Benkeser~\etal\ (2020) are quite correct in stating
that covariate adjustment is a much underutilized statistical method
in randomized clinical trials.  The resulting loss of power and
precision and inflation of sample size borders on scandalous.
But I fear that the authors' remedy will make clinical trialists \emph{less}
likely to account for covariates.  This is because the authors'
approach is overly complicated without a corresponding gain over
traditional covariate adjustment, is harder to interpret, can
provide misleading effect estimates by the averaging of unlikes as discussed in
Section~\ref{sec:est}, is harder to pre-specify (pre-specification
being a hallmark of rigorous randomized clinical trials), does not
provide a basis for interaction assessment (heterogeneity of treatment
effect), and does not extend to longitudinal or clustered data.  Symptomatic
of the authors approach  being overly complex is their need to develop
a separate outcome model for the treated group.  One can argue that
what the authors proposed should not be called \emph{covariate
  adjustment} in the usual sense, but I will leave that for others to debate.

When I read a statistical methods paper, the first questions I ask
myself are these: Does it solve a problem that needed to be solved?
Are there other problems that would have been better to solve?  Did the authors conduct a proper comparative
study to demonstrate the net benefit of the new method? 
Unfortunately, I do not view the authors' paper favorably on these
counts.  Thinking of all the many problems we have in COVID-19
therapeutic research alone, we have more pressing problems that talented
statistical researchers such as Benkeser~\etal\ could have
attacked instead.  Some of these problems include
\begin{enumerate}
\item How do we model outcomes in a way that recognizes that a
  treatment may not affect mortality by the same amount that it
  affects non-fatal outcomes?
\item What is the best statistical evidential measure for whether a
  mortality effect is consistent with the other effects?
\item What are the best ways to judge which treatment is better when
  results for different patient outcomes conflict?
\item What is the best combination of statistical efficiency and
  clinical interpretation in constructing an outcome variable?
\item What is the information gain from a longitudinal ordinal
  response when compared to a time-to-event outcome?
\item How should one elicit prior distributions in a pandemic, or how
  should one form skeptical priors?
\item How should simultaneous studies inform each other during a pandemic?
\item What is the optimal number of parameters to devote to covariate
  adjustment and what is the best way to relax linearity assumptions
  when doing covariate modeling?
\item Is it better to use subject matter knowledge to pre-specify a
  rather small number of covariates, or should one use a large number
  of covariates with a ridge ($L_{2}$) penalty for their
  effects (Chen, \etal, 2016)?
\item What is the best way to model treatment $\times$ covariate interactions?  Should
  Bayesian priors be put on the interaction effects, and how should
  such priors be elicited?
\item Can Bayesian models provide exact small-sample inference in the
  presence of missing covariates?
\item What is the best adaptive design to use and how often should one
  analyze the data?
\item What is the best longitudinal model to use for ordinal outcomes,
  is a simple random effects model as good as a serial correlation model,
  what prior distributions work best for correlation-related
  parameters, and should absorbing states be treated in a special way?
\end{enumerate}

To frame the discussion below, consider an alternative to the authors'
proposed methods: flexible parametric models that adjust for key
pre-specified covariates without assuming that continuous covariates
operate linearly (e.g., continuous covariates are represented with
regression splines in the model).  Such a standard model has the
following properties.

\begin{enumerate}
\item The parametric model parameterizes
the treatment effect on a scale for which it is possible for the
treatment effect to be constant (log odds, log hazard, etc.) and hence
represented by a single number.
\item Because the treatment effect parameter has an unrestricted
  range, the need for interactions between treatment and covariates is
  minimized.
\item The model provides a basis for interactions that more likely
  represent biologic or pharmacologic effects than tricks to restrict
  the model's probability estimates to $[0,1]$.
\item The model is readily extended to handle longitudinal binary and
  ordinal responses and multi-level models (e.g., days within patients
  within clinical centers).
\item Well studied multiple imputation procedures exist to handle
  missing covariates.
\item Full likelihood methods used in fitting the model elegantly
  handle various forms of censoring and truncation.
\end{enumerate}

\section{Estimand}\label{sec:est}

Benkeser~\etal's primary estimand is the average difference in probability in
going from treatment A to treatment B.
The authors have chosen a method of incorporating covariates that is flexible,
but their estimand hides important data features.  Standard regression models are very likely to fit
data well enough to provide excellent covariate adjustment, but an
estimand that represents a differences in averages (e.g., an overall
absolute risk reduction---ARR) is an example of combining unlikes.
ARR due to a treatment is a strong function of the base risk.  For
example, patients who are sicker at baseline have ``more room to
move'' so have larger ARR than less sick patients.  ARR is an estimand
that should be estimated only on a single-patient
basis~(Harrell and Lazzeroni, 2017; Harrell 2018, Harrell 2019).  In
the binary outcome case, personalized ARR is a simple function of the
odds ratio and baseline risk in the absence of treatment interactions.  When there is a
treatment interaction on the logit scale, the difference in ARR that is
estimated by the difference in two logistics model is only slightly more complex.

The authors did not expose this issue to the readers.  For example,
one would find that a baseline-risk-specific version of Figure~1 would
reveal much variation in ARR.

As an example, the GUSTO~I study (GUSTO Investigators, 1993) of
thrombolytics in treatment of acute myocardial infarction with its
sample size of 41,021 patients has been analyzed in detail, and
various risk models have been developed from this randomized trial's
data (Steyerberg, 2020).  As shown in Harrell (2020b), for a treatment
comparison of major interest, accelerated t-PA ($n=10,348$) vs.\
streptokinase (SK, $n=20,162$), the
average ARR for the probability of 30d mortality is 0.011.  But this is misleading as it is
dominated by a minority of high risk patients as shown in Figure~1
described below.  The median ARR is
0.007 which is much more representative of what patients can expect.
But far better is to provide individualized ARR estimates.

In the analysis of $n=40,830$ patients (2851 deaths) in GUSTO-I
presented in Harrell (2019), three treatments (two indicator
variables) were allowed to interact with 6
covariates, with the continuous covariates expanded into spline functions.  The 
likelihood ratio $\chi^{2}$ test statistic was 16.6 on 20 d.f., $p=0.68$
showing little evidence for interaction.  Thus the assumption of
constancy of the treatment odds ratios is difficult to reject.  Another
way to allow a more flexible model fit is to penalize all the
interaction terms down to what optimally cross-validates.  Using a
quadratic (ridge) penalty, the optimum penalty was $\infty$, again
indicating no reason that the treatment odds ratio should not be
considered to be a single number.  Giving interactions one more
benefit of the doubt, penalized maximum likelihood estimates were
obtained, penalizing all the covariate $\times$ treatment interaction
terms to a effectively a single degree of freedom.  This model was
used to estimate the distribution of individual patient ARR with t-PA
shown in Figure~1.

\begin{figure}
  \begin{center}
  \includegraphics[width=.65\textwidth]{darr.pdf}
  \caption{Distribution of ARR for 30d mortality with t-PA obtained
    after computing the 
    difference in two predicted values for each patient (setting
    treatment to t-PA then to SK) using a
    penalized interaction binary logistic model.  Left arrow: median
    difference over patients; right arrow: mean ARR.}
  \end{center}
\end{figure}

One sees the large amount of variation in ARR.  Other results in
Harrell (2019) show large variation in risk ratios and minimal
variation on ORs (and keep in mind that the optimal estimate of this
variation is in fact zero).  The relationship between baseline risk under
control therapy (SK) and the estimated ARR is shown in Figure~2.

\begin{figure}
  \begin{center}
  \includegraphics[width=.85\textwidth]{vsbase-1.pdf}
  \caption{Per-patient ARR estimates vs.\ their baseline (SK) risk
    obtained from the same penalized interaction model.  The darkness
    of points increases as the frequency of patients at the point
    increases, as defined in the key to the right of the plot.  About
    3200 patients had baseline risk and risk reduction near zero.
    Were interactions forced to be zero (which would have been optimal
  by both a likelihood ratio test and AIC), all the points would be on
a single curve.}
  \end{center}
\end{figure}


Through proper conditioning and avoidance of averaging of unlikes by
estimating ARR for individual patients and not averaging these
estimates over patients, it is seen that standard covariate adjustment
using well 
accepted relative treatment effect estimands is simpler,
can fit data patterns better, requires only standard software, and is
more insightful.

\section{Lack of Comparisons}
Benkeser~\etal\ showed large efficiency gains of their approach over
ignoring covariates.  But the authors did not compare the power and mean squared error of the treatment
estimate (on a scale for which the treatment effect may possibly be
constant) for their approach vs.\ standard covariate adjustment.  This
leaves readers wondering whether the new method is worth the trouble
or is in fact less efficient than the standard.

\section{New Odds Ratio Estimator}
With the aim of dealing with non-proportional odds, the authors
developed a weighted log odds ratio estimator.  But the standard
maximum likelihood estimator already solves this problem.  As detailed
in~Harrell (2020), the Wilcoxon-Mann-Whitney concordance probability $c$,
i.e., the probability that a randomly chosen patient on treatment B
has a higher level response $Y$ than a randomly chosen patient on
treatment A, also called the \emph{probability index}, is a simple
function of the maximum likelihood estimate of the treatment
regression coefficient \emph{whether or not proportional odds holds}.
The conversion equation is $c = \frac{\mathrm{OR}^{0.66}}{1 +
  \mathrm{OR}^{0.66}}$ where OR = $\exp(\hat{\beta})$.  This formula is
accurate to an average absolute error for computing $c$ from
$\hat{\beta}$ of 0.002.

The equivalence of the OR and the Wilcoxon-Mann-Whitney estimand also makes
the authors' estimand 2 in Section 3.2 somewhat moot.

\section{Lack of a Likelihood}

COVID-19 therapeutic research is an area where Bayesian methods are
being used with increasing frequency.  Frequentist methods that use a
full likelihood approach provide an excellent bridge to development of
or comparison with Bayesian analogs that use the same likelihood and
only need to add a prior.  The authors' methods are not
likelihood based, so they do not provide a bridge to Bayes.  The
proposed methods do not provide exact inference for small $n$ as
does Bayes, has no way of incorporating skepticism or external
information about treatment effects, and has no way to quantify
evidence that is as actionable as posterior probabilities of (1) any
efficacy and (2) clinically meaningful efficacy.

The authors briefly discuss missing ordinal outcomes.  Instead of this
being an issue of missingness, in many situations an ordinal
outcome is interval censored.  This is the case, for example, when a
range of ordinal values is not ascertained on a given day.  To deal
with interval censoring, a full likelihood approach is helpful, and the
authors' approach may not be extendible to deal with general
censoring.

The lack of a likelihood also prevents the authors' approach from
dealing with variation across sites in a multi-site clinical trial
through the use of random effects.  The flexibility of a Bayesian
longitudinal proportional odds model that allows for general censoring
and departures from the proportional odds assumption is described in
Harrell (2020c).

\section{Technical Issues}
The authors did not mean to imply that covariate
adjustment lowers the variance of treatment effect estimates for all
estimands, but their statement ``By incorporating baseline variable
information, covariate adjusted estimators often enjoy smaller
variance \dots'' can be misinterpreted.  In logistic and Cox models,
for example, covariate adjustment increases treatment effect variance
on the log ratio scale (see Harrell 2020b and Steyerberg 2020 for a
literature review).  Despite this increase, 
traditional covariate adjustment still results in increased Bayesian or
frequentist power because the model being more correct (i.e.,
some of the previously unexplained outcome heterogeneity is now
explained) moves the treatment effect estimate farther from
zero.  The regression coefficient increases in absolute value faster
than the standard error increases, 
hence the power gain.  For logistic and Cox models, variance reduction
occurs only on probability-scale estimands.

In their Section 6 the authors recommended that when a utility
function can be agreed upon, one should consider the difference in
mean utilities between treatment arms when the outcome is ordinal.
Even though the difference in mean utilities can be a highly
appropriate measure of treatment effectiveness, it is important to
note that the patient-specific utilities are still discrete and are
likely to have a very strange, even bimodal, distribution.  Hence
utilities may be best modeled with a semiparametric model such as the
proportional odds model.

The authors credited the proportional odds model to McCullagh (1980) but
this model was developed by Walker and Duncan (1967) and other
work even predates this.

The authors tout asymptotic accuracy of their method, which is of
interest to statisticians but not practitioners who need to know the
accuracy of the method in their (usually too small) sample.

The authors did not seem to have rigorously evaluated the accuracy of
bootstrap confidence intervals.  I have an example where none of the
standard bootstraps provides sufficient accuracy for a confidence
interval for a standard logistic model odds ratio.  Non-coverage
probabilities are not close to 0.025 in either tail when attempting to
compute a 0.95 interval.  It is important
to evaluate the left and right non-coverage probabilities, as the
confidence interval can be right on the average but wrong for both tails.

The use of categorization for continuous predictors (e.g., the
treatment of age just before section 4.1.3) does not represent
best statistical practice.

To be very picky, the authors' (and so many other authors) use of the
term ``type I error'' does not refer to the probability of an error
but rather to the probability of making an assertion.

The paper advises readers to consider using variable selection
algorithms.  Stepwise variable selection brings a host of problems,
typically ruins standard error estimates (Greenland 2000), and is not
consistent with full pre-specification.

The idea to use information monitoring in forming stopping rules needs
to be checked for consistency with optimum decision making, and it may
be difficult to specify the information threshold.

Related to missing covariates, the recommendation to use single
imputation and its implication to not use $Y$ in the imputation
process has been well studied and found to be lacking, especially with
regard to getting standard errors correct.

In the authors' Supporting Information, the intuition for how
covariate adjustment can lead to precision gains begins with a
discussion of covariate imbalance.  Imbalance has very little to do
with covariate adjustment in randomized trials.  Instead, it is all
about adjusting for easily explainable outcome heterogeneity.
Similarly, in the second paragraph the extremely crude age adjustment
is, contrary to the authors assertion, not designed ``to be
interpreted as correcting for empirical confounding by baseline
variables'', as there is no confounding in a randomized trial.  At any
rate, apparent covariate imbalances are expected to be offset by
counterbalancing covariates one did not bother to analyze. 

Methods that develop models from only one treatment arm are prone to
overfitting, which entails fitting idiosyncratic associations in that
arm in such a way that when the outcome comparison is made with the
other arm can result in bias in non-huge samples.

Omission of simulated samples with empty cells may slightly bias
simulation results and is not necessary in ordinary proportional odds
modeling.

Contrasted with the group sequential design outlined by the authors, a
continuously sequential Bayesian design using a traditional
proportional odds model for covariate adjustment is likely to provide
more understandable evidence and result in earlier stopping for
efficacy, harm, or futility.

\backmatter

%  Here, we create the bibliographic entries manually, following the
%  journal style.  If you use this method or use natbib, PLEASE PAY
%  CAREFUL ATTENTION TO THE BIBLIOGRAPHIC STYLE IN A RECENT ISSUE OF
%  THE JOURNAL AND FOLLOW IT!  Failure to follow stylistic conventions
%  just lengthens the time spend copyediting your paper and hence its
%  position in the publication queue should it be accepted.

%  We greatly prefer that you incorporate the references for your
%  article into the body of the article as we have done here 
%  (you can use natbib or not as you choose) than use BiBTeX,
%  so that your article is self-contained in one file.
%  If you do use BiBTeX, please use the .bst file that comes with 
%  the distribution.  In this case, replace the thebibliography
%  environment below by 
%
%\bibliographystyle{biom} 
%\bibliography{/home/harrelfe/bib/harrelfe.bib}

%\ifdefined\DeclarePrefChars\DeclarePrefChars{'’-}\else\fi
\begin{thebibliography}{}

%\bibitem{ } Cox, D. R. (1972). Regression models and life tables (with
%discussion).  \textit{Journal of the Royal Statistical Society, Series B}
%\textbf{34,} 187--200.

\bibitem{}
Benkeser, D., D\'{i}az, I., Luedtke, A., Segal, J., Scharfstein, D., and
  Rosenblum, M. (2000)
\newblock Improving precision and power in randomized trials for
  COVID-19 treatments using covariate adjustment, for binary,
  ordinal, and time-to-event outcomes.

\bibitem{}
GUSTO Investigators (1993)
\newblock An international randomized trial comparing four
thrombolytic strategies for acute myocardial infarction.  \textit{New
  England Journal of Medicine} {\bf 329,} 673-682.
  
\bibitem{}
Chen, Q., Nian, H., Zhu, Y., Talbot, H.~K., Griffin, M.~R., and
Harrell, F.~E. (2016)
\newblock Too many covariates and too few cases? - a comparative
study. \textit{Statistics in Medicine} {\bf 35,}
  4546--4558.

\bibitem{}
Greenland, S. (2000)
\newblock When should epidemiologic regressions use random
coefficients? \textit{Biometrics} {\bf 56,} 915--921.

\bibitem{}
Harrell, F.~E. (2020)
\newblock Violation of proportional odds is not fatal.
\newblock \texttt{fharrell.com/post/po}, accessed 2020-11-22

\bibitem{}
Harrell, F.~E. (2020b)
\newblock Analysis of covariance in randomized studies.
\newblock \texttt{hbiostat.org/bbr/md/ancova.html}, accessed 2020-11-22

\bibitem{}
Harrell, F.~E. (2020c)
\newblock R \texttt{rmsb} package.  
\newblock \texttt{hbiostat.org/R/rmsb}, accessed 2020-11-26

\bibitem{}
Harrell, F.~E. (2019)
\newblock Assessing heterogeneity of treatment effect, estimating
  patient-specific efficacy, and studying variation in odds
  ratios, risk ratios, and risk differences.
\newblock \texttt{fharrell.com/post/varyor}, accessed 2020-11-22

\bibitem{}
Harrell, F.~E. (2018)
\newblock Viewpoints on heterogeneity of treatment effect and
precision medicine.
\newblock \texttt{fharrell.com/post/hteview}, accessed 2020-11-22

\bibitem{}
Harrell, F.~E. and Lazzeroni, L. (2017)
\newblock EHRs and RCTs: Outcome prediction vs.\ optimal treatment
selection.
\newblock \texttt{fharrell.com/post/ehrs-rcts}, accessed 2020-11-22

\bibitem{}
McCullagh, P. (1980)
\newblock Regression models for ordinal data. \textit{Journal of the
  Royal Statistical Society Series B} {\bf 42,} 109--142.

\bibitem{}
  Steyerberg, E.~W. (2020)
  \newblock RCT analyses with covariate adjustment
  \newblock \texttt{fharrell.com/post/covadj}, accessed 2020-11-25

\bibitem{}
Walker, S.~H. and Duncan, D.~B. (1967)
\newblock Estimation of the probability of an event as a function of several
  independent variables. \textit{Biometrika} {\bf 54,} 167--178.

\end{thebibliography}

\section{Grant Support}

This work was supported by CTSA award No. UL1 TR002243 from the National Center for Advancing Translational Sciences. Its contents are solely the responsibility of the authors and do not necessarily represent official views of the National Center for Advancing Translational Sciences or the National Institutes of Health.
TODO
CONNECTS is supported by NIH NHLBI 1OT2HL156812-01, ACTIV Integration of Host-targeting Therapies for COVID-19 Administrative Coordinating Center from the National Heart, Lung. and Blood Institute (NHLBI).

\label{lastpage}

\end{document}
