\documentstyle[11pt]{article}
\renewcommand{\baselinestretch}{2}
%\textwidth 6.75in
%\textheight 8.50in
%\oddsidemargin 0.0in
%\evensidemargin 0.0in
%\topmargin -0.25in

\large
\hyphenation{Bio-met-rik-a}

\newcommand\bxi{ x \sb i}
\newcommand\bbeta{\beta}
\newcommand\bhbeta{ {\hat \beta} }
\newcommand\btbeta{ {\tilde \beta} }
\newcommand\btbeti{ {\tilde \beta \sb i} }
\newcommand\btau{\tau}

\pagestyle{plain}

%\hrule \vskip .375in


\begin{document}

\begin{center}
{\bf
A SIMPLE MODEL FOR ANALYZING
RECEIVER OPERATING CURVES
}
\end{center}



\begin{center}
Philip J.~Smith, Michael M. Engelgau, William H. Herman\footnote{
{\sl Address for correspondence:}
Centers for Disease Control,
Division of Diabetes Translation,
4770 Buford Highway, N.E., 
Mailstop K--10,
Rhodes Building,
Atlanta, Georgia 30341--3724 U.S.A.}
\end{center}

\begin{center}
{\today}
\end{center}


%\maketitle

\section*{\bf Abstract}


\medskip

\noindent
KEY WORDS:
bootstrap intervals,
ordered categorical data,
hazards model,
sensitivity,
specificity.

\section{Introduction }

The accuracy of a diagnostic test 
for predicting the presence or absence of a medical condition
is commonly expressed in terms of 
its sensitivity and specificity 
with respect to a traditionally used and accepted test regarded as a
`gold standard.'
In this context,
sensitivity is the probability
that the screening test indicates 
that the condition is present when
the gold standard does, also,
and
specificity is the probability
that the screening test indicates
that the condition is absent when
the gold standard indicates
that it is absent, also.

In many applications 
the data obtained from the diagnostic test
is continuous. These data are then evaluated
as being above or below each of several selected ``cut--points''
at which sensitivity and specificity is evaluated.
A plot of sensitivity 
versus 
one minus specificity 
for the range of possible cut points 
defines the receiver operating (ROC) curve for the diagnostic test
and provides a summary of disease detectability.
Metz~(1978) discusses the basic principles of ROC analysis.

In many applications disease detectability may be thought to 
confounded with important covariates. 
Although ROC curves may be estimated 
for strata defined by specific values or ranges
of the covariates, 
sample sizes within each strata may be small and
yield unstable estimates for each curve.
In this case,
pooling data accross strata
and
using a multivariate approach for estimating 
the ROC curve may be useful for obtaining
smoothed estimates for each strata.  
Tosteson and Begg (1988) and Coughlin et al. (1992)
provide previous discussion of this issue.

This paper presents simple multivariate methods  for modeling the ROC
curve as a function of covariates. In contrast to the
work of Tosteson and Begg the methods described in this paper
allow estimation of the ROC curve 
to be adjusted for continuous covariates
and require
the use of logistic regression
rather than more complex multivariate methods.

To illustrate the methods, 
we investigate
a post prandial capillary glucose measurement 
obtained from a glucometer 
as a potential diagnostic test
for persons with diabetes.
The gold standard established by
the World Health Organization~(1985)  
for diagnosing diabetes melitus include
a fasting plasma glucose value 
$\ge 140 \mbox{mg/dl}$
or a 2 hour plasma glucose value 
$\ge 200 \mbox{mg/dl}$
subsequent to a 75g glucose challange.
Because glucose levels 
are 
dependent on the duration 
of the post prandial period 
we include duration in our model to correct 
for its potential confounding effect upon
sensitivity and specificity.
Also, since glucose metabolism and levels
during a post prandial period
are believed to be
dependent upon age we consider this
as a potential predictor, also.

In Section~2 a simple model for ordered categorical data is sketched
and applied to the problem of estimating the ROC curve. Also, methods
are proposed for estimating the standard errors
of sensitivity and specificity for any values of covariates
included in the model. Section~3 uses the post prandial capillary
glucose data to illustrate the methods.

\section{A multivariate model for the ROC curve}

\subsection{A simple model for ordered categorical data}

Let $a \sb 0 < a \sb 1 < \ldots < a \sb K $ denote 
specified post prandial capillary glucose values
obtained from the diagnostic test.
An objective of ROC analysis is 
to estimate sensitivity and specificity 
at each of these values 
in order to construct the ROC curve.
%These values are candidates
%from which an ``optimal cut point''
%may be chosen to define 
%positive and negative screening test outcomes.
The $j$th category defined by these
ordered glucose values is
$I \sb j = ( a \sb {j-1} , a \sb j ].$
For generality, let $a \sb 0 = -\infty$
and $a \sb K = +\infty.$
%
Also, let 
$\{ d \sb j \}$ denote counts of capillary glucose values
observed within the 
ordered categories $\{ I \sb j \}$ 
from a random sample of size $n = \sum d \sb j .$ 

Let
$f(\cdot),$ 
$S(\cdot),$
and
$h(\cdot)$
denote 
the density function,
the complement of the cumulative distribution function,
and
the hazard function of the underlying continuous
distribution of post prandial capillary glucose values, $A.$
Given the ordered categories $\{ I \sb j \},$
we define the probability mass function on the ordered
categories by
\begin{eqnarray*}
\label{eq:massp}
f \sb j =
P(A \in I \sb j ) =
s \sb j - s \sb {j+1}
\end{eqnarray*}
\noindent
where
\begin{eqnarray}
\label{eq:survp}
s \sb j = P(A > a \sb {j-1}) = \sum \sb {k=j} \sp K f \sb k
\end{eqnarray}
\noindent
is the probability of exceeding the $j-1$st cut point, and
\begin{eqnarray}
\label{eq:hazdp}
\pi \sb j = P (A \in I \sb j | A > a \sb {j-1} ) = f \sb j / s \sb j 
\end{eqnarray}
\noindent
is the hazard probability for interval $I \sb j .$


Assuming that the $i$th sampled unit's 
capillary glucose value $A$ 
belongs to interval $I \sb {i \sb j},$ 
the contribution to the likelihood of the $i$th sampled unit is
\begin{eqnarray*}
L \sb i
=
f \sb {i \sb j}
=
\pi \sb {i \sb j} s \sb {i \sb j}
=
\pi \sb {i \sb j}  \Pi \sb {j=1} \sp {{i \sb j}-1} (1 - \pi \sb j).
\end{eqnarray*}
\noindent
Letting
$
w \sb {ij} =
     1 $ if $A \in I \sb j$ for unit $i$
     and
     $ = 0, $ otherwise,
the likelihood over all sampled units is
\begin{eqnarray}
L 
& = &
\Pi \sb {i=1} \sp n L \sb i
\nonumber
\\
& = &
\Pi \sb {i=1} \sp n 
     \Pi \sb {j=1} \sp {i \sb j}
     \left[
     \pi \sb j  \sp {w \sb {ij} }
      (1 - \pi \sb j) \sp {1- w \sb {ij}}
     \right].
\label{eq:like0}
\end{eqnarray}
\noindent
More generally, letting the hazard for interval $I \sb j$
depend upon characteristics of the $i$th unit (\ref{eq:like0})
is
\begin{eqnarray}
L 
=
\Pi \sb {i=1} \sp n 
     \Pi \sb {j=1} \sp {i \sb j}
     \left[
     \pi \sb {ij}  \sp {w \sb {ij} }
      (1 - \pi \sb {ij}) \sp {1- w \sb {ij}}
     \right].
\label{eq:like1}
\end{eqnarray}    
\noindent
Equation (\ref{eq:like1})
may be regarded as
the product of $\sum i \sb j$ independent Bernoulli densities.
In the parlance of generalized linear models,
the random component of
each density is $ w \sb {ij}$
having a Bernoulli distribution
with
``success'' probability $\pi \sb {ij}. $

To complete the specification 
of the generalized linear model
for  ordered categorical data,
let
\begin{eqnarray}
\label{eq:lp}
\eta \sb {ij}
=
\theta \sb j + \beta \sp T x \sb i
\end{eqnarray}
\noindent
denote the linear predictor corresponding to $w \sb {ij}$ where
$x \sb i = ( x \sb {i1} , \ldots , x \sb {ip} )$
denotes a vector of $p$ continuous and/or categorical
covariates for the $i$th sampled unit
and
$\{\theta \sb 1 , 
\ldots , 
\theta \sb K , 
\beta \sb 1 , 
\ldots , 
\beta \sb p \}$
is a vector of regression coefficients to be estimated. 
%
The link function 
%$g ( \cdot ) $ 
we use to connect
the linear predictor $\eta \sb {ij}$
to the mean of $E ( w \sb {ij} ) = \pi \sb {ij}$
is the logit link
\begin{eqnarray}
\label{eta}
\log \{ \pi \sb {ij} / ( 1 - \pi \sb {ij} ) \}
%= 
%g( \pi \sb {ij} )
= 
\theta \sb j + \beta \sp T x\sb i .
\end{eqnarray}
\noindent
Alternate choices of link functions include
the probit and complementary log--log links.
%   
%To obtain inferences on the regression parameters,
%$ w \sb {ij}$ may be treated as 
%independent random variables. 
The iteratively reweighted least squares algorithm
implemented in many standard statistical software packages
such as GLIM and S may be used to obtain
maximum likelihood estimates of the regression parameters, standard errors,
and deviance tests.

\subsection{Modeling the ROC curve}


Let
$\{ \hat \pi \sb {x\sp{\prime},j} \}$ 
denote the smoothed hazard  probabilities
obtained by computing the anti--logit of the linear predictor~(\ref{eq:lp})
evaluted at the maximum likelihood estimates of the regression parameters
for specified covariate vector, 
$x \sp {\prime} = 
( x \sb {1} \sp {\prime} , \ldots , x \sb {p} \sp {\prime} ).$
%
Using equations~(\ref{eq:survp}) and (\ref{eq:hazdp})
the corresponding smoothed probability mass functions 
can be computed using 
$\hat f \sb {x \sp{\prime},1} 
=
\hat \pi \sb {x \sp{\prime},1}
$ and the recursion
\begin{eqnarray*}
\hat f \sb {x \sp{\prime},j} 
=
\hat \pi \sb {x \sp{\prime},j}
(
1- \sum \sb {k=1} \sp {j-1}   \hat f \sb {x \sp{\prime},k}
),
\end{eqnarray*}
\noindent
$j=2, \ldots , K.$

Let
\[
x \sb {1} \sp {\prime} =
\left\{
\begin{array}{ll}
1 & \mbox{if the gold standard test is positive, and} \\
0 & \mbox{otherwise.}
\end{array}
\right.
\]
\noindent
In the absence of covariates other than the outcome of the
gold standard test, the number of covariates is $p=1.$  
Using~(\ref{eq:survp})
the maximum likelihood estimate of the sensitivity 
of the diagnostic test 
at $a \sb {j-1}$
is
\begin{eqnarray}
\label{eq:sen}
\hat s \sb {x \sp{\prime} , j }
=
 \sum \sb {k=j} \sp K \hat f \sb {x \sp{\prime} , k }   
\end{eqnarray}
\noindent
when  $x  \sp {\prime} \sb 1 = 1.$
Also, 
the maximum likelihood estimate of the specificity 
of the diagnostic test 
at $a \sb {j-1}$
is
\begin{eqnarray}
\label{eq:spec}
1-\hat s \sb {x \sp{\prime} , j }
\end{eqnarray}
\noindent
when  $x  \sp {\prime} \sb 1 = 0.$
The smoothed version of the ROC curve may be obtained
by plotting 
(\ref{eq:sen}) and the complement of (\ref{eq:spec})
for each
cut point $a \sb j,$ $j=1, \ldots, K.$

When, 
in addition to the outcome of the gold standard test,
there are additional covariates available (i.e., $p>1$),
equations (\ref{eq:sen}) and (\ref{eq:spec})
provide estimates
of sensitivity and specificity 
at designated values of all of the covariates.
Confidence intervals for the adjusted values of sensitivity and specificity
may be conveniently obtained 
using the bootstrap percentile method  (Efron 1982).

\section*{References}

\medskip
\noindent
Coughlin, S.S., Trock, B., Criqui, M.H., Pickle, L.W., Browner, D.,
and Tefft, M.C. (1992).
The logistic modiling of sensitivity, specificity, and predictive value
of a diagnostic test.
{\sl Journal of Clinical Epidemiology,}
{\bf 45}(1), 1--7.

\medskip
\noindent
Efron, B. (1982).
{\sl The Jackknife, Bootstrap, and other Resampling Plans.}
The Society for Industrial and Applied Mathematics:
Philadelphia.


\medskip
\noindent
Metz, C.E. (1978).
Basic principles of ROC analysis.
{\sl Seminars in Nuclear Medicine,}
{ \bf 8}(4), 283--298.

\medskip
\noindent
Tosteson, A.N. and Begg, C.B.B. (1988)
A general regression methodology for ROC curve estimation.
{\sl Medical Decision Making,}
{\bf 8}(3), 204--215.


\medskip
\noindent
World Health Organization (1985).
{\sl Diabetes Melitus,
Technical Report Series 727.}
World Health Organization: Geneva
 

\end{document}
\end{input}






\left\{ 
     \Lambda ( a \sb {j} ) 
     -
     \Lambda ( a \sb {j-1} ) 
\right\} .
$

