% Handout on Bayesian Software for RSS Workshop, 11-May-1993.
% Apologies to everyone who thinks their work is under-represented
%  or mis-represented.
% Apologies also for various \TeX hacks - I left my TeXbook at work.
% Free distribution is encouraged; comments are welcome.
%	-- Ewart Shaw (e-mail: strgh@uk.ac.warwick)
%
\font\cscfnt=cmcsc10 at 10pt
\font\mib=cmmib10 at 10pt
\def\caps{\cscfnt}
\def\thetav{{\lower.1ex\hbox{\mib\char'022}}}
\def\xv{{\bf x}}
\def\yv{{\bf y}}
\def\nl{\hfill\break}
\def\sect#1{\goodbreak\vskip20pt\centerline{\bf #1}}
%
\def\AMSTAT{{\sl The American Statistician}}
\def\ANS{{\sl Ann.\ Statist.}}
\def\APS{{\sl Appl.\ Statist.}}
\def\BKA{{\sl Biometrika}\/}
\def\BSfour{{\sl Bayesian Statistics 4}}
\def\BSthree{{\sl Bayesian Statistics 3}}
\def\CSTM{{\sl Commun.\ Statist.--Theor.\ Meth.}}
\def\ECA{{\sl Econometrica}}
\def\JAEX{{\sl J.~Appl.\ Econometrics}\/}
\def\JASA{{\sl J.~Amer.\ Statist.\ Assoc.}}
\def\JEX{{\sl J.~Econometrics}}
\def\JIMA{{\sl J.~Inst.\ Math.\ Appl.}}
\def\JRSSB{{\sl J.~Roy.\ Statist.\ Soc.~B}}
\def\PTRSLA{{\sl Phil.\ Trans.\ R. Soc.\ Lond.~A}}
\def\SC{{\sl Statist.\ Comput.}}
\def\STAT{{\sl The Statistician}\/}
%
\parindent=0pt
\parskip=8pt plus2pt minus1pt
%
\centerline{\bf STATISTICAL COMPUTING for BAYESIAN APPLICATIONS}
\centerline{(Handout for R.S.S.\ One-day Workshop, 11--May--1993)}
\vskip10pt
\centerline{\bf J. E. H. Shaw, University of Warwick}

\sect{1. Background}
Practical Bayesian analysis has become increasingly used in
many different fields
(e.g.~[1, 5, 22, 38, 61]).
This talk concentrates on computation for standard statistical problems
like regression (linear or nonlinear),
time series etc.\ rather than areas like image analysis [16]
which may have millions of unknown quantities.

Bayesian inference handles {\sl all} uncertainty and unknown quantities
-- whether potential data $\xv$ or model parameter
$\thetav = (\theta_1,\theta_2,\dots,\theta_d)$ --
via probability.
Once actual data $\xv$ have been obtained,
inferences are based on the posterior distribution
$$
p(\thetav|\xv) = c\,l(\thetav;\xv)\,p(\thetav),\eqno(1.1),
$$
where $l(\thetav;\xv)=p(\xv|\thetav)$ is the likelihood,
$p(\thetav)$ is the prior,
and $c = 1/\int l(\thetav;\xv) p(\thetav) d\thetav$
is the normalising constant that makes the right hand side
integrate to 1.

As described by Naylor \& Smith [32],
many quantities \& functions useful for summarisation, inference,
prediction, decision-making etc.~involve integrals of the form
$$
\eqalignno{
S(q) &= \int_\Theta q(\thetav) l(\thetav;\xv) p(\thetav) d\thetav&{(1.2)}\cr
\noalign{\hbox{or, more generally,}}
S_I(q) &= \int_{\Theta_{I'}}
 q(\thetav) l(\thetav;\xv) p(\thetav) d\thetav_{I'},&{(1.3)}\cr
}
$$
where $I\subseteq\lbrace 1,\dots,d\rbrace$
is a set of subscripts of interest,
and $d\thetav_{I'}$ denotes integrating over all $\theta_i$
for $i\not\in I$.

For example: the normalising constant is $c = 1/S(1)$,
the posterior marginal density of $\thetav_I$ is
$$
\eqalignno{
p(\thetav_I|\xv) &= S_I(1)/S(1),&{(1.4)}\cr
\noalign{\hbox{and posterior expectations are given by}}
E[q(\thetav)|\xv] &= S(q)/S(1).&{(1.5)}\cr
}
$$
In particular, $q(\thetav)=\theta_i$, $\theta_i^2$, $\theta_i\theta_j$
etc.~give the posterior moments,
$q(\thetav) = p(y|\thetav)$ gives the predictive distribution
for future data $y$,
and $q(\thetav) = f^{-1}(\theta_i)$ gives the posterior mean of
an original parameter $\phi_i$ that had been reparametrised
to $\theta_i = f(\phi_i)$.

\sect{2. Methods}
{\bf Bayes linear estimates} \& second order methods (e.g.~[24, 36, 65])
explore the means, variances \& covariances of the unknown quantities,
and highlight the structure of one's beliefs and information.
%See comments on [B/D] in section 4.

{\bf Conjugate analysis}
-- simple likelihoods with conventional/reference priors
(e.g.~[35, 23, 48]) --
can often be analysed using standard statistical packages [13, 48],
and are invaluable for teaching and illustration,
but are rather restrictive for general statistical modelling.

If we want to analyse posterior distributions arising from
arbitrary likelihoods and priors,
then we must use analytic or numerical approximations.
Fortunately, the central limit theorem implies that in many cases
$p(\thetav|\xv)$ is approximately Normal.
More generally, for smooth functions $q$
(in particular, $q=1, \theta_i, \theta_i^2, \theta_i\theta_j$),
the integrands in (1.2) \& (1.3) can be well approximated
by a Normal `kernel' multiplied by a low-order polynomial in $\thetav$.

{\bf Asymptotic methods} fit a series expansion
to the posterior density, typically about its mode.
In particular, {\bf Tierney-Kadane} approximations [28, 54--58]
estimate $S(q)$ by integrating analytically the Normal curve
% HELP: IS THERE AN OFFICIAL NAME FOR AN UNNORMALISED NORMAL??
fitted to the mode and Hessian at the mode
of the corresponding integrand.
The leading error terms in the numerator and denominator of (1.4)
or (1.5) then cancel, making the approximations very accurate
for large data sets.

{\bf Numerical integration} [11, 32, 33, 38, 40--44]
estimates $S(q)$ by forming a weighted average of the
integrand evaluated at carefully-chosen points (`nodes'):
$$
\hat S(q) =
 \sum_{i=1}^n w_i\,q(\thetav_i)\,l(\thetav_i;\xv)\,p(\thetav_i).
\eqno{(2.1)}
$$
The nodes $\thetav_i$ and weights $w_i$ together constitute a
{\sl numerical integration rule}.
Iterative adjustment for the estimated posterior means and
variance-covariance matrix,
as in the Naylor-Smith algorithm [32],
implies that we're particularly interested in rules that are exact for
a multivariate standard Normal kernel multiplied by any multivariate
polynomial of degree $\le k$, say, i.e.\ rules of {\sl degree $k$}.

Direct {\bf simulation} from the posterior
is sometimes possible (e.g.~[17]),
giving a natural statistical estimate of $E[q(\thetav)|\xv]$
as the observed average value of $q(\thetav)$ in the simulation.

{\bf Importance sampling} and other {\bf Monte Carlo} methods
[1, 17--19, 33, 59] also use simulation to estimate the desired integrals;
generating nodes $\thetav_i$ from some probability density $g(\thetav)$,
and calculating (2.1) with $w_i = 1/ng(\thetav_i)$.
The resulting estimate $\hat S(q)$ is unbiased and will have
small variance if $g(\thetav)$ has a similar shape to
$|q(\thetav) l(\thetav;\xv) p(\thetav)|$.

There has recently been an explosion of interest
in {\bf Markov chain} methods like {\bf Gibbs sampling}
[5, 8, 12, 14--16, 20--22, 26, 46, 47, 60, and many papers in 4].
These iterative methods aim to end up with an approximate sample from the
posterior distribution, and are generally very easy to program.
Related sampling approximations are outlined in [46],
see also [37].

More detailed overviews of the above approaches are contained in
[45] \& [52].

\sect{3. Problems}
Most approximate methods are unreliable if the
posterior density is highly non-Normal, with features like multimodality
or banana-shaped contours, heavy tails, high skewness etc.

Reparametrisation is often important [27, 41],
but is regarded by many potential users as a black art.

Asymptotic methods usually entail numerical differentiation,
a notoriously unstable procedure in high dimensions,
and only some posterior expectations can be estimated accurately:
$q(\thetav)$ must typically have negligible probability
of being negative.

The `Curse of Dimensionality': the problems multiply,
and the amount of computation often increases unacceptably,
as the number of parameters increases
(particularly for numerical integration).
Several families of efficient numerical integration rules become
unstable in high dimensions, with negative weights [51].

There are theoretical problems with Monte Carlo [34],
partially overcome by using quasirandom sequences [42].
A more practical difficulty with importance sampling
is that the weights are unbounded,
and (particularly when iterating in high dimensions)
the whole process can blow up.
Also, an appropriate generating distribution for estimating (say)
$S(1)$ may be inappropriate for estimating other $S(q)$'s.

Gibbs sampling demands knowledge of various conditional distributions.
Also, convergence properties of Markov chain methods
aren't (yet) clear.
Even after convergence,
a sample from the posterior distribution may be inadequate for
estimating posterior expectations that are sensitive to tail behaviour
(such as higher-order moments or properties of transformed parameters).
\vfill\eject

\sect{4. Current \& Imminent Software}
See [23] for reviews of early versions of some of the following,
and other software,
for example {\bf BRP} (Bauwens) \& {\bf CADA} (Novick {\sl et al.}).
Apologies to anyone who feels their work is under-represented
or mis-represented.
Thanks to
Stephen Evans,
David Firth,
Jeff Harrison,
Wilfrid Kendall,
Sandy MacRae,
B. Narasimhan,
John Naylor,
Tony O'Hagan,
Ajay Shah,
David Spiegelhalter,
Clark Thomborson,
Luke Tierney,
Hal Varian
\& David Wooff
for recent e-mail and personal conversations.

{\parskip=10pt
{\bf Computer Algebra Packages}.\nl
Packages like Mathematica [64] allow a battery of
numerical \& analytic methods, simulation \& graphics.
See e.g.~[29] \& [30].

{\bf LISP-STAT} (Tierney).\nl
An object-oriented package for general statistical analysis
under Unix or MS-Windows (other versions are also being developed).
Powerful \& free [57], well-supported and well-documented [56].
Comes with some tools for approximate Bayesian analysis
(e.g.~display of marginal densities -- [54]).
Further code is likely to be freely available soon, e.g.~[12].

{\bf SBAYES} (Tierney).\nl
Also free [58]; comprises similar Bayesian tools to those in LISP-STAT,
for use in S [2].

{\bf BART}: `Bayesian Approaches to Randomised Trials'
(Spiegelhalter, Freedman \& Parmar).\nl
S code (preliminary version) for analysing
approximately Normal likelihoods with some standard priors [48].
Send e-mail to {\tt david.spiegelhalter@mrc-bsu.cam.ac.uk}.

{\bf BATS}: `Bayesian Analysis of Time Series'
(West, Harrison \& Pole).\nl
Package for interactive Bayesian forecasting \& dynamic modelling,
see [61] for extensive theory and examples.
An APL version [62] is available for a small administrative charge
(requires a PC with APL*PLUS);
C version is due mid-1993.

{\bf MSB}: `Mathematical Statistics B' (O'Hagan).\nl
Teaching package for analysis of some standard models
(e.g.\ Normal/binomial/Poisson samples with
mixture priors of the corresponding conjugate distributions) [35].
Available from Warwick for small administrative charge;
free distribution is encouraged.
Requires a PC with APL*PLUS.

{\bf [B/D]}: `Beliefs adjusted by Data' (Goldstein \& Wooff).\nl
Implementation [25, 65, 66] of the theory of belief analysis
as developed in [24] \& subsequent papers.
Provides a powerful language \& framework for model development,
and for investigating the consequences of assumptions.
Final version due by Easter 1994.

{\bf BUGS}: `Bayesian inference Using Gibbs Sampling'
(Spiegelhalter, Thomas \& Gilks).\nl
Preliminary version [49, 50, 53] of a very general package
for Gibbs sampling.
Available for Unix and PC's, and can be used in conjunction with S.

{\bf BAYES FOUR} (Naylor \& Shaw).\nl
A library of Fortran-77 routines for numerical integration 
and importance sampling [33].
Available in VAX, Unix \& Atari ST versions,
costs approximately {\it\$}100.
% NOTE: No I don't get any money for it!
User supplies Fortran code for a likelihood, and optionally
specifies a prior, parameter transformations,
desired posterior expectations, etc.
See e.g.~[11, 32, 40--45]
for underlying theory and examples.

% SHAMELESS PLUG ALERT
{\bf BINGO}:
`Bayesian Inference -- Numerical, Graphical \& Other stuff' (Shaw).\nl
Portable APL workspaces,
an initial version will be freely available Real Soon Now.
Features a `multi-kernel' version of the Naylor-Smith algorithm [32],
highly efficient (`designed') numerical integration rules,
device-independent graphics,
and facilities for other approaches.
User supplies APL code for likelihood etc.\ similarly to BAYES FOUR.
See next section for an overview of BINGO's features.\par
}
\vfill\eject

\sect{5. BINGO}
Main features --\par
{\parindent=20pt
\item{(a)}
Parallel (`multi-kernel') generalisation of the Naylor-Smith algorithm [32],
compare [63].
Tends to adjust automatically to multimodality etc.,
reducing the need for sophisticated initial reparametrisation.

\item{(b)}
`Designed' integration rules using ideas from coding \& design theory [7].
All have positive weights and are highly efficient:
the number of nodes needed for degree-7 rules
in $d = 5, 10, 15, 20$ dimensions is 83, 985, 3655, 8521 respectively
(compared to $4^d$ nodes for the rescaled Gauss-Hermite rules
originally used in [32]!)
\item{}
Rule definitions are stored as short text files,
so they can easily be used in other computer packages
(C code to form the nodes \& weights will also be made
publicly available Real Soon Now).
As an example, the following text file defines a 1697-node degree-7 rule
in 12-dimensions, comprising one node at the origin,
24 at radius 1.6567, etc.:
\vskip10pt
%EXAMPLE
{\tt\parskip=0pt\baselineskip=9.5pt
\advance\leftskip by30pt\obeylines\obeyspaces
Name de12
Dim 12
Components 4
\% (1697 nodes, sum of wts.~= 1, degree 7 for Enr2)
\%
\% Component 1
\% (1 node, scale = 0, radius = 0, weight = 0.022607)
Params 1 0
Origin
Scale 0 0 1 33
Weight 9 -1 144 33
\%
\% Component 2
\% (24 nodes, scale = 2.7446, radius = 1.6567, weight = 0.012093)
Params 12 1
All 1 12
Scale -3 1 1 33
Weight 27 5 4608 33
\%
\% Component 3
\% (264 nodes, scale = 30.861, radius = 7.8564, weight = 5.3159E-7)
Params 66 2
All 2 12
Scale 33 5 2 33
Weight 891 -155 1115136 33
\%
\% Component 4
\% (1408 nodes, scale = 1.2602, radius = 2.7498, weight = 0.00048795)
Params 22 6
Blocks
~~~ABCEIL ABCFHJ ABDEGJ ABDFKL ABGHIK ACDEHK ACDFGI ACGJKL ADHIJL AEFGHL
~~~AEFIJK BCDGHL BCDIJK BCEFGK BDEFHI BEHJKL BFGIJL CDEFJL CEGHIJ CFHIKL
~~~DEGIKL DFGHJK                                                        
Scale 99 -1 74 33
Weight 7425 223 17842176 33
\par}
%END EXAMPLE

\item{(c)}
Graphical methods developed from those in (e.g.)
[40, 41, 43, 62],
mainly using tensor-product splines [6] on transformed parameters to
approximate marginal densities etc.\par
}

Other facilities include:
built-in reparametrisations,
Metropolis-Hastings algorithm [26],
quasirandom sequences [42], Tierney-Kadane approximations [54],
function maximisation, importance sampling,
APL\TeX,
and the compact device-independent graphics language WGL
(`Warwick Graphics Language' -- pronounced `wiggle').

BINGO is largely the result of research assisted by grants from
U.C.~Berkeley (1989) and the SERC (GR/E 83054: 1989--92).
\vfill\eject

\sect{6. Books}
{\advance\leftskip by20pt
Practical Bayesian statistics: [3, 4].\nl
Statistical computing: [52 -- particularly chapter 5].\nl
Numerical integration: [10, 51].\nl
Simulation: [39].\nl
Splines: [6].\nl
Coding theory \& designs: [7, 9, 31].\nl
\par
}
\sect{7. References}
{%REFS
\parindent=20pt
\parskip=4pt plus1pt minus1pt
%
% polyt, Monte
\item{[1]}
{\caps L. Bauwens}
(1984).
{\sl Bayesian Full Information Analysis of Simultaneous Equation
Models using Integration by Monte Carlo.
Lecture Notes in Econometrics 232.}
Springer-Verlag, New York.
%
% S
\item{[2]}
{\caps R. A. Becker, J. M. Chambers \& A. R. Wilks}
(1988).
{\sl The New S Language: A Programming Environment for Data Analysis
and Graphics}.
Wadsworth and Brooks, Pacific Grove, CA.
%
% general
\item{[3]}
{\caps J. M. Bernardo, M. H. DeGroot, D. V. Lindley \& A. F. M. Smith}
eds.~(1988).
\BSthree.
Clarendon Press, Oxford.
%
% general
\item{[4]}
{\caps J. M. Bernardo, J. O. Berger, A. P. Dawid \& A. F. M. Smith}
eds.~(1992).
\BSfour.
Clarendon Press, Oxford.
%
% gibbs
\item{[5]}
{\caps J. Besag \& P. Green}
(1993).
Spatial statistics \& Bayesian computation.
\JRSSB~{\bf 55}: 25--37 (with discussion: pp.~53--102).
%
% splines
\item{[6]}
{\caps C. de Boor}
(1978).
{\sl A Practical Guide to Splines}.
Springer-Verlag, New York.
%
% codes ref.
\item{[7]}
{\caps P. J. Cameron \& J. H. van Lint}
(1991).
{\sl Designs, Graphs, Codes and their Links}.
London Mathematical Society Student Texts 22.
Cambridge University Press, Cambridge, U.K.
%
% gibbs
\item{[8]}
{\caps G. Casella \& E. I. George}
(1992).
Explaining the Gibbs sampler.
\AMSTAT~{\bf 46}: 167--174.
%
% codes ref.
\item{[9]}
{\caps J. H. Conway \& N. J. A. Sloane}
(1988).
{\sl Sphere Packings, Lattices and Groups}.
Springer-Verlag, New York.
%
% integration ref.
\item{[10]}
{\caps P. J. Davis \& P. Rabinowitz}
(1984).
{\sl Methods of Numerical Integration}, 2nd ed.
Academic Press, Orlando, Florida.
%
% bayes4, integration
\item{[11]}
{\caps P. Dellaportas \& D. E Wright}
(1992).
A numerical integration strategy in Bayesian analysis.
\BSfour\/: 601--606.
%
% gibbs, graphics, xlispstat, statlib
\item{[12]}
{\caps H. J. Doss, B. Narasimhan}
(1993).
Bayesian Poisson regression using the Gibbs sampler:
a case study in dynamic graphics (draft copy).
Department of Statistics, Florida State University.
%
% glim, reference priors
\item{[13]}
{\caps D. Firth}
(1992).
Bias reduction, the Jeffreys prior and GLIM.
pp 91--100 in
{\sl Advances in GLIM and Statistical Modelling},
eds.~Fahrmier, Francis, Gilchrist \& Tutz.
Springer-Verlag, New York
%
% gibbs
\item{[14]}
{\caps A. E. Gelfand \& A. F. M. Smith}
(1990).
Sampling-based approaches to calculating marginal densities.
\JASA~{\bf 85}: 398--409.
%
% gibbs
\item{[15]}
{\caps A. E. Gelfand, S. E. Hills, A. Racine-Poon \& A. F. M. Smith}
(1990).
Illustration of Bayesian inference in normal data models
using Gibbs sampling.
\JASA~{\bf 85}: 972--985.
%
% gibbs, image
\item{[16]}
{\caps S. Geman \& D. Geman}
(1984).
Stochastic relaxation, Gibbs distributions and the Bayesian
restoration of images.
{\sl IEEE Trans.\ Patt.\ Anal.\ and Machine Intelligence}\/~{\bf 6}:
721--741.
%
% simulation from posterior
\item{[17]}
{\caps J. Geweke}
(1986).
Exact inference in the inequality constrained normal linear
regression model.
\JAEX~{\bf 1}: 127--141.
%
% antithetic
\item{[18]}
{\caps J. Geweke}
(1988).
Antithetic acceleration of Monte Carlo integration in Bayesian inference.
\JEX~{\bf 38}: 73--90.
%
% importance sampling
\item{[19]}
{\caps J. Geweke}
(1989).
Bayesian inference in econometric models using Monte Carlo integration.
\ECA~{\bf 57}: 1317--1339.
%
% gibbs
\item{[20]}
{\caps C. J. Geyer \& E. A. Thompson}
(1992).
Constrained Monte Carlo maximum likelihood for dependent data
(with discussion).
\JRSSB~{\bf 54}: 657--699.
%
% gibbs
\item{[21]}
{\caps W. R. Gilks \& P. Wild}
(1992).
Adaptive rejection sampling for Gibbs sampling.
\APS~{\bf 41}: 337--348.
%
% gibbs
\item{[22]}
{\caps W. R. Gilks, D. G. Clayton, D. J. Spiegelhalter, N. G. Best,
A. J. McNeil, L. D. Sharples \& A. J. Kirby}
(1993).
Modelling complexity: applications of Gibbs sampling in Medicine.
\JRSSB~{\bf 55}: 39--52 (with discussion: pp.~53--102).
%
% software review
\item{[23]}
{\caps P. K. Goel}
(1988).
Software for Bayesian analysis: current status and additional needs
(with discussion).
\BSthree\/: 173--188.
%
% b/d
\item{[24]}
{\caps M. Goldstein}
(1981).
Revising previsions: a geometric interpretation (with discussion).
\JRSSB~{\bf 43}: 105--130.
%
% b/d
\item{[25]}
{\caps M. Goldstein}
(1992).
Bayes linear methods I. Adjusting beliefs: concepts and properties.
Technical report, Department of Mathematical Sciences,
University of Durham, U.K.
%
% metropolis-hastings
\item{[26]}
{\caps W. K. Hastings}
(1970).
Monte Carlo sampling methods using Markov chains and their applications.
\BKA~{\bf 57}: 97--109.
%
% parametrisation
\item{[27]}
{\caps S. E. Hills \& A. F. M. Smith}
(1992).
Parametrisation issues in Bayesian inference (with discussion).
\BSfour\/: 227--246.
%
% asymp
\item{[28]}
{\caps R. E. Kass, L. Tierney \& J. B. Kadane}
(1988).
Asymptotics in Bayesian computation (with discussion).
\BSthree\/: 261--278.
%
% mathematika
\item{[29]}
{\caps R. J. Korsan}
(1993).
Decision analytica: an example of Bayesian inference and
decision theory using Mathematika.
Chapter 17 (pp.~344--367) of H. R. Varian (ed.)\ {\sl Economic
 and Financial Modeling with Mathematika}, Springer-Verlag, New York.
%
% mathematika
\item{[30]}
{\caps E. Ley \& M. F. J. Steel}
(1993).
Bayesian econometrics: conjugate analysis and rejection sampling.
Chapter 15 (pp.~407--458) of H. R. Varian (ed.)\ {\sl Economic and
 Financial Modeling with Mathematika}, Springer-Verlag, New York.
%
% coding ref.
\item{[31]}
{\caps F. J. MacWilliams \& N. J. A. Sloane}
(1977).
{\sl The Theory of Error-Correcting Codes}.
North-Holland, Amsterdam.
%
% bayes4
\item{[32]}
{\caps J. C. Naylor \& A. F. M. Smith}
(1982).
Applications of a method for the efficient computation
of posterior distributions.
\APS~{\bf 31}: 214--225.
%
% bayes4
\item{[33]}
{\caps J. C. Naylor \& J. E. H. Shaw}
(1991).
{\sl Bayes Four User Guide}.
Commercial Centre, Trent University, Nottingham, U.K.
%
% Monte
\item{[34]}
{\caps A. O'Hagan}
(1987).
Monte Carlo is fundamentally unsound.
\STAT~{\bf 36}: 247--249.
%
% msb
\item{[35]}
{\caps A. O'Hagan}
(1989).
{\sl MSB -- APL functions for Bayesian statistics}.
Department of Statistics, University of Warwick.
%
% able, ble
\item{[36]}
{\caps A. O'Hagan, E. B. Glennie \& R. E. Beardsall}
(1992).
Subjective modelling and Bayes linear estimation in the UK water
industry.
\APS~{\bf 41}: 563--577.
%
% cf Smith&Gelfand sampling
\item{[37]}
{\caps M. J. D. Powell \& J. Swann}
(1966).
Weighted uniform sampling -- a Monte Carlo technique for reducing
variance.
\JIMA~{\bf 2}: 229--236.
%
% examples
\item{[38]}
{\caps A. Racine, A. P. Grieve, H. Fl\"uhler \& A. F. M. Smith}
(1986).
Bayesian methods in practice: experiences in the pharmaceutical
industry.
\APS~{\bf 35}: 93--150.
%
% simulation ref.
\item{[39]}
{\caps B. D. Ripley}
(1987).
{\sl Stochastic Simulation}.
John Wiley \& Sons, New York.
%
% bayes4, example
\item{[40]}
{\caps J. E. H. Shaw}
(1987)
Numerical Bayesian analysis of some flexible regression models.
\STAT~{\bf 36}: 147--159.
%
% bayes4, graphics
\item{[41]}
{\caps J. E. H. Shaw}
(1988)
Aspects of numerical integration and summarisation (with discussion).
\BSthree\/: 411--428.
%
% quasi
\item{[42]}
{\caps J. E. H. Shaw}
(1988)
A quasirandom approach to integration in Bayesian statistics.
\ANS~{\bf 16}: 895--914.
%
% bayes4, graphics
\item{[43]}
{\caps A. F. M. Smith, A. M. Skene, J. E. H. Shaw, J. C. Naylor
\& M. Dransfield}
(1985).
The implementation of the Bayesian paradigm.
\CSTM~{\bf 14}: 1079--1102.
%
% bayes4, graphics, importance sampling
\item{[44]}
{\caps A. F. M. Smith, A. M. Skene, J. E. H. Shaw \& J. C. Naylor}
(1987).
Progress with numerical and graphical methods for practical
Bayesian statistics.
\STAT~{\bf 36}: 75--82.
%
% review
\item{[45]}
{\caps A. F. M. Smith}
(1991).
Bayesian computational methods.
\PTRSLA~{\bf 337}: 369--386.
%
% sampling, computer-intensive!
\item{[46]}
{\caps A. F. M. Smith \& A. E. Gelfand}
(1992).
Bayesian statistics without tears: a sampling-resampling perspective.
\AMSTAT~{\bf 46}: 84--88.
%
% gibbs
\item{[47]}
{\caps A. F. M. Smith \& G. O. Roberts}
(1993).
Bayesian computation via the Gibbs sampler and related Markov chain
Monte Carlo methods.
\JRSSB~{\bf 55}: 3--23 (with discussion: pp.~53--102).
%
% bart
\item{[48]}
{\caps D. J. Spiegelhalter, L. S. Freedman \& M. Parmar}
(1991).
{\sl BART -- Bayesian Approaches to Randomised Trials}.
MRC Biostatistics Unit, Cambridge, U.K.
%
% bugs
\item{[49]}
{\caps D. J. Spiegelhalter, A. Thomas \& W. Gilks}
(1993).
{\sl BUGS -- Bayesian inference Using Gibbs Sampling}.
MRC Biostatistics Unit, Cambridge, U.K.
%
% bugs
\item{[50]}
{\caps D. J. Spiegelhalter, A. Thomas \& W. Gilks}
(1993).
{\sl BUGS Examples}.
MRC Biostatistics Unit, Cambridge, U.K.
%
% integration ref.
\item{[51]}
{\caps A. H. Stroud}
(1971).
{\sl Approximate Calculation of Multiple Integrals}.
Prentice-Hall, New Jersey.
%
% general ref (esp.chapter 5)
\item{[52]}
{\caps R. A. Thisted}
(1988).
{\sl Elements of Statistical Computing -- Numerical Computation}.
Chapman \& Hall, New York.
%
% gibbs, bugs, S, modula2
\item{[53]}
{\caps A. Thomas, D. J. Spiegelhalter \& W. R. Gilks}
(1992).
BUGS: a program to perform Bayesian inference using Gibbs sampling.
\BSfour\/: 837--842.
%
% asymp
\item{[54]}
{\caps L. Tierney \& J. B. Kadane}
(1986).
Accurate approximations for posterior moments and marginal densities.
\JASA~{\bf 81}: 82--86.
%
% asymp, graphics
\item{[55]}
{\caps L. Tierney, R. E. Kass \& J. B. Kadane}
(1989).
Approximate marginal densities of nonlinear functions.
\BKA~{\bf 76}: 425--433.
%
% lispstat, asymp
\item{[56]}
{\caps L. Tierney}
(1991).
{\sl LISP-STAT, An Object-Oriented Environment for Statistical
 Computing and Dynamic Graphics}.
John Wiley \& Sons, New York.
%
% lispstat
\item{[57]}
{\caps L. Tierney}
(1991).
{\sl XLISP-STAT}.
Available from statlib or ftp from umnstat.stst.umn.edu.
%
% lispstat
\item{[58]}
{\caps L. Tierney}
(1991).
{\sl sbayes}.
Available from statlib or ftp from umnstat.stst.umn.edu.
%
% Monte Carlo
\item{[59]}
{\caps H. K. Van Dijk, T. Kloek \& A. S. Louter}
(1986).
An algorithm for the computation of posterior moments and densities
using simple importance sampling.
\STAT~{\bf 35}: 83--90.
%
% gibbs
\item{[60]}
{\caps J. C. Wakefield, A. E. Gelfand \& A. F. M. Smith}
(1991).
Efficient generation of random variates via the ratio-of-uniforms
method.
\SC~{\bf 1}: 129--133.
%
% bats
\item{[61]}
{\caps M. West \& P. J. Harrison}
(1989).
{\sl Bayesian Forecasting and Dynamic Models}.
Springer-Verlag, New York.
%
% bats, apl
\item{[62]}
{\caps M. West, P. J. Harrison \& A. Pole}
(1991).
{\sl BATS: A User Guide}.
Department of Statistics, University of Warwick, U.K.
%
% cf pns
\item{[63]}
{\caps M. West}
(1993).
Approximating posterior distributions by mixtures.
\JRSSB~{\bf 55}: 409--422.
%
% mathematika
\item{[64]}
{\caps S. Wolfram}
(1991).
{\sl Mathematika, a System for Doing Mathematics by Computer
 (second edition)}.
Addison-Wesley, Reading, Massachusetts.
%
% b/d pascal
\item{[65]}
{\caps D. Wooff}
(1992).
[B/D] works.
\BSfour\/: 851--859.
%
% b/d
\item{[66]}
{\caps D. Wooff}
(1992).
Bayes linear methods II. An example with an introduction to [B/D]. 
Technical report, Department of Mathematical Sciences,
University of Durham, U.K.
\par
}%END REFS
%
\vfill\hskip5in Ewart Shaw, 8--May--93\nl
\null\hskip5in {\tt strgh@uk.ac.warwick}
\bye

