| Expert judgments are frequently used in Probabilistic Safety Assessments (PSA). However, the methods employed in practice are very crude and a large gap exists between the theoretical methods available and actual practice. A taxonomy of issues related to the use of expert judgments in PSA was considered necessary to identify the needs of the practitioners and the applicability of existing models. A taxonomy is proposed and reviewed with examples from several case studies. Issues can be classified into two categories--(a) elicitation, and (b) the use of expert judgments. Various elements of these categories, such as model and parameter uncertainty, decomposition, the use of multiple experts, expert selection, training, calibration, elicitation, effect of evidence, opinion aggregation and dependence amongst experts are discussed. Sources of expert bias and dependence are discussed using examples from selected case studies. Guidance on the use of the Bayesian judgment aggregation model is provided by conducting a sensitivity analysis on the Bayesian aggregation model. Experts' weights and posterior standard deviation are found to be most sensitive to experts' standard deviations and not as sensitive to inter-expert dependence. Experts' location biases influence the posterior mean, but not the posterior variance and weights. The results are illustrated with the help of two examples and indicate that the Bayesian aggregation model is a suitable candidate for aggregating expert judgments, especially if there is phenomenological uncertainty. Phenomenological uncertainty can be represented through the dependence parameter of the Bayesian model. This is because the sharing of assumptions by the experts tends to introduce dependence between them. The results show that ignoring dependence can lead to an underestimation of uncertainty. On recognition of the fact that the experts in the second example are highly dependent on a common information source, it is assumed that the common information source is the actual expert and the participants are assessing its biases and credibility. The results thus obtained are compared with those obtained by a conventional application of the Bayesian model. The results lend validity to the use of weighted averaging schemes, because the Bayesian aggregation method encompasses arithmetic and geometric averaging techniques. |
