Integrating data and compute intensive workflows for uncertainty quantification in large scale simulation - Application to model based hazard analysis

Ensemble based simulation methods used in Uncertainty Quantification can often lead to twin computational challenges of managing large amount of data and performing CPU intensive processing. The problem of dealing with large data gets compounded when data warehousing and data mining are intertwined with computationally expensive tasks. We present here an approach to solving this problem by using a mix of hardware suitable for each task in a carefully orchestrated workflow. The computing environment is essentially an integration of Netezza database and high performance cluster. It is based on the simple idea of segregating the data intensive and compute intensive tasks and assigning the right architecture for them. We present here the layout of the computing model and the new computational scheme adopted to generate probabilistic hazard maps.