Methods and Apparatus for Analytical Processing of Provenance Data for HPC Workflow Optimization

Methods and apparatus are provided for analytical processing of provenance data for High Performance Computing workflow optimization. Prediction models for a workflow composed of a plurality of activities are created by (i) generating a plurality of prediction functions from input features and output features of the workflow, wherein each of the prediction functions predicts at least one output feature of at least one of activities of the workflow based on the input features of at least one activity; and (ii) combining the plurality of prediction functions to generate the prediction models, wherein each of the prediction models predicts a final output feature of the workflow based on an input of the workflow for a given execution plan of the workflow. A plurality of the prediction models can be evaluated to select, among the possible execution plans, an instantiation of the workflow for a given input that optimizes a given user goal.