A Sensitivity Analysis of ELECTRE Based Stepwise Benchmarking for Policy: the Case of EU Digital Agenda

Abstract

Research question: This paper introduces a new parameter for evaluating the sensitivity of development paths in ELECTRE (ELimination Et Choix Traduisant la Realité) based stepwise benchmarking model. Motivation: The aim of the research was to devise a 'path sensitivity coefficient' which measures the sensitivity of a development path depending on parameters set by decision-maker (DM). The need for such measure comes from the fact that the hierarchical pre-order of benchmarking units are affected by the DM's subjective decisions on the threshold for declaring outranking relations among them. This consequently may disrupt the sequence of intermediate benchmarks - transitional targets to be followed on the route to the ultimate policy goals. The proposed approach is an extension of a stepwise benchmarking procedure devised by Petrovic et al. (2014) which evaluates development paths based on their gradualism. Idea: In this paper, we propose to characterize development paths by considering both gradualism and sensitivity to outranking thresholds. The standpoint of our research is that in order to make a reliable final choice the DM should be aware that the most gradual path is not necessary the most stable one. Tools: The proposed 'path sensitivity coefficient' combines four level of uncertainty coming from two threshold values set by DM. The coefficient has a relative nature, it points to the “distance” from the theoretically ideal solution - a path fully indifferent to threshold values. An approach to merge coefficients associated with gradualism and stability is also introduced. Data: To illustrate the proposed approach we applied it in the field of digital transformation of EU countries. We exploited indicators and data from Digital Agenda Scoreboard regarding specific policy goals to be achieved until 2020. Findings: The findings imply that the decision on the most gradual development path should be reconsidered in regards to the influence of model parameters. The final choice depends whether DM’s prefers the smoothness of the path or the overall stability in terms of underlying partial preorder.  Contribution: This work contributes to the existing ELECTRE-based stepwise benchmarking by reducing the uncertainty coming from subjective parameterization and allows the DM to be more confident in the final path selection.