An Improved Rank Order Centroid Method (IROC) for Criteria Weight Estimation: An Application in the Engine/Vehicle Selection Problem

2.1The Proposed Idea

Definitely, each feasible point in the weight space is a solution to assign the weights to the ranked criteria. Among these solutions, the defining vertices of the convex polyhedral of the weight space can be considered as Vertex Methods (VM) which are (1,0,…,0),(12,12,0,…,0),…, and (1n,1n,…,1n). The coordinates of the weight space centroid (i.e. the ROC weights) are calculated by ordinary averaging the corresponding coordinates of the VMs. As a matter of fact, as shown in equation (2), the ROC method is a convex linear combination of the VMs in which all the coefficients equal 1/n.

2.2Determining the IROC Coefficients

Two approaches can be used to obtain the coefficients φjn (j=1,2,…,n) in equation (3). An idea in point is the extraction of the coefficients from the DM’s perspectives, i.e. a subjective approach. Another notion, the focus of this paper, is to determine default coefficients, i.e. an objective approach. This approach is useful in cases where the DM has no idea or consent to propose his/her preferences on the VMs, lack of enough time, etc.

In order to estimate the coefficients, a set of systematic simulation experiments was performed, with regard to MADM problem. MADM problem refers to selecting the most appropriate candidate among m predetermined alternatives or prioritizing them in the presence of usually conflicting n criteria (Hwang and Yoon, 1981; Hatefi, 2021). Generally, a MADM problem is shown by matrix [aij]m×n in which aij (i=1,…,m; j=1,…,n) is called performance score of ith alternative with respect to jth criterion. In the simulation, we use the Multi-Attribute Additive Value (MAV) function as the evaluation index to calculate the aggregated value of ith alternative. This function, with equation (4), is widely used as the underlying analysis model to calculate the overall value of the alternatives (Danielson and Ekenberg, 2016). In the MAV function, it is assumed that sum of all weights ωj equals one and 0⩽aij⩽1 (Keeney and Raiffa, 1993). If we use the weights produced by a given method in the MAV function to select the best alternative or to prioritize the alternatives, let us call the result decision made by that given method.

The systematic simulation study was firstly proposed by Barron and Barrett (1996), and is a broadly accepted framework to address the performance of any approximate weighting method. Many investigations have employed such a simulation study, such as Hatefi (2019), Ahn (2017), and Ahn and Park (2008a). According to the basic notion of this approach, there exists a set of true weights as the reference weights in the DM’s mind which are not accessible in its pure form by any elicitation method. The decision made by the true weights is called true decision. The idea is to generate the weights by the method to be examined (herein the IROC method) as well as the true weights from an underlying random distribution and address how well the decision made by the method match the true decision in terms of a given efficacy measure. To this end, Hit Ratio (HR) and Rank order Correlation (RC) have been widely used as efficacy measures. The HR evaluates how frequently a method selects the same best alternative as the true weights. Equation (5) presents the HR function for a given method in which π is the total number of simulation runs, and γ is the number of simulation runs in which the method selects the same best alternative as the true weights do. The HR ranges from 0 to 1, in the way that 1 means the best alternative of the two rank orders are the same, throughout whole simulation runs. The RC indicates the similarity of the overall rank structures of the alternatives made by the true weights and by the method. This measure is calculated by Kendall’s formula as equation (6) (Winkler and Hays, 1985). In this function, m is the number of alternatives, and θ is the number of pairwise preference violations between the rank structures of the alternatives by the method and by the true weights. Obviously, the values range from −1 to 1 for the RC, the value 1 stands for perfect correspondence between the two rank orders.

The simulation was designed with four levels of the alternatives (m=3,5,7,10) and twenty four levels of the criteria (n=2,3,…,25). For each combination of the number of alternatives and the number of criteria (96 combinations), the following procedure was repeated N=15000 times.

Step 1: Set π=1. In addition, lets γ=0 and θ=0 for the method to be examined.

Step 2: Generate a normalized random MADM matrix: Firstly, random performance scores aij are generated from independent uniform distribution on interval (0,1). These scores constitute an m×n MADM matrix. Secondly, the performance scores in each column are normalized by equation (7). In this equation, the values ajmax and ajmin are the maximum and minimum scores in column Cj:

Step 3: Generate criteria true weights: Firstly, n−1 random numbers are generated from independent uniform distribution on (0,1). It should be noted that assuming the uniform distribution represents the DM’s uncertainty about the weights. Secondly, the generated random numbers are sorted in ascending order and named as u1,u2,…,un−1. The differences between adjacent numbers in sequence 1>un−1⩾un−2⩾⋯⩾u1>0 are (1−un−1),(un−1−un−2),…,(u1−0). Thirdly, these differences are ordered by size in descending. The outcome would be the true weights which are uniformly distributed on the weight space.

Step 4: Compute the criteria weights by the method.

Step 5: Determine the corresponding ranks of the alternatives: For the true weights and for the method, the MAV of each alternative are calculated using the MADM matrix generated in Step 2 and the weights achieved in Steps 3 and 4. Next, the alternative with the biggest MAV is placed at the first rank, one with the second biggest at the next rank, and so on.

Step 6: Compare the ranks of the alternatives by the method and by the true weights. If the method selects the identical alternative at the first rank as the true weights, then set γ=γ+1. Moreover, compare the overall rank structure of alternatives constructed by the method and by the true weights, and for any violation between these structures set θ=θ+1.

Step 7: If π=N, then go to Step 8, else set π=π+1 and go to Step 2.

Step 8: Use equations (5) and (6) to calculate the overall value of the HR and RC for the method. This is the stop point of the run.

The simulation experiment was conducted with the use of a Visual Basic for application in the Excel programming language on a personal computer. The simulation runs (i.e. 15000 times) were made in five rounds. Finally, the averages HR and RC of 5 rounds were considered. Calculation of the Pearson’s correlation coefficients between the HR and RC data for 96 combinations showed that the performance values for the two efficacy measures HR and RC were highly correlated, with overall average of 0.9815. Hence, we employ only the HR to derive the coefficients. We chose the HR because it is easier to understand and simpler to handle.

To calculate the coefficients of the VMs, this procedure was done: First, for each combination of the number of alternatives and the number of criteria, the HR values are normalized to add up to 1. As an instance, in combination with m=3 and n=4, four HR values were obtained: 0.76933, 0.76353, 0.75947, and 0.70593 for the vertices (1,0,0,0), (12,12,0,0), (13,13,13,0), and (14,14,14,14), respectively. Their normalized values are 0.25659, 0.25466, 0.25330, and 0.23545. Second, we know that for a given n, there are four sets of normalized HR values according to different levels of alternatives which are 3, 5, 7, and 10. The study showed that the trend of the four sets is fairly similar for any number of the criteria. Figure 1 depicts these trends for some number of the criteria.

Fig. 1

Typical trends of the normalized HR at various VMs.

Correspondingly, geometric mean was then used over the combinations with identical number of the criteria. By way of example, in combinations 3×4, 5×4, 7×4, and 10×4 (all with 4 criteria), the normalized HR vectors are (0.25659,0.25466,0.25330,0.23545), (0.24035,0.26781,0.25828,0.23356), (0.22565,0.27585,0.26416,0.23434), and (0.20865, 0.27677, 0.27549, 0.23908), respectively; and the geometric means would be (0.23213, 0.26862, 0.26268, and 0.23560). Finally, to normalize the coefficients to unity, each geometric mean was divided by sum of the geometric means. Given the above-mentioned example in relation to 4 criteria, sum of the geometric means is 0.99903, thus the default coefficients would be φ14=0.23236, φ24=0.26888, φ34=0.26293, and φ44=0.23583 for (1,0,0,0), (12,12,0,0), (13,13,13,0), and (14,14,14,14), respectively. Table 1 presents the obtained coefficients for n=2 to n=25.

The coefficients in the IROC function have been portrayed in Fig. 2. As a major consequence from this figure, we can conclude that the curves are not uniform. Moreover, each set of the coefficients tends to follow a concave shape with the maximum at point VM⌈n/2⌉ or a near point for larger number of the criteria. For example, the maximum is at VM5 for n=10 and at VM9 for n=21. Additionally, most the curves present a little skewed to the right distribution.

Fig. 2

Variations of the IROC coefficients for different number of the criteria.

2.3Comparison

In this section, we report a set of simulation experiments that was conducted with the purpose of comparing the behaviour of the ROC method versus the IROC method. All the characteristics of this simulation scheme were like the previous simulation experiments (described in Section 2.2), unless: (I) the two methods ROC and IROC were considered to be tested simultaneously, and (II) four levels of the alternatives (m=3,5,7,10) and five levels of the criteria (n=3,5,7,10,15) are considered.

Table 2 depicts the efficacy measures data obtained from the experiments. To sum up, throughout the simulation results, we can conclude that the IROC method appears to be a better performer than the ROC method as expected. In respect to the HR, the data indicates that the IROC method outperforms the ROC method over 17 out of 20 (= 85%) cases. Among these 17 cases, in 14 cases the numerical data for the IROC method and ROC method differ only in the third decimal place, and in 3 cases (3×5, 5×15, and 10×10) the differences are even in the second decimal place. The same is true of the mean values of the HR (0.85924 for the ROC method versus 0.86026 for the IROC method). As regards the RC measure, like the HR, the IROC method is superior to the ROC method over 17 out of 20 combinations. Interestingly, in 6 cases out of these 17 combinations, the differences between the IROC method and ROC method data are even about 0.01 which is considerable in turn. In addition, the mean row of the table shows that the use of the IROC method leads to a mean RC of 0.46230, while the use of the ROC method yields a mean RC of 0.46488. In the table, the columns entitled improvement (%) are subtracting the ROC method performance value from the IROC method performance value, divided by the ROC method performance value. From this point of view, the numbers indicate an improvement of the IROC method over the ROC method up to about 1%.

Even though Table 2 obviously shows the superiority of the IROC method over the ROC method; two tests as equation (8) and equation (9) are built, the former to compare the ROC HR population mean and the IROC HR population mean, and the latter o compare the ROC RC population mean and the IROC RC population mean.

Table 2 shows that the data as for the ROC HR/RC minus the IROC HR/RC are paired. In fact, there are two samples in which each observation in one sample is paired with one observation in another sample. Hence, firstly, we employ the Shapiro–Wilk tests (Shapiro and Wilk, 1965), as seen in equation (10) and equation (11), to survey whether the HR/RC differences are normally distributed.

In equation (10), the test statistic equals 0.9563, and the Shapiro–Wilk critical value using 99% confidence is 0.868. Because 0.9563>0.868, we conclude that the HR differences are normally distributed. In equation (11), to check the normality of the RC difference data, the Shapiro–Wilk statistic is 0.9506 that is greater than 0.868, thus at a 99% of confidence the RC differences are normally distributed.

Both the HR differences and the RC differences are normally distributed. Thus, the one-way paired t-student test is applied for the tests in equation (8) and equation (9). For equation (8), the t-student test statistic is calculated as −4.0986, and the critical range at 99% confidence level is T<−2.539. Because −4.0986 < −1.729, we reject null hypothesis in equation (8), and deduce that there is an absolutely significant difference between the two populations. In fact, the HR values in the ROC method are significantly less than that of in the IROC method. For equation (9), the statistic is equal to −5.1884, thus because −5.1884 < −1.729 we reject null hypothesis and put our trust in this fact that the IROC RC averages are significantly greater than the ROC RC averages.

The ROC and IROC weights for n=2 to 15 are displayed in Table 3. The difference percentage between each two given corresponding weights is calculated as subtracting the ROC weight from the IROC weight divided by the ROC weight. Figure 3 shows how the difference percentage varies as the rank of criteria increases. For illustration, in case with 3 criteria, the ROC weights are 0.6111, 0.2778, and 0.1111, the IROC weights are 0.6086, 0.2845, and 0.1069, and the difference percentages are (0.6086−0.6111)/0.6111=−0.40%, +2.42% and −3.83%, respectively. The curves in the figure disclose a decrease from the ROC weight to the IROC weight for the criterion at the first rank, increase from the ROC weight to the IROC weight for some criteria at the middle ranks, and decrease from the ROC weight to the IROC weight for the criteria at the tail end ranks.

Fig. 3

Variation of the difference percentage between the ROC weights and the IROC weights.

4.1A Brief Introduction

Public transport development often requires participatory decision making procedures (Duleba et al., 2021). One of the major decision making context is energy management. Energy is the fundamental and essential core of the public transport in countries. Experts have estimated that the global need for energy may rise by more than 50% up to 2030 (Singh et al., 2018). The Energy Information Administration (EIA) outlook report 2020 shows that the public transport accounts for about 25% of all energy consumption in the world. Today, countries are faced with several technologies for their public transport vehicles. These technologies, among others, are (Sperling, 1995; Morita, 2003; Tzeng et al., 2005; Patil et al., 2010; Mousaei and Hatefi, 2015; Erdogan et al., 2019; Rani and Mishra, 2020; Andersson et al., 2020; Cui et al., 2022; Abbasi and Hadji-Hosseinlou, 2022):

From the above list, some kind, e.g., conventional diesel engine/vehicle, are based on burning fossil fuels (Bhan et al., 2022), which generates carbon dioxide and other air pollutants such as unburned hydrocarbons and oxides of nitrogen, resulting in global warming and climate unwelcome changes. On the contrary, the modern technologies, e.g., exchangeable-battery electric engine/vehicle, have cleaner engines, which do not use fossil resources. Governments always need to choose the proper engine/vehicle technology to invest in and to develop in their public transport network. This challenge is often modelled as a MADM problem. In this regard, governments often have to respond the two following preliminary important questions:

The proposed methodology (explained in Section 3) is employed to answer the above questions. There are a number of researches related to the current case, most of them have determined list of the related criteria. Let’s review some samples. Poh and Ang (1999) used Analytic Hierarchy Process (AHP) in order to evaluate the transportation fuels in Singapore. Winebrake and Creswick (2003) employed the AHP method to analyse the outlook of hydrogen-based engines for transportation systems. Tzeng et al. (2005) is a seminal work in the field of the current study case. They used Technique for Order Preference by Similarity to Ideal Solution (TOPSIS) for the sake of determining the best alternative fuel buses compatible with urban area circumstances. Patil et al. (2010) developed a framework to model the interactions between different aspects of a transportation system, and showed up the strategies which affect decision making about engines/fuels with regard to public transport. For fuel selection in public transport, a fuzzy decision making framework was developed by Vahdani et al. (2011). Scott et al. (2012) carried out a review of those academic investigations attempting to deal with issues arising within the bioenergy, using MCDM techniques. Asilata and Keswani (2015) addressed a systematic analysis for selection of fuel by using the AHP method. Shah et al. (2017) presented an overview of available liquid and gaseous fuel, commonly used as transportation fuel in Bangladesh, and illustrated the potential of bio-CNG conversion from biogas. Oztaysi et al. (2017) concentrated on the alternative fuel selection problem of a company in the USA. They developed a multi-expert MCDM technique using Interval-Valued Intuitionistic Fuzzy Sets (IVIFS) with linguistic data. Erdogan and Sayin (2018) performed a study to choose the best fuel for the compression ignition engine. They employed the SWARA method to determine the criteria weights, and used Multi-Objective Optimization on the basis of Ratio Analysis (MULTIMOORA) to rank the selected fuels. Erdogan et al. (2019) used hybrid models SWARA-MOORA and ANP-MOORA to select the optimum fuel for the compression ignition engine/vehicle. Karasan and Kahraman (2020) made use of Interval-Valued Neutrosophic (IVN) ELECTRE I method to select renewable energy alternative for a municipality. Rani and Mishra (2020) proposed a novel decision making model based on the operators of q-Rung Ortho-Pair Fuzzy Sets (q-ROFSs), weighted aggregated sum product model, score function and similarity measure to deal with the alternative-fuel technology selection problem, wherein the decision experts and the criteria weights were completely unknown. Andersson et al. (2020) evaluated which criteria have an influence on the fuel choice between ethanol and gasoline for owners of Flex-Fuel Vehicles (FFVs) in Sweden. Major results showed that price, perceptions about quality, age and environmental attitudes influence the willingness to choose ethanol.

4.2The Criteria List

This section reports findings of the criteria identification in the engine/vehicle selection problem. A complete criteria list is founded based upon both the published literature and the expert’s judgments. We did the best to extract all the criteria reported in the relevant literature, among others, Poh and Ang (1999), Winebrake and Creswick (2003), Tzeng et al. (2005), Patil et al. (2010), Vahdani et al. (2011), Scott et al. (2012), Farkas (2014), Mousaei and Hatefi (2015), Asilata and Keswani (2015), Shah et al. (2017), Oztaysi et al. (2017), Hatefi (2018), Erdogan and Sayin (2018), Erdogan et al. (2019), Karasan and Kahraman (2020), and Rani and Mishra (2020). After that, a Delphi evaluation, using 9 related participants who were experts in the field of various engines/vehicles, was preformed to reach consensus on the criteria. In each round of the process, the respondents had to answer the questions to refine the criteria, i.e. to screen, to add, to combine, or to decompose them. We made use of the advantage of being performed by email in the Delphi process. The type of attendance meeting was not selected for the reason of Covid-19 conditions. In Table 5, let’s review the final list of the criteria obtained from the above-mentioned process. This list includes main criteria (1st level) and sub-criteria (2nd level).

4.3The Criteria Weights

A sample country is considered to be analysed using the proposed methodology. Regarding all the criteria displayed in Table 5, a decision making group including 9 experts (E=9) were prioritized the 1st level criteria (n=8), and 2nd level criteria (n=4,4,6,2,2,3,2, and 3, respectively) individually. As example, Table 6 shows the ranks for the 1st level criteria. The Kendall’s coefficients were at strong or higher level for all of the ranks. Table 7 represents the final findings. In this table, the final weight is computed by multiplying the two related columns, for instance, the weight of energy supply (= 0.0914) is obtained by 0.3251×0.2810.