Polarisation assessment in an intelligent argumentation system using fuzzy clustering algorithm for collaborative decision support

2.1. Polarisation assessment

Polarisation is a phenomenon in which people tend to form groups based on the similarity of the members’ opinions. Sunstein (1999) explained the phenomenon of polarisation as well as its association with both social cascades and social influence. Flache and Michael (2011), present polarisation as

Consolidation, clustering, correlation and continuing diversity are the four group-level phenomena (Latane and Wolf 1981; Latane 1996; Harton, Green, Jackson, and Latane 2000) that a group holds. Clustering here means the formation of polarisation groups within a discussion group. Stakeholders in a cluster are as close as possible and as dissimilar as possible from stakeholders in others. The dynamic social impact theory proposed by Latane (1996) presents three different principles:

Social influence is one of the reasons that stakeholders in a decision-making group both polarise and support one another. In his extended research, Latane identified polarisation groups as dynamic. They change throughout the discussion process as stakeholders change their opinions that quantify (Harton et al. 2000). This dynamic quality was an additional motivation for us to develop a method that quantifies a stakeholder's membership degree within each polarisation group. The dynamic social impact theory states that stakeholders form groups. These groups tend to polarise the stakeholders’ opinions.

The strength between social agents in a network also impacts social influence. Flache and Macy (2011) have conducted research based on Granovetter's (1973) theory of the strength of weak ties. The strength between social agents plays an important role in the formation of polarisation groups. Centola and Michael (2007) presented both the strengths and the weaknesses of long ties. Macy, Kitts, Flache, and Benard (2003) investigated the effect of polarisation in dynamic networks. They also investigated both the dynamics of influence and the attraction between agents in a dialogue process. Walton and Krabbe (1995) characterised dialogue into seven types. Persuasions, negotiation, inquiry, deliberation, information seeking, eristic and mixed are the seven different types of dialogues which are explored in the context of goals of stakeholders in a collaborative setting. The above-mentioned typology is the basis for interactions in groups.

Macy et al. (2003) discovered that the population self-organises into antagonistic groups in a social group. They claim that social agents are attracted to others within the same group. These agents become influenced by others with similar opinions. They are conditioned by both the strength and the valence of social ties. Social agents within the social network can self-organise into antagonistic factions without either the knowledge or intent of the social agents. Takacs (2005) analysed both the network segregation and the intergroup conflicts in a social group. Dense in-group and scarce out-group relations are known as segregation. Segregation in a social group supports the emergence of conflicts between polarisation groups (Takacs 2005). Simpson and Macy (2004) focused on the effects of social identity on the formation of coalitions in a social group.

According to Sillince (1995) and Sillince and Saeedi (1999), several important aspects of face-to-face meetings and discussions such as social, emotional and symbolic requirements, are missing in the current group decision support systems and computer-supported argumentation systems. Polarisation dynamics are part of the social dimension involved in face-to-face meetings and debates (Flache and Macy 2011). Klein (2012a, 2012b) identified the importance of identifying polarisation and balkanisation issues in computer-supported argumentation systems and other social media systems. Our effort is to incorporate this social dimension aspect into computer-supported collaborative argumentation systems. We are not aware of any other existing solutions for computing the degree of stakeholders’ membership within a polarisation group in the domain of computational argumentation.

2.2. Community detection in social networking

Since the advent of social networking sites in early 2000, many researchers have focused on different aspects of social networks. Several scientists have focused on problems such as community detection, information diffusion and more. The community detection problem (Newman 2004; Newman and Girvan 2004; Zhang, Wang, and Zhang 2007) differs from the polarisation assessment problem. Polarisation assessment focuses on how agents with similar opinions come together as a faction. Community is formed based on agents’ social connections.

Zhang et al. (2007) presented methods for identifying communities using both the K-means clustering algorithm (Forgy 1965; Macqueen 1967) and the fuzzy c-means clustering algorithm (Bezdek 1981). Du, Wu, Pei, Wang, and Xu (2007) presented a novel algorithm on the detection of communities in large-scale social networks.

Polarisation analysis discussed here occurs in argumentation systems, while the community detection problem occurs in social networks. Community detection approaches cannot be directly used for argumentation polarisation in argumentation systems since, network structure of argumentation system is different than social networks and community detection is based on social connections and argumentation polarisation is based on the similarity of participants’ opinions. The data collection and pre-processing process involved in the argumentation systems are different. Community detection problem is well investigated in social networks. The polarisation assessment problem, however, is yet to be explored.

2.3. Argumentation systems

Researchers developed many argumentation models and tools (Kunz and Rittel 1970; Conklin and Begeman 1988; Gordon and Karacapilidis 1997; Kraus, Sycara, and Evenchik 1998; Karacapilidis and Papadias 2001; Atkinson, Bench-Capon, and McBurney 2006; Karacapilidis et al. 2009; Vesic, Ianchuk, and Rubtsov 2012). We will review a list of examples of argumentation systems in the context of social interactions for collaborative decision support to make it relevant.

Many existing argumentation models follow either Dung's (1995) abstract model or Toulmin's (1958) informal argumentation model. Formal argumentation models are logically sound though difficult in practice to use. Informal argumentation models are easy to use in practice although they might be less logically sound.

Karacapilidis and Papadias (2001) developed the HERMES system. This system both supports argumentation discourse among stakeholders and works on the World Wide Web platform. It allows stakeholders to post their decision-making issues, candidate solutions and participate in the argumentation process by posting both their views and opinions. Each candidate solution is quantitatively assessed by the arguments that both support and attack that candidate solution. Based on the aggregate support and attack an argument receives, a label is assigned to each argument. This label signifies the status of an argument. The HERMES system detects both conflicts and consistencies in the arguments, and identifies a suitable candidate solution from the discourse.

Karacapilidis (2006), Karacapilidis et al. (2009), Tzagarakis, Gkotsis, Hatzitaskos, Karousos, and Karacapilidis (2009) and Tzagarakis et al. (2010) developed CoPe_it, a Web-based argumentation system for collaborative learning. CoPe_it allows stakeholders to participate in the argumentation process and thereby support discourse for knowledge sharing and building knowledge graphs. CoPe_it! supports weighing of arguments, and its weighing process is not based on either the notion of fuzzy logic concepts or probability. The decision-making features and capabilities in CoPe_it! can be further enhanced by making use of those weights provided by the agents in the dialogue.

PARMENIDES (Persuasive ARguMENt In DEmocracieS) (Atkinson et al. 2006) is a web-based argumentation system which encourages agents to post a position and justify a particular action and give them an opportunity to critique that position by disputing various points. This system allows several kinds of attacks and supports for persuasive argumentation. This system provides a form-based questionnaire to collect views from the stakeholders and builds an argumentation structure.

Vesic et al. (2012) recently developed the Synergy system to support argumentation-based decision-making. Synergy allows a stakeholder to post arguments supporting or attacking an option with a probability measure although no further specific use of them in argumentation computation is reported (Vesic et al. 2012). This system might be used as a basis of a reference system for pros and cons, just as one might use Wikipedia to collect thoughts on issues. The researchers here need to present how goals, arguments and options are defined.

Argument mapping systems such as Debatepedia (http://idebate.org/debatabase), TruthMapping (http://www.truthmapping.com/), DebateGraph (http://debategraph.org) and LivingVote (http://www.livingvote.org/) provide primitive discussion services. Several of them do not support scoring or weighing arguments quantitatively.

Social and emotional aspects of argumentation including argumentation polarisation, which do exist in face-to-face meetings and debates, need to be incorporated in computer-supported argumentation systems. Existing systems do not support these features although Klein (2012b) presented several deliberation metrics of argumentation which are related to social interactions (Sillince 1995; Sillince and Saeedi 1999; Klein 2012a). Balkanization and dysfunctional argumentation presented in the works of Klein (2010, 2012) are related to polarisation. While Klein (2012b) identifies the importance of polarisation problem, he does not discuss any method of detection of polarisation groups and how they are implemented in the Deliberatorium in his papers (Klein 2010, 2012a, 2012b).

Arvapally et al. (2012) proposed an approach to identify polarisation groups in argumentation systems. Their approach classifies stakeholders using K-means clustering algorithm (Arvapally et al. 2012). Their method assumes that stakeholders are non-overlapping and generates the results.

2.4. Clustering algorithms

Clustering algorithms can be broadly classified as exclusive clustering, overlapping clustering, hierarchical clustering and probabilistic clustering algorithms. Exclusive clustering algorithms such as K-means can be used, but they produce only crisp results. Hierarchical clustering would be more appropriate if we wanted to analyse the intra-group polarisation assessment. The probabilistic clustering methods such as Gaussian mixture model, if used for the argumentation polarisation analysis in our system, would identify membership of stakeholders in polarisation groups with probability. However, a degree of membership of stakeholders in polarisation groups is more desirable in assessing their memberships in the argumentation polarisation analysis. We would like to have a stakeholders’ degree of membership in a polarisation group, not the probability of being in a group. Hence, fuzzy-based clustering algorithms seem to be more appropriate. Models such as the latent semantic analysis (Deerwester, Dumais, Furnas, Landauer, and Harshman 1990), the probabilistic latent semantic analysis (Hofmann 2001) or the Latent Dirichlet Allocation (Blei, Ng, and Jordan 2003) are more appropriate if used in the clustering textual argument analysis. The K-means clustering algorithm was used earlier (Arvapally et al. 2012) to identify non-overlapping polarisation groups. Overlapping clustering algorithms such as fuzzy c-means are more suitable and output more social dynamic information than hard clustering algorithms.

3.1. Elements of an argumentation tree

In our previous research work, we developed an OLIAS on the World Wide Web platform. Below is a brief discussion of its building blocks for a better understanding. For more information, please refer to our previous papers on the system (Liu et al. 2007, 2009, 2012).

This sub-section presents the elements of an argumentation tree in our OLIAS.

Project: Project node is at the first level of the argumentation tree where the project name and other information such as the stakeholder who posted the project are posted. Any stakeholder can post a project, and under the node project, and any number of relevant and concerned issues of that project can be posted.

Issue: Issue node is at the second level in the argumentation where a stakeholder can post concerned decision-making issues under the project node. Positions or alternative solutions are posted under the issue in the argumentation tree. A decision-making group can participate in several decision-making issues related to a project. The decision issues we are discussing here are very strategic and are of high importance to an organisation. These issues are usually cognitively complex. The results from these decisions affect several others financially and economically.

Position: Since the alternatives allow the stakeholders to explore the solution space through interactions, any number of positions can be posted under an issue. Arguments are either directly or indirectly associated with a position node. Position or alternatives are the two keywords which are used interchangeably in this article.

Argument: Every argument carries a strength which expresses its association with its parent node. Stakeholders are also responsible for posting argument strength explicitly along with the argument and the argument strength ranges from −1 to +1. A negative strength conveys that the argument is attacking its parent argument, an argument with positive strength conveys that the argument is supporting its parent node, and an argument with strength zero expresses its indecisiveness. Stakeholders can strengthen their arguments by posting evidence which support their arguments (Liu et al., 2007, 2009, 2012). The dialogues exchanged by stakeholders in the argumentation process are for deliberation, persuasion and negotiation.

Evidence: The OLIAS allows the stakeholders to post evidence supporting their arguments. Evidence can be attached to arguments posted by others and any number of pieces of evidence can be posted.

In Figure 1, E1 and E2 represent the pieces of evidence posted by stakeholders supporting arguments in the argumentation tree. The weight assigned to evidence represents the degree of confidence that the participant has in the evidence or in the probability of the evidence holding true (Liu, Khudkhudia, and Leu 2008). For example, evidence E1 has been assigned 0.9 and 0.1. This signifies that the owner of that evidence has 0.9 probability score for the evidence holding true and 0.1 for the evidence holding false. The weight value of evidence is used to reassess the strength of the argument using evidence-based fuzzy inference engine (Liu et al. 2008).

Figure 1.

Position dialogue graph.

Figure 1 presents an example of a position dialogue graph and it illustrates example arguments in a position dialogue graph. Based on the strength of the argument, the system identifies labels such as medium support, strong support, indecisive, medium attack and strong attack. The labels are linguistic terms whose semantics are captured by their membership functions. The degree of strength of an argument posted by their owners will be used for fuzzy inference by the fuzzy inference engine based on fuzzy inference rules using the labels.

Before the argumentation process, stakeholders initially have an idea and an opinion towards the decision problem. As the argumentation process unfolds, they get to know the opinions and views of other stakeholders. In the process, stakeholders also have the opportunity to express their arguments on other individual's arguments. Contrasting opinions lead to conflicts and resolving conflicts lead to refining the opinions of stakeholders. At the end of the process, consensus will be developed among stakeholders. This exchange of information in groups leads to collective decision-making. Please see our article (Liu, Wanchoo, and Arvapally 2011) to know more about the opinions and goals of stakeholders in an argumentation process.

The OLIAS supports multiple scenarios in the collaborative decision-making environment. A decision-making group is led by the decision-maker who also participates in the argumentation process along with other stakeholders. In another context, all stakeholders collectively participate in the argumentation process, and take decisions effectively.

Figure 2 presents a snapshot of an example argumentation tree in the OLIAS. The following sub-section presents a detailed discussion about the argumentation reduction fuzzy inference system in an argumentation tree.

Figure 2.

Snapshot of argumentation tree in the OLIAS.

3.2. Fuzzy argumentation reduction inference engine

The fuzzy argumentation reduction inference system (Liu et al. 2007, 2009) carries out the inference process on an argumentation tree. This process is carried out on all the arguments in a tree. Arguments are reduced level by level such that all the arguments are directly associated with their respective alternatives. The favourability factor of an alternative is then computed by using the argument strengths. Both the strength of an argument and the strength of its parent argument are provided as inputs to the fuzzy inference engine. Child argument and parent argument are put on the same level of an argumentation tree based on the inference. The child argument is reduced by one level in the tree such that both arguments are siblings, providing a new strength value which is relative to its parent argument.

Based on the 4 fuzzy heuristic rules, 25 fuzzy rules were derived based on fuzzy labels. These 25 rules in the fuzzy rule base are used by the fuzzy inference engine argumentation reduction (Liu et al. 2007, 2009). The following four are the fuzzy argumentation reduction heuristic rules (Liu et al. 2007, 2009).

The argumentation reduction fuzzy inference system takes strengths of an argument that has to be reduced and its parent argument as inputs. Based on the fuzzy membership functions, these strength values undergo fuzzification process. The output from the fuzzification process is given as input to the fuzzy inference engine, and appropriate fuzzy rules are applied from the fuzzy rule base for inference. Based on the rules, a relative strength value is derived with respect to its parent argument. This new score undergoes the defuzzification process. In the defuzzification process, the obtained inputs are converted to crisp outputs. For further information on the fuzzy argumentation inference system, please read our articles (Liu et al. 2007, 2009).

After the argumentation reduction process, the favourability of alternatives is computed by aggregating the argument strengths associated with an alternative. The favourability factor produced by the system for each alternative represents the favourability of the decision-making group for that alternative. The higher the favourability factor of an alternative is, the more favourable it is to the group. It is computed based on the fuzzy argumentation reduction inference engine.

4.1. Argumentation process

This is the first step in the framework, stakeholders undergo a dialogue process here. Figure 4 presents a sample argumentation tree. Figure 4 illustrates both a decision-making issue (root node) and three positions, i.e. alternatives posted under the issue node. Sixteen arguments posted by three different stakeholders are listed under position 1, position 2 and position 3. S₁, S₂, S₃ represent the three participating stakeholders. Arg1, Arg2, Arg3 are some of the arguments in the sample argumentation tree. Each stakeholder's arguments are represented in a different colour.

Figure 4.

Sample argumentation tree before the argumentation inference process.

4.2. Data collection

After the argumentation process, the framework runs argumentation reduction fuzzy inference engine on the argumentation tree to compute a stakeholder's favourability for an alternative. In Figure 4, stakeholder S₂ has contributed three arguments under position 1. While one argument is directly associated with position 1, and the other two are associated with the arguments posted by stakeholder S₁.

The fuzzy inference rules presented in Section 3.3 were used for the argumentation reduction process. The association between (Arg1, position 1) and (Arg4, Arg1) is considered for using the appropriate fuzzy inference rules. Based on the suitable fuzzy rule, the Arg4 is reduced level by level such that it is directly associated with position 1. The same procedure was conducted for Arg6. The system ensures that all arguments posted by a stakeholder are directly associated with an argument. The argumentation-based fuzzy inference system then reassesses the strengths of the arguments based on the inference rules. The new strength that an argument is assigned is relative to the solution alternative.

Once all arguments are directly associated with the alternatives, the strengths of the arguments posted by a stakeholder under every alternative are aggregated. Hence, the favourability of a stakeholder towards every alternative is derived. This process is conducted for the stakeholders at every position posted in the tree. The favourability of a stakeholder is represented by a numerical value. This value is the sum of the arguments’ strength of a stakeholder for a position.

See Figure 5 for the argumentation tree after the fuzzy inference process. The favourability of stakeholder S₂ for position 1 is the aggregate of the argument's strength: Arg4, Arg2 and Arg6 (Figure 5). Similarly, the favourability of stakeholder S₂ for positions 2 and 3 is derived. If the favourability value of a stakeholder for a position is negative, the stakeholder has more attack than support for his/her arguments for that position. If the favourability of a stakeholder for a position is positive, the stakeholder has more support than attack for the position.

Figure 5.

Argumentation tree after the argumentation inference process.

If the favourability factor of a stakeholder for a position is zero, the stakeholder is neutral in opinion about the position. Because the aggregate of both support and attack of the argument's strength is neutralised. In another case, stakeholders may not have posted any arguments under that position in the tree. Following the argumentation process, the argumentation system computes the favourability of each stakeholder for all the positions in the argumentation tree.

All stakeholders are encouraged to participate in the dialogue process, but if stakeholders do not present his/her complete opinions on given issues, the argumentation polarisation analysis might help detect missing opinions since the degree of stakeholders in polarisation groups from the clustering analysis might be different from their expectations, and prompt stakeholders to address the issue by adding their opinions. Of course, the polarisation analysis itself cannot solve the problem of missing opinions completely since it is not a problem of the argumentation polarisation analysis.

In the data collection phase, we acquire data from an argumentation tree, while data pre-processing phase consists of different tasks such as cleaning the data, handling missing data and normalising the data to make the data consistent (Wirth 2000).

4.3. Data pre-processing

The opinion of a stakeholder is represented as a vector after the favourability of a stakeholder for each alternative is derived. Each element in the vector represents the favourability for a position. The number of positions under an issue in an argumentation tree represents the size of the vector. The vectors are normalised to retain consistency in the data.

The min–max normalisation technique (Equation (1)) is used to normalise the elements in the vector. We refer to these vectors as opinion vectors because each vector represents a stakeholder's opinion towards the given issue. Min A and max A represent the minimum and the maximum values in the original data, respectively. New_min A and new_max A represent the new ranges for the data provided. In our experiments, we have assigned new_min A as −1 and new_max A to +1 as the new ranges. The stakeholder's favourability for an alternative is represented with numerical values ranging from −1 to +1. An element in the opinion vector between −0.1 and −1.0 signifies that the stakeholder attacks an alternative. Values between +0.1 and +1.0 signify that the stakeholder supports an alternative. After the data is normalised, the fuzzy c-means clustering algorithm is used on the opinion vectors.

4.4. Fuzzy c-means clustering

This sub-section briefly presents on how the fuzzy c-means clustering algorithm is used in our approach. After the system computes the favourability of each stakeholder across all the positions, the fuzzy c-means algorithm is applied on those favourability vectors. Because each stakeholder's opinion is represented as a vector, we have the opportunity to both compare and assess how close stakeholders’ opinions are. Let us suppose stakeholder S₃ is one among the decision-making group, and there are three different positions for the decision-making issue in the argumentation tree. S₃ has presented his opinion across all three positions. The favourability factor of S₃ is represented as (0.9, −0.2, 0.5). This signals that S₃ is supporting position 1 and position 3 and weakly attacking position 2.

The fuzzy-based clustering algorithm outputs the clusters, providing each stakeholder's membership in c clusters. The fuzzy c-means clustering algorithm produces c number of clusters from the given data. The algorithm tries to minimise the objective function over several iterations. When the objective function value remains unchanged, it produces the clusters. The algorithm provides the centroid of each cluster.

Fuzzy c-means clustering algorithm works by assigning each data point to each cluster based on the distance between the cluster centroid and the data point. The closer the data point is to the cluster centroid, the higher its membership is in that cluster. The fuzzy c-means clustering algorithm is based on minimising its following objective function (Equation (2)).

S and c are provided as inputs to the algorithm and the algorithm produces membership of each data point in multiple clusters as output.

We used the Euclidean distance metric (Equation (3)) to assess the similarity measurement among stakeholders’ opinions in the fuzzy c-means clustering algorithm.

4.5. Cluster analysis for polarisation assessment

The centroid of a cluster is a vector. This vector represents the aggregate opinion of a polarisation group. The centroid of each polarisation group can be further used to analyse the dissimilarity between polarisation groups using the Euclidean distance as a measurement metric. When analysing polarisation groups, we may encounter groups with completely contrasting opinions. In some cases, we might also see groups with similar opinions. Some groups might share similar opinions or contrasting opinions with respect to a particular alternative. These polarisation groups tend to form factions, supporting stakeholders within their group. They tend to attack stakeholders in the opposing group, using both their arguments and evidences supporting their arguments. Stakeholders might even use arguments to support their arguments. The degree of membership of a stakeholder in a group can help both the decision-maker and group leaders understand the loyal stakeholders/followers within his/her polarisation group. It also allows for further investigation on new approaches to identify leaders in each polarisation group. A stakeholder from each polarisation group with the highest degree of membership can be acknowledged as the group leader.

In some cases, a stakeholder might absolutely belong to a polarisation group. In another instance, a stakeholder might have an equal degree of membership in two different polarisation groups. This information might help polarisation leaders in pursuing each stakeholder based on stakeholders’ interest and thereby providing incentives to them. One can also arrange stakeholders in ascending or descending order based on the stakeholders’ degree of membership and generate a ranked list. Therefore, each polarisation group has a ranked list of stakeholders based on the membership value. The decision-maker can also generate top-k list from the ranked list. One could further investigate the overlapping of the ranks of a stakeholder in the multiple polarisation groups. A stakeholder might have the same rank in two or more polarisation groups.

5.1. Example

This sub-section presents a small and simple example to illustrate the framework we presented in detail. Suppose that a family plans to buy a new car and selecting a car model is an important decision-making issue in the family. The family consists of 11 members including grandfather, grandmother, father, mother and kids. Let us use the OLIAS for collaborative decision support to resolve the issue in hypothetic scenarios. Initially, the family members post the issue and their solution alternatives in the argumentation tree and the family members exchange arguments over the issue. The decision issue and solution alternatives (positions) are presented as follows:

Table 3 presents the degree of membership of all stakeholders in both polarisation group 1 and group 2. Stakeholder S₅ is sharing his opinion with group 1 with a degree of 0.9669 and is strongly associated with group 1. While stakeholder S₁₁ has a degree of membership 0.4495 in group 1 and 0.5505 in group 2, though he belongs to group 2, he is also affiliated with group 1 with a good membership score.

Figure 8 presents the values of Table 3 in a histogram. This simple example is provided to understand the process of the argumentation polarisation analysis. A scalable example with complicated decision issues can be illustrated in the OLIAS for argumentation polarisation analysis.

Figure 8.

Degree of membership of stakeholders in group 1 and group 2.

6.1. Background

To evaluate the effectiveness of our proposed framework, we conducted three experiments on a case study (Satyavolu 2010). We recruited 24 students from a software engineering class at Missouri University of Science and Technology. A hypothetical case study was developed and provided to students before the experiment began. This case study was the common platform for all three experiments. Students played the role of stakeholders in the experiments. Each experiment in our evaluation had a decision-making issue pertaining to the case study and the 24 stakeholders resolve the issues using our argumentation system. Data collected from our experiments are from the stakeholders. A demonstration was given to the students on how to use the argumentation system. A user manual was also provided. The experiment was conducted for one month. The stakeholders were provided with the access to our OLIAS. We provided the decision-making group with one week for each decision-making issue. Initially, we ran the experiment with different c values, such as c=2, 3, 4, 5, 6, 7. The objective function scores for c=2 and c=3 were much higher than c=4, 5, 6 and 7. Before the experiment began, gave out a survey to those 24 stakeholders in order to know their initial opinions on the given decision-making issue. We realised that there were four major opinions among the stakeholders. This motivated us to consider c value as 4. However, by using root mean square error as a criterion, one can try different c values. In this article, we present experiment I in detail and discuss the overview of experiment II and experiment III.

6.2. Case study

The adoption of a software metrics programme for a software project development is useful in organisations. Identifying the suitable metrics programme for a given project is both important and difficult as it depends on various factors, such as the size of the organisation, the size of the project and many more. In several situations, an organisation finds difficulty in identifying the most suitable metrics programme. In our case study, we provided a software product to be developed. The stakeholders use our argumentation system to decide the suitable metrics programme in large-scale organisation, medium-scale organisation and small-scale organisation in experiments I, II and III, respectively. We provided stakeholders with concrete information about the software development project in the case study. We also explained the positions in detail. Comprehensive metrics programme, light-weight metrics programme and no metrics programme are the three positions provided to all the three issues. In no metrics programme, organisations do not adopt any software metrics programme, fewer than 35% of the artefacts are measured using a light-weight metrics programme and from 35% to 60% of the artefacts are measured in the comprehensive metrics programme. Since the decisions issues are built up on the common case study, these three positions were the same to all three decision issues. The students’ names were replaced with stakeholder identification numbers to preserve the privacy of the participating students. The following two example arguments were posted by stakeholders under the first decision-making issue. These arguments were posted under the comprehensive metrics programme and light-weight metrics, respectively.

We provided stakeholders with some criteria in order to motivate them to participate in the argumentation process: customer satisfaction, quality, cost, volatility of requirements were some of them. Stakeholders, however, can also consider other relevant criteria.

6.3. Experiment I

Stakeholders participated in the argumentation process for selecting the suitable software metrics programme for software development in a large organisation. Stakeholders exchanged 204 arguments for one week using our system. The proposed method was then applied on the argumentation tree with c=4 as input. The fuzzy c-means algorithm has run for 32 iterations by minimising the objective function score. The framework identified four polarisation groups and presented each stakeholder's degree of membership in four polarisation groups. After 32 iterations, the fuzzy c-means clustering algorithm had stopped and thereby producing the polarisation groups as output. Figure 9 presents a plot where the objective function values are plotted against the iteration count. Table 4 presents the centroids (opinions) of each polarisation group, and stakeholders in each group produced by the method. Table 5 presents the membership of each stakeholder in the four polarisation groups.

Figure 9.

Objective function values plotted against the iteration count.

Group 1 consisted of four stakeholders who attacked the comprehensive metrics programme, strongly supported the light-weight metrics programme and weakly supported the no metrics programme. Group 2 consisted of six stakeholders who supported the comprehensive metrics programme, attacked the light weight and no metrics programme. The opinions of the polarisation group 1 and group 2 were contrasting and they were like two different factions. Group 3 consisted of 10 stakeholders who strongly supported the comprehensive metrics programme, weakly supported the light-weight metrics programme and attacked the no metrics programme. The opinion of the stakeholders in group 3 was contrasting with the opinion of the stakeholders in group 1 and group 2 under different positions. The 10 stakeholders in group 3 shared similar opinions with group 2 under the context of the comprehensive metrics and no metrics programme. They, however, had contrasting opinions under the context of light-weight metrics programme. Group 1 stakeholders had a similar opinion with stakeholders in group 3 under the context of the light-weight metrics programme. Groups 1 and 3 had contrasting opinions with respect to the other two positions. The four stakeholders from group 4 attacked all three alternatives.

Figure 10(a) presents the opinion vectors of the 24 stakeholders that were plotted in a three-dimensional co-ordinate system with position 1, position 2 and position 3 as the axis. We have three positions for the issue and the opinion vector consists of three elements. Hence, we have a three-dimensional figure. Position 1, position 2 and position 3 refer to comprehensive metrics programme, light-weight metrics programme and no metrics programme, respectively, in Figures 10(a) and 10(b). Figure 10(b) presents the opinion data of the 24 stakeholders that were plotted after the framework was applied to the collected data. Each polarisation group is represented in a different colour and a different symbol. These plots also provide more insight on the polarisation groups.

Figure 10.

(a) Opinion vectors of stakeholders plotted in three-dimensional space before polarisation assessment. (b) Polarisation groups identified by the method (best viewed in colour).

Table 5 presents the degree of membership of the stakeholders in all four polarisation groups. For example, stakeholder S₉ had a membership of 0.064793 in group 1, 0.346609 in group 2, 0.188301 in group 3 and 0.400297 in group 4. S₉ had the highest membership in group 4 when compared with the degree of membership with other groups. S₉ belonged to all the polarisation groups; however, S₉ had highest membership with group 4.

Polarisation group 1 and group 2 had contrasting opinions, stakeholders S₄, S₁₈, S₂₁ and S₂₃ were from group 1 and had a contrasting opinion with the stakeholders S₆, S₇, S₈, S₁₂, S₁₄ and S₂₄ in group 2. Stakeholder S₄ belonged to group 1 and shared opinion with group 1 with a degree of membership of 0.367586, and 0.202437 with group 2. We understand that although S₄ is from group 1, shared opinion with group 2 to a degree of 0.202437. Another interesting example from Table 5 is stakeholder S₁₄ who shared the same opinion as group 2 with a degree of 0.479735 and 0.454579 with group 3. The membership values of S₁₄ for groups 2 and 3 were very close. Stakeholders from group 3 or the polarisation leader from group 3 can possibly pursue S₁₄ to join their polarisation group and extend S₁₄s’ support. One can also understand that S₁₄ belongs to group 2; however S₁₄ also had a strong affinity for group 3 as well. At the same time, stakeholder S₂₄ belongs to group 2, S₂₄ had a membership of 0.966495 in group 2 and 0.001358 in group 1. We can conclude that S₂₄ strongly belonged to group 2 compared with S₄ who weakly belongs to group 1 and group 2.

Figure 11 presents the membership plot of the 24 stakeholders in the decision-making group. Every stakeholder has membership values for polarisation groups. The highest membership value of a stakeholder among all his/her membership values is presented in Figure 11. Some stakeholders such as S₃, S₁₀, S₁₁, S₂₁, S₂₂ and S₂₄ strongly belong to a polarisation group and they have a weak degree of membership with other polarisation groups. The rest of the stakeholders have relatively lower membership values in a polarisation group, and they actually share opinion and belong to other polarisation groups to a good degree of membership.

Figure 11.

Each stakeholder's highest degree of membership among all his memberships in polarisation groups.

Table 6 presents the opinion dissimilarity measurement of the polarisation groups. Since the centroid of each polarisation group is the opinion of that group, the Euclidean distance metric is used to measure the dissimilarity of the opinion of the polarisation groups. Information from Table 6 explains the closeness among the polarisation groups in terms of opinion for the given decision-making issue. The larger the distance value between polarisation groups, the more dissimilar the polarisation groups are under their opinion for the given decision-making issue. For example, stakeholders from group 3 may pursue stakeholders in group 2 and converge to one group, since group 2 and group 3 are close in terms of their opinion.

Based on the membership value of a stakeholder in each polarisation group, stakeholders are ranked. Table 7 presents the stakeholders ranked list in the descending order for all the polarisation groups. The ranked list is arranged from top to bottom in the descending order. Stakeholder S₂₁ in polarisation group 1 had the highest membership value in group 1, and S₂₂ had the lowest membership value in group 1. Stakeholder S₂₁ is ranked number one in polarisation group 1; however S₂₁ is ranked last in group 2 and group 3. A decision-maker can correlate the opinion of each polarisation group and the ranked list presented in Table 7 for more information on social dynamics in the decision-making group.

By further analysing the information from Table 7, one can generate the top-k list of stakeholders from each polarisation group based on the degree of membership. This information can be used to identify the top-k stakeholders who have the highest degree of membership with each polarisation group. We have assumed k value as 4 since we used c as 4 earlier in running the experiment. Although there is no association between the variables c and k, we could also generate the top-6 stakeholders from each polarisation group. The framework can even generate the bottom k stakeholders from each polarisation group. One can even use the information from Table 8 for identifying the polarisation leader in each group. A polarisation group leader is a stakeholder from a polarisation group who leads a group. We could assign the stakeholder with highest degree of membership as a leader of that group.

We could further analyse the information from Tables 7 and 8 and check for the overlapping or rankings of a stakeholder in multiple polarisation groups. For example, from Table 7, stakeholder S₁₆ had a rank of 12 in both polarisation groups 1 and 2. Similarly, S₁₁ had a rank of 21 in polarisation groups 1, 2, 4 and S₂₁ had a rank of 24 in groups 2 and 3.

The information provided by this method offers a great insight in to the social dynamics of the decision-making group. The four stakeholders in the polarisation group 4 from experiment I do not support any position provided to them. The decision-maker might use this information and request those stakeholders to come up with a new position that they think might be more suitable to the given decision-making issue. The 6 stakeholders in group 2 and 10 stakeholders in group 3 share similar opinion with respect to the comprehensive metrics programme. Since majority of the stakeholders support this alternative, the decision-maker might choose to take the decision based on this. From Figure 11, the decision-maker can understand and identify stakeholders who had both high and low degrees of memberships. The decision-maker might also possibly look in and understand to which stakeholder can be pursued more comfortably in case they had to pursue stakeholders during the decision-making process. The information produced by the approach which is presented in Tables 4–8, Figures 10(a), 10(b) and 11 can help decision-makers and stakeholders to take more informed decisions.

6.4. Analysis of experiment II and experiment III

This sub-section presents the analysis of experiment II and experiment III. The decision issues in experiment II and experiment III were about the selection of suitable software metrics programme in a medium-scale organisation and small-scale organisation, respectively. Stakeholders exchanged 314 arguments and 176 arguments and constructed argument trees in experiment II and experiment III, respectively, using our OLIAS. The same set of stakeholders in experiment I participated in experiments II and III. The framework was applied on both the argumentation trees with c value as 4, and the system produced 4 polarisation groups in both the experiments.

From experiment II results, we learnt that the opinion vectors of the four polarisations groups were unique and no group shared the same opinion with the same strength. Polarisation group 2 had the highest number of stakeholders. Group 3 consisted of only two stakeholders. With this information, decision-makers have an opportunity for closely investigating the opinions of polarisation group 2, since it has large number of stakeholders and group 3, since it is equally important to understand the opinion of polarisation groups with few stakeholders. From experiment III results, we understood that a stakeholder in one top-k list of a polarisation group was not present in another list. In an earlier experiment, a stakeholder was spotted in two different top-k lists. These top-k lists can be used to identify leaders in polarisation groups.

The degree of membership of stakeholders in polarisation groups was different from experiment to experiment. In addition, as stakeholders interact in the dialogue process, stakeholders might change their opinions. Hence, their degree of membership in polarisation groups changed dynamically. In experiment I, a higher number of stakeholders had membership to a polarisation group with a value greater than 0.9 when compared with other experiments. The number of stakeholders with membership values greater than 0.9 to a polarisation group decreased from experiment II to experiment III. Polarisation groups across experiments were very dynamic. Stakeholders with a high degree of membership in a group usually tend to have a low degree of membership in other groups. This is logically sound. The aggregate of membership values of a stakeholder in all the polarisation groups is always equal to 1.