Exploring the perspectives of medical students on application of e-learning in medical education during the COVID-19 pandemic
Abstract
BACKGROUND:
The use of e-learning has become mandatory during the COVID-19 pandemic. However, there are many barriers to applying e-learning in medical education.
OBJECTIVE:
This study aimed to explore medical students’ perspectives on the application of e-learning in medical education during the COVID-19 pandemic.
METHODS:
This Q-methodology explorative study was conducted in Kurdistan Region of Iraq. A sample of 37 medical students was purposively selected to represent different characteristics. The students distributed 37 statements representing different aspects of e-learning in medical education into a nine-point scoring grid from “least agree” to “most agree.”
RESULTS:
Data analysis revealed three distinct viewpoints. The first viewpoint, complete dependence on e-learning, emphasized a preference for e-learning to continue medical education and complete the study year with a minimal return to study halls or practical/clinical sessions. The second viewpoint, opponents of applying e-learning in medical education, included a generally negative view about e-learning and its role in medical education during the COVID-19 pandemic. The third viewpoint, e-learning as a supplement to medical education, emphasized a generally positive view about e-learning and considered it a supplement to the theoretical parts of medical education during the pandemic.
CONCLUSION:
The three diverse viewpoints are primarily distinguished by the availability of e-learning experience and skills, availability of technology, risk perception of COVID-19, and the need for in-hospital clinical teaching. Provision of necessary facilities and training is required to strengthen the role of e-learning in medical education. A safe environment is needed for on-campus or hospital clinical teaching.
1Introduction
Medical schools are motivated to formulate their change and quality improvement plans to align with the main international standards as set by the World Federation for Medical Education. These standards are based on the current understanding of fundamental principles and best practices in designing, maintaining, and enhancing medical education programs. They are intended to guide medical education program development and evaluation, facilitate diagnosis of strengths and weaknesses relating to the medical education program, and stimulate quality improvement [1].
Effective educational systems and quality improvement have been recognized to be essential for effective learning and curriculum [2]. E-learning provides an opportunity for learners to study at a convenient time, pace, and place. It also provides education at a relatively lower cost than those for classroom facilities [3]. E-learning in the medical education field is a relatively new concept, but it is rapidly growing [4]. When combined with face-to-face instructions, e-learning can help students use real and virtual environments to support their learning [5].
The use of information technology-empowered learning has been emphasized in the recent calls for reform in medical education and training [6]. Medical education, like any other health profession education, requires high-quality educational methods [3]. With the advances in technology and social media, e-learning is becoming a new and rapidly developing approach for medical education [7].
The application of e-learning in medical education has been significantly enhanced during the COVID-19 pandemic [8]. It has been suggested that e-learning might become an ideal solution to continue learning processes during the COVID-19 pandemic and other similar emergencies [7]. Worldwide, many universities have applied e-learning for medical education during the COVID-19 pandemic [8, 9]. However, the substitution of clinical sessions, which constitutes the backbone of medical education for teaching the clinical competencies, by e-learning remained an important challenge [10]. Therefore, many universities have applied blended learning to overcome this problem partially.
Blended e-learning is a mixed mode of delivery that combines traditional face-to-face learning for practical sessions using small groups of students and e-learning techniques using an electronic learning platform such as Moodle [11]. Students use e-learning technology to read lectures and course material outside the classroom, with the online classroom time is usually devoted to interactive problem-solving exercises [5]. Face-to-face learning takes into account all preventive measures against the spread of the disease.
Like other parts of the world, Iraq and Kurdistan Region of Iraq have experienced a lockdown of every part of life due to the COVID-19 pandemic. This lockdown included schools and universities, which obliged them to consider e-learning as an urgent substitute. Some universities already had their digital learning platforms; however, they were not operational at full capacity. In the context of the spread and continuation of the COVID-19 pandemic over the past few months, the universities in Iraq and Kurdistan Region of Iraq have adopted e-learning mode and blended learning to some extent to deal with the challenges and meet students’ expectations [12, 13].
Using e-learning in medical education is a new experience in many settings, including Iraq. The faculty and students had limited experience with using e-learning and related technology. The universities were not prepared for such a quick shit to e-learning in terms of facilities, technology, and platforms. Limited research is available about the impact of the quick move from traditional learning to e-learning in medical education, and the associated challenges are not well understood. The role of students’ perspectives in shifting to e-learning in medical education is not wells studied, particularly in Iraqi context. Therefore, it is essential to understand the technological, institutional, financial, and faculty and students’ barriers to successfully implementing e-learning in medical education [7]. Moreover, understanding students’ perspectives and attitudes is essential for initiating and better-delivering e-learning [14]. Therefore, this study aimed to explore medical students’ perspectives on the application of e-learning in medical education during the COVID-19 pandemic in Kurdistan Region of Iraq.
2Literature review
The learning and teaching process has been revolutionized by novel innovations in e-learning. However, students and faculty have had to shift to e-learning in a matter of days despite the technological and emotional hardships associated with such a switch [15]. Students and educators need to adapt to utilizing e-learning. COVID-19 pandemic remarkably increased the need to apply e-learning in different fields of study, including medical education.
Nowadays, most students use the internet, and many have smartphones and laptops, indicating a good technological base to implement e-learning [3, 16–18]. Although e-learning facilities are available in many universities, including the universities in Iraq, a limited number of students knew about e-learning before the COVID-19 pandemic or were interested in guidance on it [19, 20]. This may be because many universities did not use e-learning to its full potential as an educational resource but rather a limited modern technology project [21]. Many students oppose the potential usefulness of e-learning, namely, being an interactive, accessible, and flexible platform and suitable for discussion on complex subjects and acquiring better skills [3, 22]. The students’ satisfaction level about e-learning in health and medical education is better among developed countries than developing countries. While many students in developing countries agreed that e-learning is helpful for acquiring knowledge, they think it is ineffective in acquiring clinical and technical skills [23]. Research has also shown that students with experience with e-learning have a better perception of the role of e-learning and its usefulness [16, 24].
Nowadays, the majority of faculty use the internet and own a laptop. The majority have good knowledge about e-learning and used it. The attitude of the faculty is generally equivocal about e-learning’s usefulness in teaching and learning. Although faculty members find working from home to be easier and better for handling one’s workload [25], most faculties prefer a blended program combining e-learning and face to face learning [26]. This equivocal response may be due to the faculty’s continuous efforts to provide new learning resources on the one hand [27] and to the limited time available to learn the new technologies on the one hand [28]. Other advantages of e-learning from the faculty point of view could be attributed to freeing up the on-campus timetable, allowing more time to deliver more complex subjects [29], a greater opportunity for a higher level of cognitive learning [30], and finally reducing the time needed for the pre-lab explanation in laboratories and skills labs [31].
The literature shows that there are barriers to the implementation of e-learning; these include obstacles related to technology, resources, skills, institutional strategies, and support [32]. Both students and faculty face poor internet connectivity and a lack of hardware as the primary technical barriers to e-learning [20, 33]. Other barriers related to technology include other hardware equipment, learning management systems, and digital library, among others [4]. Essential prerequisites of e-learning quality include basic infrastructure maintained with regularly updated technologies [5] and technical support [34], and a sound and clear institutional policy and guide for both students and teachers [4, 26, 27].
COVID-19 has disrupted education worldwide, and the pandemic has compelled universities across the globe to halt campus learning to contain the spread of the disease. However, in times of distress, consolation can always be found. The pandemic forced the teaching community to find alternatives. Many have moved to online learning to secure the continuance of teaching and assessment [35, 36]. However, many students and faculties are skeptical of the ability of e-learning to replace face-to-face learning or practical training. Students are generally willing to attend practical sessions if the situation permits, and most of them disagree that e-learning can replace practical training [37]. Therefore, given the COVID-19 lockdown situation, there is a need for improving the e-learning methods while more blended learning is recommended in medical education [23]
3Methods
3.1Design
Q-methodology involves combining qualitative and quantitative methods to study human subjectivity and preferences in a systematic way. Q-methodology characterizes the common viewpoints of different groups of people [38, 39]. The main steps of Q-methodology include preparing a set of statements about an issue, a sample of respondents who rank-order the statements on a quasi-normal distribution according to their perspectives, and factor analysis of the individual rankings [38, 40].
3.2Setting and participants
This study was conducted in a medical school in Kurdistan Region of Iraq. Since the selection of participants in Q-methodology studies is based on maximizing the representation of wide-ranging opinions [41], a sample of medical students was purposively selected to include male and female students and students from different study years and social and economic statuses.
Q-methodology does not need sample size estimation since it is an exploratory method and does not mean to test a hypothesis. The sample size is usually, but not always, smaller than the statements’ number [42]. In principle, four or five persons are needed to define each anticipated factor, and the number of factors is usually two to four. Therefore, a sample of 37 students was selected for the study [43]. Initially, a random sample of 200 students from the different study years was selected. The main characteristics of these students were enlisted, including age, gender, residence, study year, economic level, and computer skills. Then, 37 students were purposively selected from the list with the aim of including students with different characteristics.
3.3Statements
Initially, we conducted a comprehensive review of the literature to identify the main aspects of applying e-learning in medical education [14, 44–46]. Statements representing different aspects of e-learning in medical education and medical students’ perspectives and acceptance of e-learning were selected. As a result, 56 statements related to the application of e-learning in medical education were extracted.
These statements were checked for similarities and differences, the repeated and opposite statements were removed, and similar statements were merged. In the end, 37 statements were selected. These statements were numbered randomly. A quasi-normal distribution with 9 piles (from least agree (–4) to most agree (+4)), and 37 cells was created (Fig. 1). The final data collection tool comprised the Q-grid, the list of statements, and the participants’ sociodemographic and study characteristics, such as age, gender, residence (urban or rural), study year, economic level, and computer skills. The economic level was classified as below average, average, and above average based on the student’s self-rating. Similarly, students themselves classified their computer skills as poor, good, or very good.
Fig. 1
Scoring grid used for data collection.
3.4Data collection
Data were collected from May to June 2020. During this period, e-learning was applied at the studied medical school due to the COVID-19 pandemic related closedown. The study’s purpose was explained to the participants by phone call, and their consent was obtained. The survey tool was sent to the participants by e-mail or communication tools such as Viber and WhatsApp. Clear instructions were provided to each participant by e-mail, phone contact, and Zoom meetings. The students were asked to sort the statements into the Q-grid according to their opinions on the research topic’s different aspects. The study protocol was reviewed and approved by the Research Ethics Committee of the authors’ institution.
3.5Data analysis
PQMethod 2.35 program was used for data analysis [47], which involved centroid factor extraction and varimax rotation for determining the prominent common factors. Viewpoints having at least two defining sorts and eigenvalues > 1 were extracted [48]. We applied a conservative significance level (p < 0.01) to determine the significant factor loading on a given viewpoint, which appeared to be 0.424 (2.58 x 1√n, where n = the number of statements) [43]. Different factor solutions were tested to determine the most consistent, meaningful, and coherent factors.
The factors identified represent the sorts made by the students who potentially had similar viewpoints. Each factor was interpreted subjectively based on the characterizing statements (a rank value of +4, +3, –3, or –4) and the distinguishing statements (significantly different scores on the viewpoints) [38]. Finally, a conceptual interpretation was established to define and summarize the obtained viewpoints.
4Results
Thirty-seven medical students participated in the study with a mean±SD age of 21.2±1.8 years (range 19–25 years). The details of the students’ characteristics are provided in Table 1.
Table 1
Characteristics of the participating students
| Characteristics | No. | % |
| Gender | ||
| Male | 12 | 32.4 |
| Female | 25 | 67.6 |
| Study year | ||
| 1st | 5 | 13.5 |
| 2nd | 8 | 21.6 |
| 3rd | 7 | 18.9 |
| 4th | 4 | 10.8 |
| 5th | 4 | 10.8 |
| 6th | 9 | 24.3 |
| Residence | ||
| Main urban | 27 | 73.0 |
| Semi-urban | 8 | 21.6 |
| Rural | 2 | 5.4 |
| Economic situation | ||
| Below average | ||
| Average | 31 | 83.8 |
| Above average | 6 | 16.2 |
| Computer skills | ||
| Poor | 2 | 5.4 |
| Intermediate | 23 | 62.2 |
| Very good | 12 | 32.4 |
Data analysis resulted in having three distinct viewpoints, accounting for 37%of the variance. The first viewpoint was named “Complete dependence on e-learning.” The second viewpoint was named “Opponents of applying e-learning in medical education.” The third viewpoint was named “E-learning as a supplement to medical education.” The rank scores of the statements for the three viewpoints are provided in Table 2.
Table 2
Rank scores of statements for viewpoints on the role of e-learning in medical education during the COVID-19 pandemic
| No. | Statement | Viewpoint 1 | Viewpoint 2 | Viewpoint 3 |
| 1 | During the COVID-19 pandemic, e-learning can replace face-to-face learning for the theoretical part | +4* | –1* | 0* |
| 2 | Poor internet connectivity is an important challenge facing the application of e-learning† | +4 | +3 | +3 |
| 3 | During the COVID-19 pandemic, e-learning can replace clinical training in the skills lab | –4* | –2 | –2 |
| 4 | During the COVID-19 pandemic, e-learning can replace clinical training in hospitals | –4 | –3 | –4 |
| 5 | It is easy to become skillful at using e-learning | +3* | –4* | +2* |
| 6 | I have a good idea about e-learning | +3 | 0* | +3 |
| 7 | I have willingness to continue education during the COVID-19 pandemic | +3 | +3 | –3* |
| 8 | Due to the current pandemic, I prefer postponing all theoretical and practical sessions for the summer | –3* | 0* | –4* |
| 9 | During the COVID-19 pandemic, e-learning can replace practical training in basic sciences labs | –3 | –3 | –1 |
| 10 | We can acquire better skills by e-learning | –3 | –4 | +1* |
| 11 | E-learning is an additional burden for students | –2* | 1* | 0* |
| 12 | During this pandemic, if all the precautionary measures are applied, I would like to attend practical sessions in small groups for clinical training in hospitals | –2* | +4 | +4 |
| 13 | I am interested in guidance on e-learning | +1* | –3* | –1* |
| 14 | If all the precautionary measures are applied during this pandemic, I am ready to return to the college to study. | –1* | +4 | +4 |
| 15 | During this pandemic, if all the precautionary measures are applied, I would like to attend clinical/practical sessions in small groups for the skills lab and other labs | –1* | +2 | +1 |
| 16 | Lack of computer and smartphone is an important challenge facing e-learning | 0* | +2* | –3* |
| 17 | I have some experience in using e-learning methods | 0* | –1* | +2* |
| 18 | Lack of knowledge and information about e-learning is an important challenge facing e-learning | 0* | +1* | –2* |
| 19 | Application of e-learning will widen the gap between poor and rich, rural and urban residents, in terms of accessibility to education | 0 | +3* | 0 |
| 20 | I prefer a combination of e-learning and face-to-face learning | +1 | –2* | +1 |
| 21 | We can attain more knowledge through e-learning | –1 | –2* | 0 |
| 22 | I think there is no any place for e-learning in the medical education | –2 | +2* | –3 |
| 23 | E-learning is easy to use | +2 | –1* | +3 |
| 24 | E-learning has the features of easy accessibility, flexibility, and interactivity | +1 | –1* | +1 |
| 25 | Difficulty in arranging time between teacher and students is an important challenge facing e-learning | 0 | +1* | –1 |
| 26 | Students lack adequate computer skills to be able to use e-learning | –1 | +1* | –1 |
| 27 | Learning to use e-learning is easy | +2 | 0* | +2 |
| 28 | If it is impossible to arrange for practical sessions due to this pandemic, postponing them for the next academic year is the best way to replace them | +1 | +1 | –1* |
| 29 | Lack of adequate computer skills by the teaching staff is an important challenge facing e-learning† | +2 | 2 | +2 |
| 30 | E-learning is cost-effective† | +1 | +1 | +1 |
| 31 | In e-learning, completing a task is more quickly† | 0 | 0 | 0 |
| 32 | E-learning is an interactive mode of learning† | 0 | 0 | –1 |
| 33 | If it is impossible to arrange for practical sessions due to this pandemic, e-learning is the best way to replace them† | –1 | +1 | –2 |
| 34 | E-learning can be a supplemental tool for learning | +2 | +2 | +1 |
| 35 | We have e-learning facility at the institution | –2 | 0 | 0 |
| 36 | E-learning is user friendly | +1 | –1 | 0 |
| 37 | E-learning is useful for group discussion on complex topics | –1 | 0 | –2 |
*Distinguishing statement significant at < 0.01. †Consensus statement.
Twenty-six students (70.3%) defined the three viewpoints. Four participants loaded significantly on more than one viewpoint, and they were considered confounded. Seven participants had not loaded significantly on any viewpoint. The main variables and factor loading for each participant on the three viewpoints are provided in Table 3.
Table 3
Students’ characteristics and factor loading on the three viewpoints
| No. | Age | Gender | Study year | Residence | Economic level | Computer skills | Viewpoint loadings | ||
| Viewpoint 1 | Viewpoint 2 | Viewpoint 3 | |||||||
| 1 | 21 | Male | 3 | Urban | Average | Very good | 0.782X | –0.134 | –0.017 |
| 2 | 20 | Male | 2 | Rural | Average | Poor | 0.704X | 0.040 | 0.222 |
| 3 | 23 | Male | 6 | Urban | Average | Very good | 0.691X | –0.301 | 0.194 |
| 4 | 22 | Female | 5 | Semi-urban | Average | Very good | 0.607X | 0.106 | 0.213 |
| 5 | 21 | Female | 3 | Urban | Average | Intermediate | 0.525X | 0.203 | 0.039 |
| 6 | 22 | Female | 4 | Urban | Average | Intermediate | 0.516X | 0.267 | 0.250 |
| 7 | 23 | Female | 5 | Urban | Above average | Very good | 0.494X | –0.037 | 0.160 |
| 8 | 23 | Female | 6 | Semi-urban | Average | Intermediate | 0.484X | –0.045 | 0.185 |
| 9 | 21 | Female | 3 | Urban | Average | Very good | 0.475X | 0.290 | 0.246 |
| 10 | 24 | Male | 6 | Urban | Average | Intermediate | 0.470X | 0.237 | 0.352 |
| 11 | 22 | Female | 4 | Urban | Average | Intermediate | 0.466X | 0.232 | 0.082 |
| 12 | 23 | Male | 6 | Rural | Average | Intermediate | 0.459X | 0.264 | –0.036 |
| 13 | 22 | Female | 4 | Urban | Average | Intermediate | 0.429X | –0.127 | 0.115 |
| 14 | 19 | Female | 2 | Semi-urban | Average | Intermediate | –0.123 | 0.791X | 0.006 |
| 15 | 19 | Male | 2 | Urban | Average | Intermediate | –0.036 | 0.696X | 0.213 |
| 16 | 19 | Female | 2 | Urban | Average | Intermediate | 0.057 | 0.629X | 0.196 |
| 17 | 20 | Male | 1 | Urban | Average | Very good | 0.087 | 0.600X | –0.176 |
| 18 | 20 | Female | 2 | Semi-urban | Average | Intermediate | –0.125 | 0.585X | 0.097 |
| 19 | 22 | Female | 4 | Urban | Above average | Intermediate | 0.088 | 0.563X | 0.174 |
| 20 | 19 | Male | 2 | Urban | Average | Intermediate | 0.078 | 0.466X | –0.095 |
| 21 | 24 | Female | 6 | Urban | Average | Intermediate | 0.311 | 0.432X | 0.044 |
| 22 | 20 | Female | 3 | Urban | Above average | Very good | 0.187 | –0.013 | 0.610X |
| 23 | 22 | Female | 5 | Urban | Average | Very good | 0.161 | –0.330 | 0.574X |
| 24 | 23 | Female | 6 | Urban | Above average | Intermediate | 0.069 | 0.161 | 0.564X |
| 25 | 21 | Male | 3 | Urban | Average | Very good | 0.300 | 0.233 | 0.503X |
| 26 | 19 | Female | 1 | Urban | Above average | Very good | 0.125 | 0.351 | 0.469X |
| 27 | 24 | Female | 6 | Urban | Average | Very good | 0.637 | –0.036 | 0.464 |
| 28 | 20 | Female | 3 | Urban | Average | Intermediate | 0.555 | –0.429 | 0.255 |
| 29 | 23 | Male | 6 | Semi-urban | Average | Intermediate | 0.463 | 0.319 | 0.450 |
| 30 | 25 | Male | 6 | Semi-urban | Average | Very good | 0.457 | 0.077 | 0.597 |
| 31 | 23 | Female | 5 | Urban | Average | Intermediate | 0.421 | 0.106 | 0.339 |
| 32 | 20 | Male | 3 | Semi-urban | Average | Intermediate | 0.337 | –0.054 | 0.105 |
| 33 | 19 | Female | 1 | Urban | Average | Poor | 0.283 | 0.249 | 0.027 |
| 34 | 19 | Female | 2 | Urban | Average | Intermediate | 0.206 | 0.226 | –0.158 |
| 35 | 19 | Female | 1 | Urban | Above average | Intermediate | 0.157 | 0.005 | 0.292 |
| 36 | 20 | Female | 1 | Semi-urban | Average | Intermediate | 0.082 | –0.012 | 0.339 |
| 37 | 20 | Female | 2 | Urban | Average | Intermediate | –0.062 | 0.380 | 0.296 |
Bold type indicates significant loadings. A significance level of 1%is taken as a factor loading greater than (2.58×1√n), where n = the number of statements. Thus, significant loadings included those higher than 0.424. X indicates defining sorts.
4.1Viewpoint 1 –Complete dependence on e-learning
The first viewpoint accounted for 21%of the total variance. Thirteen students defined this viewpoint, including eight females and five males, four from the 6th study year, three from each of the 3rd and 4th study year, two from the 5th study year, and one from the 2nd study year. Nine students were from the main urban areas, 12 students were from the average economic level, and seven had intermediate computer skills. The distinguishing statements for the first viewpoint are provided in Table 2.
Students holding the first viewpoint emphasized a complete dependence on e-learning to continue medical education and complete the study year with a minimal return to study halls or practical/clinical sessions. These students had a good idea about e-learning (statement 6: score+3 (6:+3)), were interested in guidance on e-learning (13:+1), and had some experience in using e-learning methods (17 : 0). They believed it is easy to become skillful at using e-learning (5:+3).
These students did not think that e-learning is an additional burden for students (11:–2). They thought that poor internet connectivity is an important challenge facing the application of e-learning (2:+4). They had a neutral view about the lack of knowledge and information about e-learning (18 : 0) and the lack of computers and smartphones being important challenges facing e-learning (16 : 0).
These students were not ready to return to the college to study (14:–1), attend clinical training in hospitals in small groups (12:–2), or attend skills lab and other labs in small groups (15:–1) during the pandemic, even if all the precautionary measures are applied. They were willing to continue education during the COVID-19 pandemic (7:+3) and did not agree with postponing all theoretical and practical sessions for the summer (8:–3).
These students thought e-learning could replace face-to-face learning for the theoretical part (1:+4) during the COVID-19 pandemic. However, they did not think that e-learning can replace clinical training in skills lab (3:–4), clinical training in hospitals (4:–4), or practical training in basic sciences labs (9:–3) during the COVID-19 pandemic.
4.2Viewpoint 2 –Opponents of applying e-learning in medical education
The second viewpoint accounted for 11%of the total variance. Eight students defined this viewpoint, including five females and three males, five from the 2nd study year and one from each of the 1st, 4th, and 6th study years. Six students were from the main urban areas, seven had an average economic level, and seven had intermediate computer skills. The distinguishing statements for the first viewpoint are provided in Table 2.
Students holding the second viewpoint had a generally negative view about e-learning and its role in medical education during the COVID-19 pandemic. Compared to the other groups, they least agreed with having a good idea about e-learning (6 : 0) and the simplicity of using e-learning (23:–1) or learning to use e-learning (27 : 0).
These students were least interested in guidance on e-learning (13:–3). They disagreed that it is easy to become skillful at using e-learning (5:–4) and the ability to attain more knowledge through e-learning (21:–2) or acquire better skills by e-learning (10:–4). They indicated that they do not have experience in using e-learning methods (17:–1). They least agreed with e-learning having the features of easy accessibility, flexibility, and interactivity (24:–1). They did not prefer a combination of e-learning and face-to-face learning (20:–2) and thought that there is no place for e-learning in medical education (22:+2).
These students believed that e-learning is an additional burden for students (11:+1). They thought that the application of e-learning would widen the gap between poor and rich, rural and urban residents, in terms of accessibility to education (19:+3). They emphasized the difficulty in arranging time between teachers and students as an important challenge facing e-learning (13:+1). They thought that students lack adequate computer skills to use e-learning (26:+1). They emphasized the lack of computers and smartphones (16:+2), poor internet connectivity (2:+3), and lack of knowledge and information about e-learning (18:+1) as important challenges facing the application of e-learning.
These students indicated that during this pandemic and if all the precautionary measures are applied, they are ready to return to the college to study (14:+4) and attend practical sessions in small groups for the clinical training in hospitals (12:+4). They indicated that they have a willingness to continue education during the COVID-19 pandemic (7:+3) and preferred postponing all theoretical and practical sessions for the summer (8 : 0). They did not believe that e-learning can replace face-to-face learning for the theoretical part (1:–1), clinical training in hospitals (4:–3), or practical training in basic sciences labs* (9:–3).
4.3Viewpoint 3 –E-learning as a supplement to medical education
The third viewpoint accounted for 5%of the total variance. Five students, four females and one male, defined this viewpoint. All of the five students were from the main urban areas. Three students had a low economic level, and two had an average economic level. Four students had very good computer skills, and one had average computer skills. The distinguishing statements for the first viewpoint are provided in Table 2.
Students holding the third viewpoint had a generally positive view about e-learning and considered it a supplement to medical education during the COVID-19 pandemic. They indicated that they have a good idea about e-learning (6:+3) and have some experience using e-learning methods (17:+2). They thought that e-learning is easy to use (23:+3), and it is easy to become skillful at using e-learning (5:+2). They had a neutral view about being interested in guidance on e-learning (13:–1).
These students believed that they could acquire better skills by e-learning (10:+1). They least agreed with having no place for e-learning in medical education (22:–3). They had a neutral view of e-learning being an additional burden for students (11 : 0). They least agreed that lack of knowledge and information about e-learning (18:–2) and lack of computer and smartphone being important challenges facing e-learning (16:–3). They agreed that poor internet connectivity is an important challenge facing the application of e-learning (2:+3).
These students had the willingness to continue education during the COVID-19 pandemic (7:–3). They least agreed to postpone practical sessions for the next academic year (28:–1) and postpone all theoretical and practical sessions for the summer (8:–4). They had a neutral view on replacing face-to-face learning of the theoretical part with e-learning (1 : 0). They most agreed with returning to the college to study (14:+4) and attend practical sessions in small groups for clinical training in hospitals (12:+4) if precautionary measures are applied. They least agreed with having clinical training in hospitals by e-learning (4:–4).
4.4Consensus statements
The three groups of students had a general agreement about several issues. They thought that the lack of adequate computer skills by the teaching staff is an important challenge facing e-learning (statement 29: view1: +2, view2: +2, view 3: +2) (29: +2, +2, +2) and considered poor internet connectivity an important challenge facing the application of e-learning (2: +4, +3, +3). They thought that e-learning is cost-effective (30: +1, +1, +1). They had a neutral view of completing a task more quickly by e-learning (31: 0, 0, 0). They had some neutral to negative views about considering e-learning an interactive mode of learning (32: 0, 0,–1) and considering e-learning the best way to replace practical sessions during this pandemic (33: –1, +1, –2).
5Discussion
During the past decades, medical education witnessed a shift from traditional teaching to other forms using e-learning. However, this transition is not without challenges as new demands are placed on students and teachers [49]. In this explorative study, we assessed medical students’ perspectives on the application of e-learning in medical education during the COVID-19 pandemic. We used Q-methodology features to determine the diversity of the available perspectives about e-learning and its role in medical education during the COVID-19 pandemic. Three distinctive viewpoints were observed, ranging from complete opposition to applying e-learning in medical education to complete dependence on e-learning, even for practical and clinical sessions. The characterization of each viewpoint and the possible reasons for these differences in opinion are discussed below.
The first viewpoint, which was labeled complete dependence on e-learning, emphasized full reliance on e-learning to continue medical education and complete the current academic year with a minimal return to study halls or practical/clinical sessions. Several factors might be responsible for adopting this viewpoint by the students, including the uncertainties about the disease and the pandemic, stress and anxiety from COVID-19 infection, difficulty to adhere to the protective behavior by the students, and the nature of the study in the medical schools in terms of increased contact with patients in hospital training.
The citywide lockdown was enforced in Kurdistan Region of Iraq during the COVID-19 pandemic, with the closure of all learning schools and universities from the beginning of March 2020. It lasted for nearly three months with periods of interrupted, restricted movements. Later, only 30%of public sector workers, including universities, continued to work, while the others continued to work from home. Most universities considered utilizing online media modes of learning to complete the academic year 2019–2020.
Lockdown is a painful experience for those who went through it due to loss of freedom and ambiguity of disease status [50]. Academic performance, pressure to succeed, and fear of being infected if they return to college and hospitals were additional concerns for medical students [51, 52]. This might have contributed to their preference for accomplishing the remaining curriculum through online learning, especially the theoretical part. Furthermore, this viewpoint’s supporters preferred not to attend hospital or skill labs for their clinical training even with all the precautionary measures.
The second viewpoint, which was labeled, opponents of applying e-learning in medical education, emphasized a generally negative view about e-learning and its role in medical education during the COVID-19 pandemic. Factors contributing to this viewpoint might include students’ lack of knowledge and skills of the necessary technology, lack of access to e-learning tools, lack of necessary capacity and facilities in the college, and poor technological skills of the faculties. The same opposition to e-learning was observed in a study in Pakistan, where most medical students preferred face-to-face learning and felt that e-learning has little influence on their learning [53]. The application of e-learning and its integration in medical education might face many challenges, including technological and social challenges [4]. It has been suggested that e-learning widened the gap between different socioeconomic groups in terms of accessibility to e-learning. In developing countries, the lack of computers, smartphones, digital infrastructure, low digital literacy, and electricity were important challenges facing the application of e-learning [54–56]. However, it is essential to highlight the opportunities to address these issues moving forward. For example, the local schools might be able to introduce solutions to this issue of accessibility to devices and/or the internet by providing devices to those unable to obtain them and providing free or discounted internet for students. Moreover, user-friendly website design and integration of self-assessment features are important for learners’ engagement with the online material [57]. However, due to the COVID-19 pandemic, the sudden shift from traditional to online learning was challenging for many academic physicians to develop suitable online material that might not have been attractive for students.
The medical college’s impact on students’ academic performance is based on students’ perspectives instead of the objective reality of the activities and interpersonal relations in the educational environment. These include student input regarding effective learning methods that might result in suitable teaching methods with less cost and more effectiveness. Research has assessed the effect of students’ insight into the universities’ educational process and their validity. Students’ attitudes, satisfaction, gain knowledge, and skills towards the e-learning intervention were considered as an outcome. Several financial and resource-related elements of e-learning were considered as secondary outcomes [58]. In line with the second viewpoint, research from other settings has shown a poor recognition of the benefits of e-learning in medical education by the students [59].
The third viewpoint, which was labeled e-learning as a supplement to medical education, emphasized a generally positive view about e-learning and considered it complementary to on-campus learning during the COVID-19 pandemic. This viewpoint is considered a more rational and balanced viewpoint regarding e-learning and medical education. While some aspects of the study, such as the theoretical and pre-lab session, can be taken by e-learning, most parts of the skills lab and hospital training will require on-campus presence. On-campus study and hospital training can be considered during the pandemic, given the application of the necessary protection and following the proper protection protocols and guidelines. Students with this viewpoint share the perspective of the second viewpoint in that clinical training cannot be done online. However, at the same time, they recognize the possibility of e-learning to substitute the theoretical part. Blended learning is considered more convenient for medical education. There is a need to combine hands-on skills-based training at a practical level and self–directed learning, which is difficult to conduct through online teaching [60, 61]. Students preferred to return to hospitals in small groups with safety precautionary measures applied.
In the current study, a considerable number of students indicated that they were familiar with e-learning tools despite inadequate training. However, they thought that the teaching staff’s lack of adequate computer skills is an important challenge facing e-learning. Poor educator skills were considered as a barrier to implementing e-learning [28]. Medical colleges and universities face challenges in providing quality education to students during the lockdown period. Countries with advanced technology have established systems for e-learning and online education [62]. However, most of the low-income countries like the Kurdistan Region and Iraq do not possess such facilities.
Although e-learning is cost-effective for many developing countries [63], students considered poor internet connectivity an important challenge facing e-learning. This was also described in other studies conducted in developing countries where the lack of institutional support and faculty training, internet connectivity issues, difficulty maintaining student engagement, online assessment concerns, poor performance of the students and hardware, and difficulties in understanding the unique dynamics of e-learning were the major challenges for online learning [62].
E-learning presents many opportunities for universities. However, a combination of online and offline learning (blended learning environment) might be the most effective way to maximize the benefits [64] and allowing more flexibility in delivering medical education [65]. This might explain students’ neutral to negative views about considering e-learning the best way to replace practical sessions during this pandemic.
In the current study, we applied purposive sampling to select the 37 medical students, which is in line with the standard procedures for selecting samples for Q-methodology studies. While there has been a selection of students to ensure diverse and representative backgrounds, there may still be residual selection bias, such as selecting those most vocal, in favor or against e-learning. However, generalization is not usually intended in Q-methodology as it is exploratory in nature. Nevertheless, Q-methodology is aimed to provide a useful insight to the available viewpoints in the society and helps to determine the main characterization of each viewpoint without quantifying them [66]. The channels of distributing the survey tool and communicating with the participants involved e-mail, WhatsApp, and Viber. These tools might be new and innovative, given the lockdown and social distancing situation during the survey. However, data privacy and response and selection bias might have been encountered that might affect the solidity and credibility of the results and the viewpoints.
Another limitation of the current study is using some more objective statements related to having specific knowledge or experience. Q-methodology studies are intended to study subjectivity, and the use of such objective statements might affect our findings. Obtaining a relatively low percentage of explained variance or the shared meaning of the factor solution of the three viewpoints (37%) is another limitation of the current study. The acceptable level of explained variance is generally considered to be 35–40%or above [67], but the percentage of this study is still low compared to many other Q-methodology studies. This study did not include concurrent interviews with the respondents due to the outbreak and lockdown situation and the online administration of the study questionnaire. Concurrent interviewing is commonly applied in Q-methodology and can provide a better understanding of the reasons for each viewpoint. It would have been particularly useful to understand the reasons for viewpoint 2, being opponents of applying e-learning in medical education.
6Conclusion
Three different viewpoints were recognized among the medical students regarding the role of e-learning in medical education during the COVID-19 pandemic. These viewpoints ranged from completely opposing e-learning to using it as a supplement and completely depending on it. The positive and negative aspects of e-learning in medical educations were recognized from the students’ perspectives. The three viewpoints are primarily distinguished by the availability of experience and skills of e-learning, availability of technology, risk perception of COVID-19, and the need for clinical teaching in hospitals. The students’ perspectives reflect the impact of the quick transition from traditional learning to e-learning on the preparedness of faculty, students, and the institution alike. These diverse perspectives need to be considered by the institutions to move forward with the curriculum and the learning process in this and other situations. For strengthening the role of e-learning in medical education, there is a need to provide the students and teaching staff with the necessary skills, proper technology, and platforms; meanwhile, a safe environment is needed for on-campus or hospital clinical teaching.
Conflict of interest
The authors declare no conflict of interest.
References
[1] | World Federation for Medical Education (WFME). Basic Medical Education WFME Global Standards for Quality Improvement. Copenhagen, Denmark; 2015. |
[2] | Genn JM . AMEE Medical Education Guide No, 23 (Part 1): Curriculum, environment, climate, quality and change in medical education-a unifying perspective. Med Teach. (2001) ;23: (4):337–44. |
[3] | Linjawi AI , Alfadda LS . Students’ perception, attitudes, and readiness toward online learning in dental education in Saudi Arabia: a cohort study. Adv Med Educ Pract. (2018) ;9: :855–63. |
[4] | O’Doherty D , Dromey M , Lougheed J , Hannigan A , Last J , McGrath D . Barriers and solutions to online learning in medical education –An integrative review. BMC Med Educ. (2018) ;18: (1):130. |
[5] | Mirmoghtadaie Z , Kohan N , Rasouli D . Determination and Comparison of the Factors Related to Effective Blended Learning in Medical Sciences from the Viewpoints of Instructors and Learners. Adv Med Educ Pract. (2020) ;11: :205–14. |
[6] | Battat R , Jhonson M , Wiseblatt L , Renard C , Habib L , Normil M , Remillard B , Brewer TF , Sacajiu G . The Haiti Medical Education Project: Development and analysis of a competency based continuing medical education course in Haiti through distance learning. BMC Med Educ. (2016) ;16: (1):275. |
[7] | Al-Balas M , Al-Balas HI , Jaber HM , Obeidat K , Al-Balas H , Aborajooh EA , Al-Taher R , Al-Balas B . Distance learning in clinical medical education amid COVID-19 pandemic in Jordan: current situation, challenges, and perspectives. BMC Med Educ. (2020) ;20: :341. |
[8] | Mian A , Khan S . Medical education during pandemics: A UK perspective. BMC Med. (2020) ;18: (1):100. |
[9] | Khalil R , Mansour AE , Fadda WA , Almisnid K , Aldamegh M , Al-Nafeesah A , Alkhalifah A , Al-Wutayd O . The sudden transition to synchronized online learning during the COVID-19 pandemic in Saudi Arabia: a qualitative study exploring medical students’ perspectives. BMC Med Educ. (2020) ;20: :285. |
[10] | Hilburg R , Patel N , Ambruso S , Biewald MA , Farouk SS . Medical education during the COVID-19 pandemic: learning from a distance. Adv Chronic Kidney Dis. (2020) ;27: (5):412–7. |
[11] | Langenau EE , Lee R , Fults M . Blended learning educational format for third-year pediatrics clinical rotation. J Am Osteopath Assoc. (2017) ;117: (4):234–43. |
[12] | Al-Azawei A , Parslow P , Lundqvist K . Barriers and Opportunities of E-Learning Implementation in Iraq: A Case of Public Universities. IRRODL. (2016) ;17: :5. |
[13] | Hussein NR , Saleem M. ZS , Musa DH , Ibrahim N , Naqid IA . Impact of COVID-19 on the Medical Education: Experience from Kurdistan Region of Iraq. J Med Edu. (2020) ;19: (1):e106889. |
[14] | Bin Mubayrik HF . Exploring Adult Learners’ Viewpoints and Motivation Regarding Distance Learning in Medical Education. Adv Med Educ Pract. (2020) ;11: :139–46. |
[15] | Zimmett J . Redesigning (virtual) learning with a “love ethic” in the wake of COVID-19. Education for Information. (2020) ;36: (2):207–9. |
[16] | Visalam , Kumar AP , Om Prakash , Padmvathi R . Knowledge, attitude and practice towards e–Learning among medical undergraduate students. IOSR J Appl Phys. (2015) ;7: (4):1–4. |
[17] | Ghanizadeh A , Mosallaei S , Dorche MS , Sahraian A , Yazdanshenas P . Use of e-learning in education: Attitude of medical students of Shiraz, Iran. Internal Medicine and Medical Investigation Journal. (2018) ;3: (3):108–11. |
[18] | Almarabeh T . Students’ perceptions of E-learning at the University of Jordan. Int J Emerg Technol Learn. (2014) ;9: (3):31–5. |
[19] | Bediang G , Stoll B , Geissbuhler A , Klohn AM , Stuckelberger A , Nko’O S , Chastonay P . Computer literacy and E-learning perception in Cameroon: The case of Yaounde Faculty of Medicine and Biomedical Sciences. BMC Med Educ. (2013) ;13: :57. |
[20] | Mohamed Azmi H , Zeehan SI , Fahad S , Maryam F , Hisham A . Assessment of students’ perceptions towards e-learning management system (e-LMS) in a Malaysian pharmacy school: A descriptive study. Malaysian J Public Heal Med. (2012) ;12: (1):14–20. |
[21] | Barteit S , Guzek D , Jahn A , Bärnighausen T , Jorge MM , Neuhann F . Evaluation of e-learning for medical education in low- and middle-income countries: A systematic review. Comput Educ. (2020) ;145: :103726. |
[22] | Keller C , Cernerud L . Students’ Perceptions of E-learning in University Education. J Educ Media. (2002) ;27: (1-2):55–67. |
[23] | Abbasi MS , Ahmed N , Sajjad B , Alshahrani A , Saeed S , Sarfaraz S , Alhamdan RS , Vohra F , Abduljabbar T . E-Learning perception and satisfaction among health sciences students amid the COVID-19 pandemic. Work. (2020) ;67: (3):549–56. |
[24] | Eldeeb RA . Students’ Perceptions to e-learning. IOSR J Res Method Educ. (2014) ;4: (3):33–6. |
[25] | Ali S , Alam BF , Noreen S , Anwar M , Qazi SH , Hussain T . Motivation and job satisfaction levels among medical and dental faculty members of Pakistan amidst the COVID-19 outbreak. Work. (2021) ;69: (2):359–66. |
[26] | Kim KJ , Kang Y , Kim G . The gap between medical faculty’s perceptions and use of e-learning resources. Med Educ Online. (2017) ;22: (1):10–2. |
[27] | Kim K-J , Kim G . Development of e-learning in medical education: 10 years’ experience of Korean medical schools. Korean J Med Educ. (2019) ;31: (3):205–14. |
[28] | Pettersson F , Olofsson AD . Implementing distance teaching at a large scale in medical education: A struggle between dominant and non-dominant teaching activities. Educ Inf Technol. (2015) ;20: (2):359–80. |
[29] | Toumas M , Basheti IA , Bosnic-Anticevich SZ . Comparison of small-group training with self-directed internet-based training in inhaler techniques. Am J Pharm Educ. . (2009) ;73: (5):85. |
[30] | Buzzell PR , Chamberlain VM , Pintauro SJ . The effectiveness of Web-based, multimedia tutorials for teaching methods of human body composition analysis. Am J Physiol - Adv Physiol Educ. (2002) ;26: (1-4):21–9. |
[31] | Jeffries PR , Woolf S L . Technology-based vs. traditional instruction. A comparison of two methods for teaching the skill of performing a 12-lead ECG. Nurs Educ Perspect. (2003) ;24: (2):70–4. |
[32] | Andersson A , Grönlund Å . A Conceptual Framework for E-Learning in Developing Countries: A Critical Review of Research Challenges. Electron J Inf Syst Dev Ctries. (2009) ;38: (1):1–16. |
[33] | Kelly CM , Vins H , Spicer JO , Mengistu BS , Wilson DR , Derbew M , Bekele A , Mariam DH , Del Rio C , Kempker RR , Comeau DL . The rapid scale up of medical education in Ethiopia: Medical student experiences and the role of e-learning at Addis Ababa University. PLoS One. (2019) ;14: (9):e0221989. |
[34] | Frehywot S , Vovides Y , Talib Z , Mikhail N , Ross H , Wohltjen H , Bedada S , Korhumel K , Koumare AK , Scott J . E-learning in medical education in resource constrained low-and middle-income countries. Hum Resour Health. (2013) ;11: (4):1–15. |
[35] | Goh P-S , Sandars J . A vision of the use of technology in medical education after the COVID-19 pandemic. MedEdPublish. (2020) ;9: (1):49. |
[36] | Zayapragassarazan Z . COVID- Strategies for Online Engagement of Remote Learners. FResearch. (2020) ;9: (246):1–11. |
[37] | Alsoufi A , Alsuyihili A , Msherghi A , Elhadi A , Atiyah H , Ashini A , Ashwieb A , Ghula M , Ben Hasan H , Abudabuos S , Alameen H . Impact of the COVID-19 pandemic on medical education: Medical students’ knowledge, attitudes, and practices regarding electronic learning. PLoS One. (2020) ;15: (11):e0242905. |
[38] | van Exel NJA , de Graaf G . Q methodology: A sneak preview. 2005 [Cited 2020 Oct 15]. Available from: https://qmethodblog.files.wordpress.com/2016/01/qmethodologyasneakpreviewreferenceupdate.pdf |
[39] | Shinebourne P , Adams M . Q-methodology as a phenomenological research method. Existential Analysis. (2007) ;18: (1):103–16. |
[40] | Smith NW . Current systems in psychology: history, theory, research, and applications. Belmont, CA:Wadsworth; (2001) . |
[41] | Cross RM . Exploring attitudes: the case for Q methodology. Health Educ Res. (2005) ;20: (2):206–13. |
[42] | Brouwer M . Q is accounting for tastes. J Advert Res. (1999) ;39: (2):35–9. |
[43] | Stainton Rogers R . Q methodology. In Rethinking methods in psychology. Edited by Smith JA, Harre R, Van Langenhove L. Thousand Oaks, CA: Sage; (1995) . p. 178–92. |
[44] | Mahajan MV , Kalpana R . A study of students’ perception about e-learning. Indian Journal of Clinical Anatomy and Physiology. (2018) ;5: (4):501–7. |
[45] | Tagoe M . Students’ perceptions on incorporating e-learning into teaching and learning at the University of Ghana. International Journal of Education and Development using ICT. (2012) ;8: (1):91–103. |
[46] | Subhash TS , Bapurao TS . Perception of medical students for utility of mobile technology use in medical education. International Journal of Medicine and Public Health. (2015) ;5: (4):305–11. |
[47] | Schmolck P . PQMethod Q methodology analysis program. 2002 [Cited 2020 Oct 7]. Available from: http://www.lrzmuenchen.de/schmolck/qmethod/ |
[48] | Shinebourne P . Using Q method in qualitative research. IJQM. (2009) ;8: (1):93–7. |
[49] | Shachar M , Neumann Y . Differences between traditional and distance education academic performances: A meta-analytic approach. IRRODL. (2003) ;4: :2. |
[50] | Brooks SK , Webster RK , Smith LE , Woodland L , Wessely S , Greenberg N , Rubin GJ . The psychological impact of quarantine and how to reduce it: rapid review of the evidence. Lancet. (2020) ;395: (10227):912–20. |
[51] | Beiter R , Nash R , McCrady M , Rhoades D , Linscomb M , Clarahan M , Sammut S . The prevalence and correlates of depression, anxiety, and stress in a sample of college students. J Affect Disord. (2015) ;173: :90–6. |
[52] | Savitsky B , Findling Y , Ereli A , Hendel T . Anxiety and coping strategies among nursing students during the covid-19 pandemic. Nurse Educ Pract. (2020) ;46: :102809. |
[53] | Abbasi S , Ayoob T , Malik A , Memon SI . Perceptions of students regarding E-learning during Covid-19 at a private medical college. Pakistan Journal of Medical Sciences. (2020) ;36: (COVID19-S4):S57–61. |
[54] | Naresh B , Reddy BS . Challenges and opportunity of E-learning in developed and developing countries-a review. International Journal of Emerging Research in Management & Technology. (2015) ;4: (6):259–62. |
[55] | Naresh B , Reddy DB . An Exploratory Study on Learner’s Perception Towards E-Learning Courses. Mediterranean Journal of Social Sciences. (2015) ;6: (5):562. |
[56] | Naddeo A , Califano R , Fiorillo I . Identifying factors that influenced wellbeing and learning effectiveness during the sudden transition into eLearning due to the COVID-19 lockdown. Work. (2021) ;68: (1):45–67. |
[57] | Schneider SL , Council ML . Distance learning in the era of COVID-19. Arch Dermatol Res. (2020) ;313: :389–90. |
[58] | Baeten M , Kyndt E , Struyven K , Dochy F . Using student-centred learning environments to stimulate deep approaches to learning: Factors encouraging or discouraging their effectiveness. Educ Res Rev. (2010) ;5: (3):243–60. |
[59] | Al-Shorbaji N , Atun R , Car J , Majeed A , Wheeler E . learning for undergraduate health professional education: A systematic review informing a radical transformation of health workforce development. World Health Organization; 2015[Cited 2020 Oct 17]. https://www.who.int/hrh/documents/14126-eLearningReport.pdf?ua=1 |
[60] | Ekenze SO , Okafor CI , Ekenze OS , Nwosu JN , Ezepue UF . The value of Internet tools in undergraduate surgical education: Perspective of medical students in a developing country. World J Surg. (2017) ;41: (3):672–80. |
[61] | George PP , Papachristou N , Belisario JM , Wang W , Wark PA , Cotic Z , Rasmussen K , Sluiter R , Riboli–Sasco E , Car LT , Musulanov EM . Online eLearning for undergraduates in health professions: a systematic review of the impact on knowledge, skills, attitudes and satisfaction. J Glob Health. (2014) ;4: (1):010406. |
[62] | Farooq F , Rathore FA , Mansoor SN . Challenges of Online Medical Education in Pakistan During COVID-19 Pandemic. J Coll Physicians Surg Pak. (2020) ;30: :67–9. |
[63] | Yusuf N , Al-Banawi N . The Impact of changing technology: The case Of e-learning. CIER. (2013) ;6: (2):173–80. |
[64] | Pei L , Wu H . Does online learning work better than offline learning in undergraduate medical education? A systematic review and meta-analysis. Medical Education Online. (2019) ;24: (1):1666538. |
[65] | Warnecke E , Pearson S . Medical students’ perceptions of using e-learning to enhance the acquisition of consulting skills. Australas Med J. (2011) ;4: (6):300. |
[66] | Amin Z . Q methodology-A journey into the subjectivity of human mind. Singapore Med J. (2000) ;41: (8):410–4. |
[67] | McHugh N , Baker R , Biosca O , Ibrahim F , Donaldson C . Who knows best? AQ methodology study to explore perspectives of professional stakeholders and community participants on health in low-income communities. BMC Health Serv Res. (2019) ;19: :35. |
