Using cartoon videos to survey children and adolescents in the global south: A Tanzanian example

1.Introduction

One out of every eight Africans is a young person between the ages of 12 and 17. These older children, with the ability to speak and reason, are rarely survey respondents and are instead measured (weight, healthiness, standardized tests) as if adults’ only interest in them is instrumental. Proxy respondents, often parents, are assumed to know everything else relevant about a child. Yet the United Nations Convention on the Rights of the Child (UNCRC [1]), in Article 12, states that children (that is, people under the age of 18) have the right to be heard on matters that concern them:

If statistics are truly to “leave no one behind,” then children’s experiences and perspectives need to be captured in the next generation of nationally-representative statistics, including those produced by National Statistical Offices (NSOs). It is not enough to count and identify this population. It is imperative that children be included on a large scale to consider the effects of policies that are implemented on their behalf, and to assess whether they are in children’s best interests. The perspectives of older children are especially relevant for adults to understand, as the adolescent years often include major life decisions and transitions. How are young people thinking about the challenges common in their societies?

Even those desiring to learn more from children and adolescents (under age 18) have been hesitant to include them in survey research due to their vulnerability, which makes them a protected group from the perspective of institutional review boards (IRBs). The standard alternative, proxy responses by parents or other adults, are unlikely to capture young people’s thinking, especially on sensitive subjects. School-based studies miss out-of-school populations and have other difficulties.11 Most studies that aim to understand children’s lives and perspectives do so with qualitative approaches, typically including small numbers of children and especially underrepresenting those in the Global South. Studies with children are able to identify topics where young people’s viewpoints align with those of many adults, and topics where they do not align. While qualitative research undoubtedly provides the clearest understanding of children’s perspectives and actions, it is time-consuming, expensive, and almost never representative at a population level. To include the large numbers of Africans who are ages 12–17 in surveys, we argue that official statisticians need another tool to engage with young people and, with more representative data, enhance local capacities to reach Sustainable Development Goals (SDGs). In this paper we demonstrate that it is possible to use modern technology to include older children in larger representative surveys via a low-risk yet inexpensive alternative to standard survey instruments, with the goal of expanding understanding of children’s perspectives and thus enhancing their human rights.

Animating Children’s Views (ACV) is a survey research tool for use with young people ages 12–17. ACV uses a modified vignette format that does not presume literacy. To reduce the risk that nearby adults will disagree with and punish children, respondents watch and listen to short (2–3 minute) cartoon videos shown on tablet computers such as those commonly used by enumerators. Recorded voiceovers in the local language are heard via headphones that provide privacy, followed by survey questions referencing the stories rather than the young respondent’s own life. This kind of indirect questioning, about the vignette situations, makes it possible for children to feel safer even while revealing their perspectives [2]. This method can be tailored to be appropriate to different contexts and topics; an online archive of images will be made available to users to construct different stories. ACV enables low-cost, large-scale assessments of the views of 12–17-year-old children on matters that affect them, from education to health to sexuality to livelihoods issues, including many important topics covered by the Sustainable Development Goals. We report on exploratory research and two pilots conducted in households in rural and urban Tanzania in 2018, with passing mention of a third pilot that took place in Nepal in 2019. Analysis of follow-up interviews and focus groups helped us interpret and validate the quantitative results.

Official statistics play an important role in highlighting the situations and experiences of vulnerable groups. We argue that until children’s perspectives are included in large-scale survey research, they are unlikely to be treated as full human beings by adults designing and implementing policies and programs. Scholars argue that generational power and bias exists yet is typically invisible to those with such power – adults – in the same way that racial privilege may be invisible to Whites in the authors’ country among others (e.g., [3, 4]). Adults are limited in their capacity to find solutions to concerns involving children when they conduct research in ways that exclude or ignore children and their experiences. The UNCRC declares that children have the right to be heard, presumably by adults in general and adult policy-makers in particular. Yet without a way to implement Article 12 on a large-scale, it remains largely theoretical. Finding funding to innovate in ways that can better include such groups is always a challenge, yet it has been done time and again. Regarding the inclusion of young people speaking for themselves, there is much yet to learn.

Survey researchers know that understanding opinions has a whole host of challenges; interviewing children has an additional set of difficulties [5, 6]. We focus on older children, ages 12–17, following deLeeuw’s [7] guidance about age-related linguistic development; we use language that is appropriate for children age 10 or older but only include ages 12+ because of variability in child development. Besides physiological considerations, age-based power differentials affect what can be learned by simply asking children for their perspectives. In Tanzania, our qualitative data indicate that children are generally not “free” (as they say) to initiate or engage in conversations with adults. The adults with whom young people spend the most time are authority figures: parents or guardians and teachers. When questioned by an adult, children know that there is usually a “correct” answer, and that it would be in their interests to provide that answer. This might be considered “adult-pleasing behavior” to distinguish it from more usual aspects of social desirability bias [8]. Saying the wrong thing in response to an adult’s question can be problematic for young people: it can affect how they are viewed by higher status (older) people and can lead to reprimands and/or punishment. For the survey researcher, the fact that children are susceptible to retribution by their elders for overheard survey responses makes them an especially vulnerable population.

Because at ages 12–17 young people are at the peak of their conventionality,22 social desirability bias – defined here as systematic bias in data as a result of respondents attempting to answer questions in a way that is “correct” or socially acceptable to the interviewer [9] – may be more than usually prominent. However, just because a response is socially desirable does not mean that the respondent is changing her/his response to be consistent with perceptions of what an adult interviewer wants to hear. It might be that the respondent truly believes the discourse of her/his culture about what is desirable. In Tanzania, for example, many people – adults and children – repeat and appear convinced of the truth of the Swahili proverb, “Education is the key to life.”33

We do not argue that the ACV methodology is best for all purposes. We simply have not found any other techniques that fulfill the three requirements of (1) being understandable to (older) children but not presuming literacy; (2) being usable on the scale at which NSOs work and (3) not increasing risk to vulnerable child respondents. In the future, when assuming literacy is more reasonable, young people may be able to be handed a device and allowed to answer their own questions directly. Until then, indirect questioning through vignettes provides a way to mitigate the challenges of social desirability, adult-child power dynamics, and adult pleasing behaviors present in other traditional survey designs. Our goal was to create an innovative, cost-effective methodology for use with large-scale representative surveys while also ensuring reliable data on children’s perspectives. Essentially, we aim to represent older children’s (qualitative) perspectives in a quantitative way, with the help of modern technology. To this end, our pilot studies needed to be mixed methods. After three smaller randomized pilots mimicking large household surveys by including an adult survey along with the child modules, this new methodology is ready for scaling up. Applying the ACV methodology in conjunction with nationally representative surveys can help NSOs in Africa and elsewhere improve policies targeting children at the local and national level.

2.ACV methodology

Large-scale survey research in the Global South has transitioned in recent years from paper surveys to the use of hand-held devices (tablet computers) to record respondents’ answers to questions. Producing short videos has also become relatively inexpensive in general as well as accessible via free software, making it affordable for survey research. The use of images, animation, and audio recordings to convey a story also allows illiterate children to participate more fully than they might via written questionnaires. Animating Children’s Views uses a modified vignette methodology in the form of short video stories. Child respondents can decide what they think should happen next in a video’s storyline, thus revealing their perspectives through this indirect questioning. In addition, their suppositions about what is most likely to happen next (after the end of the story) provides information about their experiences and about prevalent discourses – how people in their communities are talking about something. ACV innovates by attaching video cartoon stories and follow-up questions for young people to a standard adult survey; in our field research we modeled the adult survey on typical household surveys.

Vignettes have been growing in popularity as a research tool in qualitative and quantitative methods, and research has validated the method when respondents are children and adolescents [11]. Our story themes were initially selected to be illustrative of topics for which NSOs and researchers might use ACV methodology. Development of vignettes was done iteratively to make sure the stories would be culturally relevant. Initially we drew upon qualitative literatures in sociology, education, child rights, and geography, among others, as well as the expertise of our colleagues to create stories that would be appropriate for a pilot study in Tanzania. For example, Mojola [12] found that Kenyan girls turn to older boyfriends when they feel themselves in need of money, so that became an element of a vignette. Another study documented that boys on Zanzibar (Tanzania) tended to drop out before completing secondary school because of the lack of job possibilities, so that became a story line [13]. Our approach has been to describe challenging situations faced by young people in the vignettes, and then end each vignette on a question: what should the main character (a young person) do? Each vignette describes some possible outcomes of the situation before ending. In Tanzania, those outcomes were developed in part with local collaborators, using exploratory research. A flowchart documenting the story development process is available on the ACV website.44

Since we proposed to use images, finding a style of image that could be used in multiple country contexts was a priority. In 2017, Bolgrien consulted with children in Sierra Leone, exploring how children would respond to and critique different image styles. We intended to avoid details about facial structure, body type, hair or hair styles, skin color, clothing, and footwear (for example) that may indicate community membership and socio-economic status. Because such details can influence responses in complicated and invisible ways, we began experimenting with minimalist styles. The children who see these images may not be used to cartoons, so we aimed to avoid the use of standard cartoon conventions (for example, thoughts in bubbles, or !! over a head to indicate surprise). The final ACV cartoon style, drawn by artist Hillary Carter-Liggett and illustrated in Fig. 1, avoids as many details as possible while indicating movement and, sometimes, emotion. Instead of fully animated movement, we used a combination of still images (with fade-ins and fade-outs) and occasional animated scenes. We have committed to making a free or open source library of images and animated scenes that NSOs and other researchers can use to create a variety of vignettes.55

Figure 1.

(a) Sick parent; (b) playing soccer.

Another innovation, again driven by a desire to provide respondents as much privacy as possible, was to use a modified 5-point “Smiley scale” to capture young people’s initial responses to possible outcomes to a cartoon story, as shown in Fig. 2. Smiley scales have been used by health care researchers, to record, for example, pain levels of young patients (e.g., [14]). The Smiley scale was printed and laminated, allowing respondents to hold it and point at responses rather than say them out loud. Future pilots of ACV methodology aim to test the feasibility of child respondents independently operating the survey software with pre-recorded audio for the question and possible responses. This would maximize privacy but so far has been incompatible with survey software used by our partner organizations. One of our goals is for this tool to be inexpensive for others to adapt.

Figure 2.

Smiley scale.

Other follow-up questions were unable to avoid the necessity of an oral answer, as discussed below. However, even those adults who refused to stay at a distance from young respondents – perhaps to protect them from unknown survey researchers – could not tell what situation the questions referenced. For a few potentially sensitive questions that might be understood by listeners, the field researchers had a pre-programmed option: “Skip question because someone is listening.” It was used a small number of times. We had no situations where young people were put at risk, to our knowledge; this was confirmed in a small follow-up study, discussed below.

3.ACV methodology pilots

To test the ACV survey methodology, we developed and piloted child modules in Tanzania in 2018. A later pilot was conducted in Nepal in 2019. Vignette topics were selected based on country- or region-specific reports and studies; topics were modified after exploratory research in June 2018. While we developed seven vignettes for Tanzania and used all seven in exploratory research, we focused on four of them for the survey component of the first pilot conducted in July 2018 in a rural village in Tanzania, in order to have greater sample sizes for each one (see Table 1). Most had separate voiceovers (in Swahili) by gender: if the respondent were male, he would hear a male voiceover describing a boy as the main character, and similarly for females. The four video stories used in the rural survey involved a child with a sick father, a student being beaten at school, a teacher making (sexual) advances to a schoolgirl, and a teen pregnancy scenario. Each video had follow-up questions, then another video could be watched, with the vignette order randomized. Not all respondents watched all four videos; some had after-school chores they needed to get to, for example. In the second (December 2018) pilot in an urban area, we included three vignettes in a fixed order: a newly-developed story about child labor, the pregnancy story, and the story about being beaten at school.

While this paper will focus on the two Tanzanian pilots, examples from a 2019 Nepali pilot illustrate the flexibility of the ACV methodology. For peri-urban Kathmandu, vignette stories were developed in collaboration with a local parther and in consultation with a non-governmental organization (NGO) that works with children. The challenge of cultural relevance and appropriateness implies potentially different stories for any given country or region. We found that in Nepal, schoolgirls becoming pregnant was not common and the topic was not considered appropriate, although in rural areas early marriage would have been a reasonable story. Similarly, peri-urban schools included much more verbal than physical punishment of children, so we modified the Tanzanian story about a teacher beating a student into a Nepali story about a teacher verbally abusing a student. We were able to use the story about street harassment of girls from Tanzania, and we added two completely new vignettes about bullying and family violence. We were concerned that the family violence story would be high-risk but were encouraged by the response of NGO staff. A protocol was put in place in the event of field researchers talking to a young person who desired help in a family violence situation.

All the vignettes that we developed were, per the UNCRC’s Article 12, on topics that young people might very well be thinking or even worrying about. Each one set up a challenging (problematic) situation involving a character who was clearly a young person; each one talked about possible outcomes or next steps; and each one ended with a question: what should this “cartoon kid” do? We purposefully did not include “easy outs” (obviously best outcomes) among the possibilities for the cartoon kid, to make the story as realistic as possible, while also allowing some room for young respondents to come up with their own solutions via an open-ended option to at least one question. The Smiley scales immediately followed each video, with questions asking how the cartoon kid felt about each option presented in the vignette. These were followed by a series of questions including, first, “What should [the cartoon kid] do?” In rural Tanzania (only) we next asked “What is the best outcome in this situation?” thinking that this might give the respondent a chance to give the socially desirable answer. This was followed by “People do not always do what is best. Around here, what is the likely outcome of this situation?” Additional questions, such as two asking who would decide which outcome would be selected and whether the respondent had ever known someone (including her-/himself) in that situation, helped us understand more about the respondent’s familiarity with the situation. Post-video questions sometimes also addressed issues of policy.

To simulate the experience of using this tool with a large-scale survey, and also to gain contextual information to use in analysis, we conducted a small-scale randomized survey in a large rural village in Tanzania in July 2018. We returned to Tanzania in November-December 2018 to conduct a similar urban study in the city of Arusha. A third study took place on the outskirts of Kathmandu, Nepal, in May-June 2019. In all three pilots, the same basic process was followed: After a household was sampled66, the mother or another adult was interviewed and was asked for consent to interview children ages 12–17 from that household. The field researcher looked for a quiet and private place to interview the child (if assent was given), preferably in view but not hearing of others. The child respondent was given a tablet and a headset and shown how to use them. After watching one video, and listening to the voiceover in Swahili or Nepali, the respondent answered questions about it; she/he then watched up to two or three more videos, with follow-up questions after each. Field researchers held other tablets on which they entered survey responses using SurveyToGo software. (It would be possible, however, to have videos and the survey software on one device, handed back and forth.) A small number of survey questions ended the child module (e.g., confirming level of education to compare to proxy report; confirming birth order). Selected respondents were invited to participate in qualitative interviews (mostly very short) and same-sex focus groups. The qualitative research was vital to the validation of the survey responses, and mixed method examples of our results are discussed below. Table 1 describes the numbers of participants in each aspect of the Tanzanian research.

A number of strategies were used to reduce the adult interviewer/child respondent gap in status and power and thus (ideally) the amount of adult-pleasing behavior and other social desirability bias, especially for the qualitative research. One strategy involved conducting interviews at children’s homes instead of in a school setting, to reduce “pleasing the teacher” responses – that is, children saying what they guess is “correct” rather than what they think. As another strategy, field researchers stressed that “there are no correct answers and no wrong answers” in their initial conversations with children. They also used informal language and forms of address with young respondents, and they aimed at seating arrangements to minimize power dynamics (see [15]).

Because of our interest in potential risk to child respondents, we did a small follow-up study of urban Tanzanian child respondents who had been interviewed in December 2018. In June 2019 members of the original field research team were able to locate and re-interview 37 young people. To avoid leading questions about risk, interviewers asked what had happened after the original interview. Most respondents felt that the original interview was a neutral experience and told the field research they had not experienced any reactions after participating. About half of the respondents reported that they had discussed the stories with their parents. Only five respondents reported not discussing the original interview with anyone. Of the eight respondents who reported having a negative memory of the original interview, all eight were referring to feeling sad or angry about the difficulties faced by the cartoon character. Based on detailed debriefing with the follow-up team, it seems unlikely that any of the re-interviewed participants felt at increased risk during or after the interview process. There is selection bias in the re-sampling process, so it is impossible to universally rule out increased personal risk to young respondents, but we found no indication of such.

4.Methodological findings

Analysis of quantitative and qualitative leads to a number of conclusions about the ACV methodology:

Finding (1): Young people understand the images and can follow the videos. Children were able to repeat stories back in detail, in exploratory research. When specifically asked about the images used in the videos, in focus groups, children reported that it was easy to understand what was going on based on the images and the audio. There were a small number of cases where field researchers felt that the story had not been understood. In debriefings we sometimes learned of indications that such respondents had a cognitive disability. These cases were outliers.

Finding (2): Vignette situations were familiar. That is, similar situations occurred in real life in the locality. This was important insofar as we hoped to learn what young people thought about situations about which they had already had time to develop their own perspectives. Survey research is sometimes criticized for asking respondents to instantly form an opinion on a topic that they had not previously come across. Thus, we aimed to include stories about situations so common that most if not all young respondents would have heard about similar situations, or possibly experienced them.

Quantitative evidence on this finding comes from the question about whether the respondent had ever known anyone (including him- or herself) in that situation. Percentages ranged from a low of 16% in a story about a male teacher making sexual advances to a female student to a high of 54% for a story about a student being beaten at school. This question did not ask, “Have you heard of this happening to anyone?” but rather, did the respondent personally know someone who had experienced this. With that in mind, we consider these percentages quite high. Qualitative responses show that respondents had also heard of these situations happening, even if they did not personally know someone in a similar situation. For example, regarding the pregnancy story:

Finding (3): Vignette dilemmas were (mostly) understood. During the pilots, field researchers debriefed at the end of each day. If a respondent appeared to have difficulty understanding part of a story, this was apparent from his/her responses to the survey and (sometimes) a post-survey interview. Most frequently, these (rare) cases involved 12–13 year-old boys who did not have a correct understanding of human reproduction, or for whom our phrasing was too indirect. For example, as one possible outcome in the pregnancy scenario, the video suggests that a girl could “get herbs or medicines to bring back her period” instead of mentioning abortion directly. This tactic of using euphemisms, as discussed by Suh [16] and Plummer et al. [17], allowed us to avoid the use of the word abortion in out-loud follow-up questions. While some of the youngest boys did not understand this part of the story, other qualitative data have convinced us that the wording was understood by female and older male respondents.

This finding can also be seen quantitatively. For example, in the story involving a teacher making sexual advances to a schoolgirl – and threatening to fail her on an important exam if she did not cooperate – one of the options was “do what the teacher wants.” Out of 88 respondents in the rural survey, six were “happy” or “very happy” about this option – and five of the six were 12–13 year-olds. They most likely did not grasp that this meant the student should have sex with her teacher. The rest of sample seems to have grasped the implications of this part of the story – and indicated a non-happy response.

Because boys are known to developmentally lag girls, and because this gap peaks about age 12, it is not surprising that it is the youngest boys are most likely to occasionally misunderstand (e.g., [18]). This may be accentuated if even young girls experience sexual harassment, or if girls are given some sex education by family members upon hitting puberty.

Finding (4): The child module is low-risk. This conclusion is based on debriefing notes, observations in both rural and urban field research, and a small follow-up survey for the urban sample, discussed above. There were absolutely no indications that participating in the survey put a respondent at increased risk of harm. The use of the “skip this question because someone is listening” option in urban Tanzania suggests that it is possible that responding to some kinds of questions, if there is a listener, could be problematic.77

Finding (5): The child module is time-efficient. We structured our child survey to mimic a module added to a household survey. Child respondents were asked several basic demographic questions to establish rapport and create familiarity with tablet computers. The vignettes were presented first by asking the young person to watch a short (1–3 minutes long) video and then to answer six to 16 questions about it. On average, each video and question combination took between five and eight minutes. Respondents were asked if they would like to watch another vignette at the end of each set of questions, and most assented to watch all the videos we had prepared. In the Tanzania pilots, participating in the ACV child module took approximately 30 minutes total and included four vignettes in the rural pilot and three vignettes in the urban pilot. The duration of the ACV modules depends greatly on the length of the cartoon video and the number of survey questions for each vignette. Compared to the long household and labor force surveys that adults typically respond to, adding one or more vignettes would not substantially increase time spent in a household, but it can greatly improve the data capturing adolescent perspectives in survey research.

Limitations. The rural Tanzania (July) sample was conducted while school was in session, and a number of older children could not be interviewed because they were in boarding school. Thus, our July sample is weighted towards younger children. For this reason, we interviewed the urban sample in December, when many (but not all) students were on holiday. While this did not solve another difficulty – the challenge of finding out-of-school young people who are working and living independently – it improved the age distribution. Another limitation is that people with hearing disabilities may not be able to hear video voiceovers.

5.Substantive findings

What can be learned using ACV child modules? The following examples show the kinds of substantive findings that can result from analysis of post-vignette questions. The ACV methodology is intentionally flexible and able to be adapted for any number of topics where children’s perspectives may be unique from adults. These examples represent two very different topics, reproductive and sexual health and educational policy. They show the complex understanding and nuance that children’s perspectives can bring to the conversation.

6.Conclusion

It used to be considered appropriate to collect data only from men, with the thinking being that men could speak for (and better than) the women in their households. Now it is understood that female respondents are necessary to generate needed data for a well-rounded policy discussion. Similarly, we argue that adults cannot and should not be considered adequate proxy respondents for children who are old enough to participate in survey research. The UNCRC says that we should listen to children about matters that affect them, and the SDGs indicate a number of topics that clearly affect young people. However, because they are especially vulnerable, additional care must be taken not to put child respondents at risk during an interview.

For a multitude of topics on which proxy responses are inadequate and direct questions are too risky for child and adolescent respondents, Animating Children’s Views modules can be attached to large representative surveys. “Children should be seen and not heard” is a proverb that has lost its resonance as it denies young people their human rights. Adults, moreover, need to understand the reasons behind the behaviors of children and adolescents to be able to effectively strategize responses to them. For example, beyond the number of out-of-school children, knowing motivations for dropping out of school can improve policies aimed at economic development. We argue that national statistical offices should add ACV child modules to large, representative surveys, thereby becoming leaders in inclusive survey designs.

The goal of the ACV project was to develop a methodology rather than answer any particular substantive question. The best way to use the methodology is to start with a hypothesis or question about an issue of concern, develop and test questions with close-ended responses that might help adults understand the way young people are thinking about the issue – while always leaving space for unexpected responses or ways of thinking – then develop a vignette that would provide a framework for those questions. Thus far, vignette development has been driven by common challenging situations rather than by particular questions about them. We aimed to show the ability to use stories to address different types of questions, such as those concerning health and sexuality, education, gender discrimination, labor market issues, and others. Our bullying and sexual harassment stories could be adapted to a variety of abusive situations, including within the workplace. A family violence story suggests that ACV can be used even with extremely sensitive subjects. Additional topics might include safety in conflict zones or in refugee camps, political radicalization, food insecurity, mental health, disability, drug/alcohol abuse, and social media and technology use. Many of these topics are of great interest to African policy-makers. Without enthusiastic support from official statisticians, however, policy-makers are unlikely to understand the relevance of input from young people or the importance of providing resources to support appropriate methods for collecting data from such vulnerable groups. Their experiences and concerns should be given weight in local and regional policy debates; young people are the future leaders of Africa.