Rey–Osterrieth Complex Figure – copy and immediate recall: Normative data for the Latin American Spanish speaking adult population

2.1Participants

The sample consisted of 3,977 healthy individuals who were recruited from Argentina, Bolivia, Chile, Cuba, El Salvador, Guatemala, Honduras, Mexico, Paraguay, Peru, and, Puerto Rico. The participants were selected according to the following criteria: a) were between 18 to 95 years of age, b) were born and currently lived in the country where the protocol was conducted, c) spoke Spanish as their native language, d) had completed at least one year of formal education, e) were able to read and write at the time of evaluation, f) scored ≥23 on the Mini-Mental State Examination (MMSE, Folstein, Folstein, & McHugh, 1975), g) scored ≤4 on the Patient Health Questionnaire– 9 (PHQ-9, Kroenke, Spitzer, & Williams, 2001), and h) scored ≥90 on the Barthel Index (Mahoney & Barthel, 1965).

Participants with self-reported neurologic or psychiatric disorders were excluded due to a potential effect on cognitive performance. Participants were volunteers from the community and signed an informed consent. Nine participants were excluded from the analyses, with a final sample of 3,968 participants. Socio-demographic and participant characteristics for each of the countries’ samples have been reported elsewhere (Guàrdia-Olmos, Peró-Cebollero, Rivera, & Arango-Lasprilla, 2015). The multi-center study was approved by the Ethics Committee of the coordinating site, the University of Deusto, Spain.

2.2Instrument administration

The examiner administered the ROCF Figure A (copy), and after 3 minutes, the immediate recall. The Spanish-language ROCF manual scoring guidelines were followed (Rey, 2009). The ROCF includes 18 elements, and the maximum score for each of the two tasks (copy and immediate recall) is 36. Two points are given when the element is correctly reproduced, 1 point when the reproduction is distorted, incomplete but placed properly, or complete but placed poorly; 0.5 point is credited when the element is distorted or incomplete and placed poorly. A 0 score is given when the element is absent or is not recognizable (Osterrieth, 1944).

2.3Statistical analyses

The detailed statistical analyses used to generate the normative data for this test are described in Guàrdia-Olmos, et al., 2015. In summary, the data manipulation process for each country-specific dataset involved five steps: a) t – tests for independent samples and effect sizes (r) were conducted to determine gender effects. If the effect size was larger than 0.3, gender was included in the model with gender dummy coded and female as the reference group (male = 1 and female = 0). b) A multivariable regression model was used to specify the predictive model including gender (if effect size was larger than 0.3), age as a continuous variable, and education as a dummy coded variable with 1 if the participant had >12 years of education and 0 if the participant had 1– 12 years of education. If gender, age and/or education was not statistically significant in this multivariate model with an alpha of 0.05, the non-significant variables were removed, and the model was re-run. Then a final regression model was conducted that included age (if statistically significant in the multivariate model), dichotomized education (if statistically significant in the multivariate model), and/or gender (if effect size was greater than 0.3) [yˆi=β0+(βAge·Agei)+(βEduc·Educi)+(βGender·Genderi)]; c) residual scores were calculated based on this final model (ei=yi-yˆi); d) using the SD (residual) value provided by the regression model, residuals were standardized: z = e _i /SD _e , with SD _e (residual) = the standard deviation of the residuals in the normative sample; and e) standardized residuals were converted to percentile values (Strauss et al., 2006). Using each country’s dataset, these steps were applied to ROCF copy scores and ROCF immediate recall scores.

3.1ROCF copy

Regarding the effect of gender on the ROCF copy scores, the t-tests showed significant differences between men and women in the countries of Bolivia, Honduras, Mexico, and Puerto Rico; however, none of these four countries had an effect size larger than 0.3. Table 1 shows the results of the gender analyses by country on the ROCF copy scores. As shown in Table 1, the effect sizes for all countries were less than 0.3, and therefore gender was not taken into account to generate ROCF copy normative data for any of the countries in the study.

The final eleven ROCF copy multivariate linear regression models for each country are shown in Table 2. In all countries, the ROCF copy score increased for those with more than 12 years of education (see Table 2) and, in all countries, ROCF copy scores decreased in a linear fashion as a function of age. The amount of variance explained in ROCF copy scores ranged from 7% (in Argentina) to 34% (in Paraguay).

3.2ROCF immediate recall

Regarding the effect of gender on the ROCF immediate recall scores, the t-tests showed significant differences between men and women in the countries of Argentina, Bolivia, Chile, Cuba, Honduras, Mexico, Paraguay, and Puerto Rico. Table 3 shows the results of the gender analysis by country on ROCF immediate recall scores. As shown in Table 3, the effect sizes for all countries except Honduras were less than 0.3, and therefore gender was only taken into account to generate the ROCF immediate recall normative data for the Honduras sample.

The final eleven ROCF immediate recall multivariate linear regression models for each country are shown in Table 4. In all countries, ROCF immediate recall score increased for those with more than 12 years of education (see Table 4) and decreased in a linear fashion as a function of age. The amount of variance explained in ROCF immediate recall scores ranged from 21% (in Guatemala) to 41% (in El Salvador).

3.3Normative procedure

Norms (e.g., a percentile score) for the ROCF copy and immediate recall scores were established using the five-step procedure described above. To facilitate the understanding of the procedure to obtain the percentile associated with a score on this test, an example will be given. Suppose you need to find the percentile score for a Chilean woman, who is 43 years old and has 14 years of education. She has a score of 30 on the ROCF copy test. The steps to obtain the percentile for this score are: a) Check Table 1 to determine if the effect size of gender in the country of interest (Chile) on this test and time point (ROCF copy) is greater than 0.3 by country. The column labelled r in Table 1 indicates the effect size and the superscript notation b next to the number indicates that the number is larger than 0.3. In this example, the effect size is 0.072, which is not greater than 0.3. For Chileans on this test, gender does not influence scores to a sufficient degree to take it into account when determining the percentile. b) Find Chile in Table 2, which provides the final regression models by country for ROCF copy. Use the B weights to create an equation that will allow you to obtain the predicted ROCF copy score. The corresponding B weights are multiplied by the actual age and dichotomized education scores and added to a constant in order to calculate the predicted value. In this case, the predicted ROCF copy score would be calculated using the equation [yˆi=36.459+(-0.164·Agei)+(2.793·DichotomizedEducationalLeveli)] (the values have been rounded for presentation in the formula). The subscript notation i indicates the person of interest. The person’s age is 43, but the education variable is not continuous in the model. Years of education is split into either 1 to 12 years (and assigned a 0) or more than 12 years (and assigned a 1) in the model. Since our hypothetical person in the example has 14 years of education, her educational level value is 1. Thus the predicted value is 36.459 + (-0.164 · 43) + (2.793 · 1) =36.459 - 7.052 + 2.793 = 32.2. c) In order to calculate the residual value (indicated with an e in the equation), we subtract the actual value from the predicted value we just calculated (ei=yi-yˆi). In this case, it would be e _i  = 30 - 32.2 = -2.2. d) Next, consult the SD _e column in Table 2 to obtain the country-specific SD _e (residual) value. For Chile, it is 7.430. Using this value, we can transform the residual value to a standardized z score using the equation (e _i /SD _e ). In this case, we have (-2.2)/7.430 = -0.296. This is the standardized z score for a Chilean woman aged 43 and 14 years of education and a score of 30 on the ROCF copy test. e) The last step is to look up the tables in the statistical reference books (e.g. Strauss et al., 2006) or use a trusted online calculator like the one available at http://www.measuringu.com/pcalcz.php. In the online calculator, you would enter the z score and choose a one-sided test and note the percent of area after hitting the submit button. In this case, the probability of -0.296 corresponds to the 38th percentile. Please remember to use the appropriate tables that correspond to each test (copy vs. immediate recall) when performing these calculations. If the percentile for the ROCF immediate recall score is desired, Tables 3 and 4 must be used.

3.4User-friendly normative data tables

The five-step normative procedures explained above can provide more individualized norms. However, this method can be prone to human error due to the number of required computations. To enhance user-friendliness, the authors have completed these steps for a range of raw scores based on small age range groupings (see Guàrdia-Olmos et al., 2015) and created tables that clinicians can more easily use to obtain a percentile range associated with a given raw score on this test. These tables are available by country and type of test (ROCF copy vs. ROCF immediate recall) in the Appendix. In order to obtain an approximate percentile for the above example (converting a raw score of 30 for a Chilean woman who is 43 years old and has 14 years of education) using the simplified normative tables provided, the following steps are recommended. (1) First, identify the appropriate table ensuring the specific country and test. In this case, the table for the ROCF copy scores for Chile can be found in Table A3. (2) Note if the title of the table indicates that it is only to be used for one specific gender. In this case, gender is not specified. Thus Table A3 is used for both males and females. (3) Next, the table is divided based on educational level (1 to 12 vs. more than 12 years of education). Since this woman has 14 years of education, she falls into the >12 years of education category. These data can be found in the top section of the table. (4) Determine the age range most appropriate for the individual. In this case, 43 falls into the column 43– 47 years of age. (5) Read down the age range column to find the approximate location of the raw score the person obtained on the test. Reading down the 43– 47 column, the score of 30 obtained by this Chilean woman corresponds to an approximate percentile of 40.

The percentile obtained via this user-friendly table method (40th) is slightly different than the more exact one (38th) obtained following the individual conversion steps above because the table method is based on an age range (e.g., individuals aged 43– 47) instead of the exact age (individuals aged 43). If the exact score is not listed in the column, you must estimate the percentile value from the listed raw scores.

4.1Limitations and future directions

The current study has several limitations, and as a result directions for future research. First, although the study was conducted in 11 countries, caution should be exercised in generalizing the norms of the ROCF from this study to other countries in Latin America where data were not collected. Future studies should establish norms for the ROCF in countries like Ecuador, Uruguay, Venezuela, and Panama, among others. However, the ROCF norms from the current study may be more accurate in these countries than the norms from Spain or English-speaking countries with different cultures which are likely currently being used, although this assertion direly needs support from futureresearch.

Second, several sampling limitations are notable. It is important to emphasize that although participants were included with fewer than 12 years of education, illiterate individuals were excluded from the current study, so the ROCF norms cannot generalize to this population. Future studies should norm the ROCF in individuals who are unable to read and write. Similarly, no participants in the current study had neurological conditions, and all participants were adults; future similar studies should be conducted in populations of various neurological conditions, as well as among pediatric populations. Future research should also collect data on participants’ bilingualism, which was not controlled for in the current study. Participants only had to have Spanish as their primary language, and performance on the ROCF could be different if people speak other languages such as English, or local dialects such as Quechua or Guaraní. Future research should explore the possible influence of bilingualism on ROCF performance. A final sampling limitation is that the data were generally collected in specific regions of the countries in the current study, as opposed to nationally in those countries. Although the current study was the largest neuropsychological normative study in the history of Latin America, it should be seen as a first step in conducting more rigorous and larger studies with nationally representative samples.

Third, although the ROCF is a one of the most common neuropsychological instruments used in Latin America, many other instruments are also common in Latin America that should be normed in the same manner. Despite its commonness, the ROCF was created in a Western culture that may be different from those in Latin America. There is a great need for future research to develop more culturally sensitive tests that are bound in local cultures, not just translate and norm those that were developed in other countries and cultures. Future research should examine the psychometric properties of common neuropsychological instruments in Latin America, as well as test whether the instruments have strong ecological validity, and if not, develop instruments in those cultures that are more ecologically valid.

Despite these limitations and although previous studies have produced Spanish-language norms for the ROCF in Spain (Palomo et al., 2013) and Colombia (Rosselli & Ardila, 2003), this study was the first to generate ROCF norms across 11 countries in Latin America with nearly 4,000 participants. This study was the largest and most comprehensive ROCF normative study conducted to date in any global region, and as a result, its norms have the potential to affect the standard of neuropsychological assessment with the ROCF in Latin America unlike any study before it.