Editorial: How can Official Statistics find a way out of the fog?

2.1Interview with Javier Carranza on the developments with official statistics in El Salvador

Recently ISI and IAOS launched a joint statement on El Salvador’s decision to dissolve DIGESTYC. ISI/IAOS expressed its grave concern about the risks involved. These particularly relate to the mission of El Salvador’s official statistics as well as their professional independence, the qualification of staff and the confidentiality of statistical data.55 Javier Carranza, director of GeoCensos, an NGO that develops, studies, and implements technology trends to support National Statistical Offices, has worked with DIGESTYC, the NSO in El Salvador. In this interview, Misha Belkindas invites Javier Carranza to give his view on the developments with official statistics in El Salvador.

2.2COVID Impact and Innovation during the COVID-19 Crisis

Alessandra Coli, Angel Panizo Espuelas and Orestas Tsigkas (Eurostat, Luxembourg) in their manuscript ‘Economic support to European households in the aftermath of COVID-19. A cross-country comparative analysis based on quarterly sector accounts’ show how the range of containment measures to prevent the spread of the COVID-19 virus, led to unprecedented short-term economic loss for national economies. They argue that official statistics is a key source for robust comparisons of the economic impact of COVID-19 and subsequent support measures across countries. They particularly use Eurostat’s quarterly non-financial sector accounts and supplementary information provided by countries to estimate and compare the support received by households in 18 European countries. The results show some heterogeneity in the type and extent of support provided.

Nureni Olawale Adeboye, Oluwatobi Nurudeen Ogunnusi and Peter Osuolale Popoola (Nigeria) in ‘Official statistics: providing data acumen in the advent and continued presence of COVID19’, compare the Internally Generated Revenue (IGR) available to states in Nigeria in the years 2020 and 2019 by taking a critical look at the differences due to the COVID-19 pandemic and its effect on its over 200 million population. The findings of their study revealed that due to the continuous presence of COVID-19 in the country, 46% of the Nigerian states including the Federal Capital Territory experienced reductions in IGR generation as the pandemic surged while 35% of the states experienced a proportional increase on IGR and confirmed COVID-19 cases. However, 16% of the states posited an inverse growth in their IGR compared to the decreasing number of confirmed COVID-19 cases. They conclude that to cope with the unprecedented shocks occasioned by the continuous presence of COVID-19, continuous evaluation of countries’ revenue sources is highly inevitable.

The third contribution in this section has a very special focus. In ‘Assessing pandemic era stadium events and infections using mobile phone-based population mobility data: an exploratory study from Ireland, 2021’, the authors, Aidan James Condron, Ph.D., Guy McGrath and Jamie Madden (Ireland) use a high-resolution human population mobility dataset to quantify journeys from a (sporting) stadium area (Croke Park in Dublin) on Mass Gathering Events (MGE) days by destination. The anonymized, aggregated, human population mobility dataset used is based on mobile phone usage, and consists of a series of fine-grained geographical origin-destination matrices presenting daily estimates of area-to-area journey numbers. With mobility from the stadium area serving as a proxy for MGE attendance, this study explores associations between MGE attendance numbers and local COVID-19 infections over subsequent five-week periods. No evidence was found of an association between attendance at any of the six 2021 All-Ireland MGEs and COVID-19 infections over subsequent five-week periods. This finding contrasts with studies of comparable MGEs in 2020, such as English Association Football matches held during the spring of 2020, and German Bundesliga football matches held during the autumn of 2020. These differing outcomes may point to the effectiveness of transmission mitigation policies and behaviors.

2.3Governance of Official Statistics

Eric Rancourt and Inbal Marcovitsch (Canada) underline in ‘A data ethics framework for responsible responsive organizations in the digital world’, the importance of data ethics and propose a general framework to support organizations in adopting ethical data practices. They provide examples of statistics and standards as two contexts within which data ethics can be advanced and where advancements have already been made. They argue that as data are an integral part of the normative world also data ethics should be. They also state that though, with the advent of big data and data science, increased attention has been given to the ethics of artificial intelligence ethics is broader and must be considered on its own as a field of ethics.

2.4Young Statisticians Price winners 2022

In her guest editorial to this section, Gemma van Halderen (Australia) introduces the IAOS Young Statisticians Prize competition. She also presents the four 2022 award winners. The first prize was awarded to Erin Lundy (Canada) for her manuscript ‘Predicting the quality and evaluating the use of administrative data for the 2021 Canadian Census of Population’. Andreea Erciulescu (Rumania) was awarded the second prize with her paper ‘Statistical data integration models to reconcile health official statistics’, and Juan Carlos Galves, Jorge Fernandez Calatrava, and Lasai Taleb (Spain) were awarded the third prize with ‘Timeliness reduction on Industrial turnover index based on Machine Learning algorithms’. The special commendation for a paper from a developing nation was awarded to Atika Nashirah Hasyyati (Indonesia under the supervision of Thomas Lumley) for her paper ‘Imputation for subsampling in Indonesian National Socioeconomic Survey’. The four award-winning manuscripts are introduced in more detail in the guest editorial.

2.5Social Statistics, Population and housing census

A collective of statisticians from National Statistical Offices, Antonella Bernardini, James Brown, James Chipperfield, Christine Bycroft, Angela Chieppa, Nicoletta Cabella, Gary Dunnet, Michael F. Hawkes, Ahmad Hleibel, Eleanor C. Law, Daniel Ward and Li-Chun Zhang, worked together in producing the manuscript ‘Evolution of the Person Census and the Estimation of Population Counts in New Zealand, United Kingdom, Italy and Israel’. Their paper gives an overview of the evolution of the Census in Israel, Italy, New Zealand, and the United Kingdom and thereby provides an insight into the challenges and solutions of the modern Census. Especially focussing on the way census data are collected is changing in many nations because of common drivers: cost pressure, web-based collection, decreasing response rates, environmental shocks and the availability of administrative data.

Vassiliki Benaki-Kyprioti and Apostolos Kasapis’ (Greece) manuscript ‘Institutional challenges and prospects regarding the 2021 Population-Housing Census in Greece’ describe how 2021 was the first time that the Greek General Censuses were conducted by virtue of a legal act that underwent extensive deliberations and legal scrutiny. They argue the strengthening of the statistical legislation demonstrates the wide recognition of the professional independence ELSTAT has established and that it has provided a solid basis for the Hellenic Statistical System (ELSS) to be able to adequately respond to both current and future challenges. They illustrate how ELSTAT was able to immediately adapt to the developing operational conditions (COVID-19) by creating a plan for a hybrid census system for all households. At the same time the legal instruments used laid the foundations for the establishment and maintenance of Statistical Registers for Population and Buildings, introducing two unique identifiers (“keys”) in the Census variables and creating the institutional infrastructure for linkage to other administrative registers.

The study Nelson Ndifwa, Elevatus Mukyanuzi, and Tolo Lamech (Tanzania) report on in ‘Decomposing Rural-Urban Preterm Birth Variations in Tanzania’ uses secondary data to determine the extent to which rural-urban preterm birth variations are attributed to socioeconomic characteristics (endowments) variations. The authors employed binary logistic regression and a multivariate Oaxaca blinderdecomposition method, resulting in several interesting findings. For example, in urban areas, low parity, women’s anemia, high birth interval, women being employed and women’s BMI were statistically significantly associated with low preterm births. Another finding is that domestic severe violence was statistically significantly associated with a high prevalence of preterm births. Based on the results the authors recommend tailoring the policies that empower women’s inaccessibility of resources and opportunities and create awareness of women’s health issues that directly affect preterm births so as to reduce the problem.

Seyifmikael Yilema (Ethiopia), Yegnanew Shiferaw and Temesgen T (both South Africa) and Essey Muluneh (Etiopia), report on a study to improve the direct survey estimates of the z scores of malnutrition for unplanned zones with small sample sizes, for which the direct survey estimates are unreliable, by borrowing auxiliary variables from the census. In their study ‘Improving Survey Based Estimates of Malnutrition using Small Area Estimation’ they applied small area estimations under a Fay Herriot (FH) model to overcome the problem of generating reliable estimates by linking the Ethiopian demographic and health survey (DHS) with the census data. They state that, based on the results of the Fay Herriot model, assumptions are satisfactorily confirmed and that the model-based estimates confirmed that the malnutrition scores produced are more reliable, efficient and precise estimates than the direct survey estimates for small sample sizes in all zones.

The final manuscript in this section ‘Identifying Outliers in Multivariate Databases with Density-based Methods: a Housing Statistics Case’ by Antal Ertl (Hungary), discusses outlier handling in house prices. In preparing the calculation of indices for housing price changes data have to be cleaned and optimized for outlying observations. By applying various techniques, such as distance-based and density-based outlier detection methods, he highlights the importance of dealing with outliers. Housing statistics is a special case, as there is a high correlation between price and the location in a specific area of the dwelling in question, but it still serves as a fine example of handling outliers in economic and transaction-data. He concludes that identifying outliers is a rather nuanced thing, where statisticians could benefit from using advanced algorithms – such as the Local Outlier Factor (LOF), or the Feature Bagging Outlier Detection (FBOD).

2.6Measuring poverty

Sharthi Laldaparsad and Yegnanew A. Shiferaw (South Africa) estimate in ‘A model-based estimation and mapping of school-age children living in poverty in the local areas of South Africa’ the different poverty measures for lower and upper-bound poverty levels for local areas. The primary data sources are the Income and Expenditure Survey 2010/2011 and the 2011 national population census. The model diagnostics and the validation of the estimates showed that the model-based estimates at the local municipality level are more reliable than the direct estimates. They argue as there are significant variations in poverty across the local municipalities of South Africa, spatially targeted poverty alleviation programs to be more beneficial.

In ‘A New Index of Measuring Multidimensional Poverty: A Synthesis Method’ Diana Dilshanie Deepawansa, Priyanga Dunusinghe (both Sri Lanka), Parta Lahiri (USA) and Ramani Gunatilaka (Sri Lanka) present a new approach to the measurement of multidimensional poverty that combines the different strengths of the so-called Fuzzy Set method and the ‘Alkire and Foster’ method to address some of their limitations. They show that the resulting method satisfies Sen’s axioms of a good poverty measure and also incorporates some statistical techniques in the selection of indicators and computation of weights for the membership function. The empirical assessment is done by applying this method to a province of Sri Lanka on three main dimensions; material deprivation, deprivation of social dimensions and deprivation of human capital. The findings reveal that the core factors of poverty are deprivation in social networks, nutrition and housing. The authors suggest that the application of this method undoubtedly encourages the analysis of further research on multidimensional poverty.

Goal 1 of the 2030 Agenda for Sustainable Development, adopted by all United Nations member States in 2015 is to end poverty in all forms everywhere. The major indicator to monitor the goal is the so-called headcount ratio or poverty rate, i.e., the proportion or percentage of people in poverty. Soumojit Das (USA), Atanushasan Basu (India), Partha Lahiri (USA) and Sherpa Sengupta (India), present in their manuscript ‘Bayesian synthetic prediction of state-level poverty using Indian Household Consumer Expenditure Survey Data’, the results of a feasibility study to explore if estimates of headcount ratios or Poverty Ratios in intervening years (between the years that a Consumer Expenditure Survey is held) can be provided in absence of poverty lines by relating poverty ratios with average Monthly Per-capita Consumer Expenditure through a statistical model. They explored in their work a few models using Bayesian methodology. While the data-based assessment of their Bayesian synthetic prediction procedure is encouraging, the authors state that there is a great potential for improvements in the models, e.g., by incorporating more auxiliary data. They expect this preliminary work will encourage researchers to think about statistical modeling as a possible way to at least partially solve a problem for which no objective solution is currently available.

2.7Business statistics

The aim of the study ‘Evaluation of regional features of electronic commerce in Europe’ by Russian Motoryn, Kataryna Prykhodko (both Ukraine), Bogulaw Slusarczyck and Patrycia Zeglen (both Ploand) (Ukraine) is to analyze the regional features of e-commerce in Europe, using the main development indicators of e-commerce. Based on the analysis of available data, reports, statistics and research they describe the developments of the e-commerce market also in the context of the impact of COVID-19.

In the manuscript, ‘The impact of estimated sub-national purchasing power parity on macroeconomic measures’. Daniel Sánchez Serra (France) , Alex Costa, Vittorio Galleto, Jaume Garcia and Josep Lluís Raymond (all of Spain) start from the (false) assumption that all regions of a country have the same cost of living, and will lead to regional GDP figures (adjusted for national PPPs) that are biased and might limit the design and implementation of regional policies. Due to the lack of Purchasing Power Parities (PPPs) at the sub-national level, regional Gross Domestic Product (GDP) figures have been traditionally adjusted using national PPPs. In this manuscript, the authors try to overcome this problem by estimating PPPs at the sub-national level through an econometric method, which uses publicly available data and is based on the Balassa-Samuelson hypothesis. The analysis is based on OECD countries (TL2 regions) and EU-27 countries (NUTS2 regions) for a time series 2000–2018. The authors show an interesting effect of adjusting regional macroeconomic figures with sub-national PPPs in terms of economic welfare, regional convergence and the impact on EU cohesion funds.

2.8Data sources and methodology

The article ‘Scanner data processing in the newest version of the PriceIndices package’ by Jacek Bialek (Poland) discusses a new R package, i.e. PriceIndices, which is used to process scanner data and to calculate bilateral and multilateral price indices, along with their window extensions. The assumptions for the construction of the package were such that it would serve both practitioners and scientists through a multitude of methods and their parametrization. The article’s main purpose is to present the package’s utility in analyzing the dynamics of scanner prices.

Isabela Coelho, Marcelo Pitta, Pedro do Nascimento Silva, (Brazil) present in ‘Combining Quota and Probability Sub-Sampling Within Enumeration Areas to Produce More Reliable Estimates’ the use of quasi-randomization and sample matching methods to assign weights to the non-probability part of the sample, considering a special case in which a single survey was designed and carried out by simultaneously using the two sampling approaches in a single field operation. Traditional surveys face increasing challenges due to rising non-response rates and the diminishing resources available to survey organizations. A recently proposed solution involves the combination of non-probability sample surveys (often cheaper) with probability sample surveys (more expensive), using the latter as a reference to weigh the former. The quasi-randomization method provided the closest point estimates and smaller standard errors (on average) when compared to the benchmark estimates.

In his manuscript ‘Nowcasting Short Term Indicators with Machine Learning’ Markus Fröhlich (Austria), evaluates different machine learning algorithms for the early estimation of missing survey data in order to further improve the timeliness of Austrian STS data and to increase the granularity of early estimates as well. In the past time lags between reference periods and release dates of STS information have been quite considerable. Therefore, such an early estimation is important also on the level of NACE-Divisions With the exception of a few Divisions with small populations, the quality of the results achieved with machine learning methods is considered satisfying.

Over the last several years and within the framework of the Sustainable Development Goals, there has been a need to improve the measurement and understanding of local geographic patterns to support more decentralized decision-making and more efficient program implementation. Benjamin K. Mayala, Rose E. Donohue, Trinadh Dontamsetti, Tomas D. Fish and Trevor N. Croft (all USA) explore in their manuscript ‘Interpolation of DHS Survey Data at Subnational Administrative Level 2’ the potential of model-based geostatistics methodology to model Dwelling and Household Survey (DHS) indicators. To achieve this they use a stacked ensemble modeling approach that combines multiple model algorithmic methods to increase predictive validity relative to single modeling. The proposed approach can help to inform the allocation of resources and program implementation in areas that need more attention. Countries can use this approach to model other DHS survey indicators at much smaller spatial scales.

Elena Catanese, Monica Scannapieco, Mauro Bruno and Luca Valentino (Italy) present in their article ‘Natural Language Processing in Official Statistics: The Social Mood on Economy Index experience’ one of the first experimental statistics on Big Data published by the Italian National Statistics Institute (Istat). The Social Mood on Economy Index (SMEI) is a daily index computed from Italian Twitter’s public stream aimed at representing the evolution of feelings on economics topics. The longevity of SMEI makes it a perfect candidate to investigate Big Data related quality issues. Its intrinsic multivariate approach hinders the interpretation of the index. The paper reports on the current discussion and solutions implemented at Istat on the quality aspects and use of the SME index, in particular it focuses on SMEI’s revision due to the COVID-19 pandemic. The present work is also aiming at contributing to the debate on setting up quality standards for processing Big Data-based statistical products.

A rather until now not well-analyzed domain in Official Statistics are the factors that influence the development of viticulture of the Republic of Armenia (RA). Hovhannes Asatryan, Varadn Aleksanyan, Meri Manucharyan and Lina Azatyan (Armenia), present in ‘Dynamics of the development of viticulture in RA: the econometric case study’, the dynamics of the development of viticulture, in particular, the impact of the “Prices” factor on viticulture development. Based on regression analysis they qualify the impact of various economic actors. The results help to assess the dynamics of viticulture indicators in the future depending on changes of the price factor.

In ‘Consumer Expenditure Structure in Rural Bihar: Analysis based on Extended Linear Expenditure System’ Jitendra Sinha (India) presents significant results of a study on the consumer demand and expenditure structure of commodities concerning the income of the sampled households using the Extended Linear Expenditure System Model in the underdeveloped/developing economy of Bihar. Consumer expenditure patterns and estimates of expenditure elasticities give an indication of demand-led growth in a particular economy and are relevant to policymakers for formulation for accelerated development of the region. The authors state that these results may be valid for any underdeveloped/developing agrarian regions or economies of the globe.

The development of information technologies and the massive generation of data in today’s digital world provide new opportunities for official statistics. The authors of ‘Developing Online Shops Sampling Frame from a Marketplace’ Wiwin Srimulyani, Usman Bustaman, and Setia Pramana (Indonesia) state that Big data, especially produced by a marketplace has a great potential to produce a list of online shops. Their research aims to develop an online shop sampling frame from marketplace data. Using the shop and item datasets, an item-level data algorithm is developed to determine whether a shop is active or not. In this study, the focus is on online shops in Jakarta Province. The frame produced consists of 13 attributes such as Shop ID, number of items, annual revenue, shop types, Business scale classification and the location (URL and physical address). The frame contains 101,443 active online shops which most of are micro-enterprises.

The aim of the research presented by Tülin Otbiçer Acar (Turkey) in ‘Reflections of the Relationship between Education Indicators and Economic, Law and Human Development Indicators’, is to interpret the relationship between indicators of education, human development, economy and justice based on current indicators measured by international independent organizations. Based on 15 indicators covering education, law, human development and economy 41 countries were divided into two clusters. The manuscripts present the results of the analysis of the correlation between the indicators.

One of the key problems associated with Statistical Disclosure Control (SDC) is ensuring an optimal trade-off between minimizing the risk of unit identification and maximizing the utility of data to be disseminated. Many aspects should be considered when trying to find this trade-off. In the article ‘The trade-off between the risk of disclosure and data utility in SDC – a case of data from a survey of accidents at work’, the authors Andrzej Mlodak, Michal Pietrzak and Tomasz Jozefowski (Poland) present a way of assessing whether an optimal trade-off has been achieved. They discuss two main aspects of measuring the risk of disclosure. The first one is an internal risk, i.e. the risk of disclosing confidential information only based on disseminated microdata after the application of SDC (i.e. no attempt of combining data with external information is made); the second one is an external risk when the user has access to an alternative data set containing information that can be linked with statistical data to identify a unit. The authors show that it is possible to measure external risk and information loss while accounting for the measurement scale of variables.

Applying their methodology (using methods implemented in the new working version of the sdcMicro R package) on empirical results from an annual survey of accidents at work for 2017, they present the benefits and drawbacks of the tools used.

I wish you pleasant readings of these interesting articles.

4.1The next two issues: March 2023, Volume 39 (1), and June 2023, Volume 39 (2)

For the March 2023 issue (Vol39 (1)) some 25 manuscripts on varied topics are in preparation. Many stemmed from papers presented at the 2022 European Quality on Statistics Conference (Vilnius) and the 2022 IAOS Conference (Krakow). Also the June 2023 (Vol39 (2) issue will have a very varied set of manuscripts, a majority from individual submissions but also from recent conferences. For this issue, a special section with some eight manuscripts is expected from the statistics on Governance, Peace, and Security, as covered also by the work of the so-called Praia group on Governance Statistics.

The Special issue on the ‘History of Official Statistics’, is still in preparation but will not make it before June 2023. The guest editorial team is still in search of more authors and relevant manuscripts, so, do not hesitate to inform me when you have a ready manuscript or an idea for a manuscript for this Special. (pevssjiaos@gmail.com).

Of course there are always slots for other manuscripts; authors are kindly invited to submit their manuscripts via the submission channel: https://officialstatistics.com.

Pieter Everaers

Editor-in-Chief

November 2022

Statistical Journal of the IAOS

E-mail: pevssjiaos@gmail.com