Beyond code of practice: New quality challenges in official statistics1
Abstract
Globalisation, technology, competition and changes in the political agenda have led to a demand for new statistics, but also provided new possibilities in terms of new data sources. However, these developments have challenged statistical institutes’ compliance with values and principles described in quality frameworks for official statistics, such as the European Statistics Code of Practice. In practice there are often trade-offs between the principles of quality frameworks and full compliance is difficult. Official statistics must be relevant, and this may challenge the independence of a statistical institution. Utilisation of new data sources implies challenges for accuracy and reliability, and meeting competition with partnership may harm confidentiality principles and equal treatment. The paper considers some of these challenges, how they are handled in the existing quality frameworks and could be met by the statistical institutes. Reflecting on these challenges may guide the way forward. Statistical professionalism is a key word in this context.
1.Introduction
New developments in demand for statistics and technology have challenged statistical institutes’ compliance with their values and principles, described in quality frameworks.
In Europe, the European Statistics Code of Practice (CoP) [1] and the Quality Assurance Framework (QAF) [2] constitute the cornerstone of a common framework for quality in official statistics. There are similar frameworks developed by UN, OECD and regional statistical cooperation bodies. They are all inspired by and built on the UN Fundamental Principles [3].
The Code of Practice was developed in 2005 and revised in 2011 and 2017 to adapt to new requirements for statistics, new data sources and technology. The focus of the revisions has been the strengthening of the statistical institutes’ independence and coordination role, the use of administrative and new data sources and innovation in general. Compliance with the CoP among the members of the European Statistical System has been assessed by peer reviews.
In practice, compliance with quality frameworks must to be balanced over their principles as they inter-relate. Even if requirements for professional independence, impartiality and confidentiality are laid down by law, these requirements are challenged by requirements for relevance, cost effectiveness and cooperation.
The paper considers these challenges. The starting point is international, but examples are mainly from Statistics Norway (SSB). However, these are believed to be representative for several statistical institutes.
2.Balancing quality principles
The trade-offs between quality principles that come most readily to mind are those between accuracy, timeliness and cost effectiveness. Another trade-off is between timeliness and punctuality. These quality dimensions are linked to statistical products or processes. However, there are situations where also principles linked to the institutional environment for official statistics must be considered in relation to other quality criteria, notably professional independence versus the relevance of the statistics. Cooperation, availability or lack of resources may contribute to weakening independence as well, and there may be a temptation to compromise impartiality and objectivity. Statistical confidentiality may also be affected by the demand for relevance, cooperation and cost effectiveness.
Both professional independence and statistical confidentiality are principles that are included, not only in the CoP, but also in legislation (both European and national statistical laws). CoP itself is self-regulatory, i.e. compliance is controlled by the organisations and people within the system.
Principles laid down by law are subject to stronger requirements and should not be compromised. The amended European Statistical Law [4] puts principal demands on the institutional environment such as:
• Professional independence of the statistical institution and its head (CoP principle 1)
• Coordination role of Eurostat and the NSIs (CoP principle 1bis)
• Mandate for data collection, including access to administrative and other data for statistical purposes (CoP principle 2)
• Statistical confidentiality and data protection (CoP principle 5)
• Equal access and release calendars (under CoP principle 6)
The European Statistical Law refers to CoP and its quality criteria for product quality (principles 11–15). Regarding these principles there is always a need to balance compliance between them. Law prescriptions are limited to the use of CoP with its accompanying Quality Assurance Framework as such, aiming at good compliance.
The principles of independence, statistical confidentiality and impartiality are also central in the ethics of statisticians and statistical institutes, as formulated in the International Statistical Institute (ISI) Declaration of Professional Ethics [5]. Here, the principles are summed up as values: Respect (protection of privacy), Professionalism (understanding user needs and quality) and truthfulness and integrity (independence, objectivity and transparency).
The balance between such basic principles of official statistics is illustrated in Fig. 1 and discussed in the following chapters.
Figure 1.
Balancing independence and other principles for European statistics.
3.Independence and relevance
3.1What is professional independence?
CoP principle 1 states that professional independence of statistical authorities from other policy, regulatory or administrative departments and bodies, as well as from private sector operators, ensures the credibility of European Statistics. But what does professional independence mean? The indicators in CoP mention the responsibility of the National Statistical Institutes (NSIs) and Eurostat for ensuring that statistics are developed, produced and disseminated in an independent manner, in other words how statistics are produced but not what or which statistics should be produced. How statistics are produced also comprises when they are disseminated.
The UN fundamental principles also use the word professional independence, stating in its principle 2: To retain trust in official statistics, the statistical agencies need to decide according to strictly professional considerations, including scientific principles and professional ethics, on the methods and procedures for the collection, processing, storage and presentation of statistical data.
Again, this refers to how the statistics are produced. How professional independence may be challenged by other quality criteria is outlined in the following paragraphs.
3.2User needs
What statistics should cover is determined by the users, ensuring that statistics are “fit for purpose”. There is a variety of users with different needs. In the Quality Declaration of the European Statistical System [1], it is said that the mission of this system is to provide independent high quality statistical information at European, national and regional levels and to make this information available to everyone for decision-making, research and debate. This means that the needs of both political authorities and the common public are equalised.
In the CoP user needs are reflected by the principles linked to the statistical output, such as principle 11 on relevance: “European statistics meet the needs of the users.” Indicators on relevance state that procedures to consult users must be in place and that relevance and user satisfaction is monitored. Statistics must change according to user needs, at the same time as comparability over time must be considered.
The question discussed here is how user focus in practise is balanced against professional independence.
3.3Funding
The CoP principle 3 on adequacy of resources states that the resources available to statistical authorities are sufficient to meet requirements for European statistics. Normally, most funds for official statistics come from the Government budgets, in principle they are controlled by a Parliament but administered through a Ministry. In many countries including Norway this is the Ministry of Finance. During the last years this funding has been reduced because of demands on cost effectiveness in the public sector. This is normally unproblematic from the point-of-view of independence, given that the respective Ministry respects the legal basis and the principles of official statistics.
A great part of this funding covers the production of European statistics, following from EU-regulations. These regulations state not only what statistics should be produced but to some extent also how they should be produced. More than 60 percent of Norwegian official statistics are regulated by European law which is incorporated in Norwegian law. In this case there might be a balance between real independence and comparability between countries. However, it is obvious that international comparability is a necessity for most official statistics. Most of these statistics would probably have been produced even without EU regulations, and some of these statistics are required also by other international organisations. This is still worth reflections, as an example of limitations to the independence of the statistical authorities.
Other funding directly from the users might be more problematic. The Leadership Group (LEG) on Quality [6] distinguised between statistics provided as a public good (main product of the NSIs) and as a “private” good (tailor-made analyses on the demand from individual customers even if these may be public bodies). The latter type of statistics often results in conflicting requirements.
In SSB, almost 30 percent of the budget stems from work on commission. Such funding offers direct contact with the users, which is valuable. However, user-funded commissions should supplement the primary tasks financed by the state budget, not replace them. The extent of user-funded commissions must not be of such a magnitude that it threatens our primary priorities and hence independence. It is important that the results are statistics or analyses available to everybody at the same time.
3.4Data management
In some cases, users pay for services like data collection and data management. Examples from SSB comprise systems for coordinated data collection and management of data from the municipalities and from the health sector. Data are often used to establish an administrative register. Ljones [7] discusses independence and ethical issues related to the existence of detailed data from registers, and the role of NSIs as responsible for data collection and management for data used for New Public Management (in addition to the role of producers of official statistics). Combining the two roles is efficient. However, the roles should not be mixed. In these cases, SSB uses the legal basis of the relevant Ministry for data collection, and then the Statistical Act for using the data for statistics. There is still a danger for compromising professional independence linked to this or jeopardising the trust in that we really act independently.
The desired development towards more use of administrative data might also imply a shift of the content of statistics in the direction that fits the authorities but not necessarily those who are governed. CoP states that definitions and concepts in administrative data systems should be adapted to those required for statistical purposes, but this is often not so easy in practice.
3.5Analyses
Users often demand analyses. Analyses add value to and make statistics more relevant. Relevance is not only what statistics cover, but also how statistics are presented. Analyses may take various forms. In the context of this paper analysis is a tool to explain the statistics and their impact, by presenting them in a suitable form (tables and graphics), combining data from different sources, interpreting data, and identifying possible causes and effects. Analyses also include modelling and more substantive analyses, such as making projections. An analysis by a statistical institute shall not advocate policies or take partisan positions. But there is still a question on how far the institution shall go in performing analyses.
Experiences from quality reviews in SSB tell us that users almost always ask for better coherence and interpretations of the statistics in a broader context, which can be achieved by simple analyses of trends and comparison with other statistics. More advanced analyses highlighting correlations casting light on political issues can increase the value for the users. But this also calls for caution to safeguard the principle on impartiality and objectivity. Particularly if conclusions from the analysis rely on model dependent assumptions that may be subject to scientific controversies.
Unlike most other NSIs SSB has a research department that carry out economic and demographic analyses (not only research on development of methodology supporting statistics production). This work counts for about 10 percent of SSB’s activity, and the results are not official statistics, thus in principle not subject to compliance with the CoP. However, the principles of research comprise independence, objectivity, sound methodology and transparency as well. The Department of Research in SSB represents a special challenge for the independence, see example below.
3.6Examples
There are a few examples from NSIs where the heads or other statisticians have been exposed to pressure or have resigned because of conflicts with governing authorities linked to or allegedly linked to the central principles for official statistics. Oppeln-Bronikowski et al. [8] mention some examples of pressure from Governments on NSIs not to release or change release time of statistics that were not in the best interest of the Government. In this case the international quality frameworks provided protection for the statisticians. Langkjær-Bain [9] has interviewed former Director Generals (DGs) of Greece and Canada, and the former responsible statistician for the CPI in Argentina. Their cases vary, but they are all linked to professional independence of statistics and in the Canadian case to statistical confidentiality as well.
There was also a case in Statistics Canada in 2010 where the DG resigned because of a disagreement with the authorities on whether a voluntary survey can become a substitute for a mandatory census [10]. His answer was no since a voluntary survey would harm the quality and thus the usefulness of the survey considerably. Just as important as the professional disagreement was that the minister went public with inaccurate claims about the advice he had received from Statistics Canada. To avoid discussing this as a political issue the DG described it as a technical matter, which sounds less dramatic, though a possible breach of the professional independence of official statistics from the Government’s side.
Though different, we can add a Norwegian example to this list. Last year the DG of SSB withdrew after pressure from the Ministry of Finance. There are different opinions about the reasons for this, but independence became an issue. Like other Government institutions, SSB is imposed to be more efficient, and a program for modernisation has been initiated. This brought about some reorganisation. The reorganisation plan comprised the Department of Research, and there were concerns that some macroeconomic analyses could be negatively affected. In addition to providing the main funding to official statistics, the Ministry is also an important customer paying for analyses such as these. Even if authorities and other stakeholders such as the labour market’s organisations were concerned, the conflict between the Minister of Finance and the SSB DG accelerated when it was proposed that one among 25 other researchers (from a total of 75) should move from the Department of Research to one of the departments producing statistics. This researcher had been the key person carrying out cost estimates linked to immigration, including long term forecasts, for a public committee [11]. With a Minister of Finance representing an immigration sceptic party, there were reasons to suspect political reasons behind her lack of confidence for the DG of SSB. The Ministry claimed that it had given strong warnings against the reorganisation program for months. This was disputed by the SSB DG. (Note the similarities with the disagreement on the content of the dialogue between the Canadian Ministry and NSI in 2010).
In the aftermath there has been an inquiry about this in the Parliament, concluding that the Ministry’s support for reorganisation of SSB for the last year, though normal concern about risks linked to this, first ceased when the specific rearrangement of the Department of Research was known. The inquiry questioned the culture in the Ministry, imposing contradictory requirements to SSB. The Ministry was accused of revising minutes from meetings with SSB after they had been agreed upon and by supressing documents that might be public. The Minister was heavily criticized by the Parliament, but the governing majority did not support a conclusion that they had lost confidence in her.
As we have discussed, independence does not mean that an NSI solely can decide to produce what it wants, but it should control how production is done. Internal organisation seems to belong to how, i.e. be the responsibility of the DG solely, regardless of how wise the reorganisations are. After the resignation of the DG all the planned reorganisations were continued, except for the reorganisation of the Department of Research. It seems that The Ministry of Finance has mixed its roles as owner and customer.
Georgiou [12] has discussed professional independence of official statistics considering the institutional links between the statistical institute and the executive branch of Government, i.e. ministries with great interest in statistics or where statistics are crucial for assessing their performance. He argues that the production of official statistics should be carried out by a separate part of Government. This is hardly realistic in Norway, but an arm’s length distance between the Ministry of Finance and SSB would not harm.
For several European countries the European peer review teams 2014–2015 recommended a revision of their statistical laws, to strengthen their professional independence and clarify the content of “official statistics” (see for example report on Norway [13]). In Norway a Government committee has just delivered a proposal for a new law amending the Statistical Act from 1989 [14]. The draft has been subject to a public consultation. New proposals include clarification of the role of the DG and the establishment of a multiannual statistical program. The program will define the content of “official statistics” regardless of who produces it, and the mechanisms for quality control of such statistics which like European statistics shall comply with the CoP.
4.Confidentiality
The CoP principle 5 on confidentiality and data protection may also be affected by some of the principles and indicators linked to statistical processes and products. Here follow some cases where also professional independence might be affected.
4.1Microdata
Statistical institutes are committed to provide microdata for research, but under strict confidentiality rules. This is stated under CoP principle 15 on accessibility and anchored in the statistical laws.
This has normally not been a problem for confidentiality, data protection has had a strong position. Complaints from researchers about use of time and costs of producing microdata, however, have been more frequent, but that is another issue.
In the new CoP data sharing and data integration is promoted (under Principle 9 on Non-Excessive Burden on Respondents), given adherence to confidentiality and data protection. In the new Norwegian statistical law draft data sharing is proposed between producers of official statistics for this and only this purpose. This is cost-effective but may be a challenge for data protection and hence for the trust in the Norwegian statistical system.
4.2Statistics on small groups
There is a question of judgment linked to confidentiality and the size of groups for which statistics should be published. This issue has two aspects: How to prevent disclosure of confidential information linked to individuals, and how to avoid publishing statistics in a way that can harm small and vulnerable groups.
The first issue is more of technical nature, even if there might be a question of considerations of acceptable risks, the second ethical. The challenges connected to technology are linked to the vast amount of available data from other sources than the NSIs, and the possibilities to identify persons and businesses by combining these data with statistics. This calls for more caution than before when disseminating statistics for small groups. There has been an increased demand for statistics on a more detailed level, for example small geographical areas for municipal planning or analyses of business possibilities by using geographical information systems. The extensive use of administrative and new data sources for statistics (e.g. data on social media) makes it easier to produce statistics for small groups and areas. This demonstrates the need to balance relevance with risks of harming privacy protection.
Legal protections relating to statistical confidentiality in most countries only pertain to identifiable microdata, and hence the technical challenge, but not the ethical one.
As quoted above, the UN principles for official statistics refers to ethics in principle 2. Selzer [15] has expressed this in the following way: “The absence of clear legal protections relating to mesodata,22 statistical agencies, together with their leadership and staff, are under heavy ethical obligations to provide as wide a protective net as possible over mesodata pertaining to such vulnerable populations.”
4.3Examples
SSB has a set of ethical guidelines from 2007, where it is stated that carrying out surveys and disseminating results such that individuals or groups are affected negatively should be avoided.
However, attitudes and policies in this area have changed over time, both in the society and in SSB. In the early 90s there was a discussion by the SSB management following a request to estimate costs and value creation by refugees and immigrants. The conclusion was negative since such estimates would be too uncertain to publish them, but SSB could provide a lot of statistics relevant for this issue. Ten years later we were asked to develop projections of the “immigration population”. The conclusion was that SSB did not have enough knowledge on future immigration to make projections with sufficient quality compared to normal and less detailed population projections. However, it was decided to continue the discussions, also on ethical aspects linked to this, and to develop ethical guidelines.
Nevertheless, as mentioned costs linked to immigration were estimated during the past years as input to a public study. Such calculations would hardly have been done by SSB 15 years ago. SSB has recently also published a report on immigration and crime [16] where immigrants have been grouped by origin country. In this work it has been important to explain not only what the statistics show, but also group by variables that normally are correlated with crime, such as age, sex, economy and relation to the labour market. And to ensure that there are reasonably many units in each group.
An example on how attitudes have changed is also how statistics on the Sami population in Norway have been disseminated. In the period before World War II social statistics was characterised by detailed statistical descriptions of marginal population groups including ethnic groups. In the Norwegian census from 1930 the number of the Sami population was published together (same report) [17] with the number of citizens from other countries, blind, deaf, retarded and mentally ill people. Statistics were not published in this way after the war, but now representatives of the Sami population are promoting more statistics about and for their minority.
5.Cooperation and coordination
Cooperation has two aspects. One is the extended coordination role with the responsibility for the NSI for quality assurance of all official statistics, following European statistical and the new CoP principle 1bis. Though only for European statistics, the same responsibility for coordination is reflected in national statistical laws such as in the new Norwegian one. The statistical institutes will have a role as national data curators/custodians. The other aspect is cooperation to ensure access to source data for development and production statistics, though laid down by law such cooperation is necessary in practice.
Other producers of official statistics normally have some administrative and operative functions as their core business. Producing statistics here constitute a small part of their respective institutions. This may represent a challenge both to professional independence and confidentiality.
To maintain professional independence, it is important that the role of producing European and official statistics is clearly delimited from other roles the producers might have. A statistical release calendar and full transparency about methods and production procedures are concrete requirements and means of ensuring this.
CoP promotes data sharing and integration among producers of European and official statistics to minimise response burden, while adhering to confidentiality and data protection requirements. Even so this data sharing may also increase the risk for confidentiality breaches and weaken public trust of the producers.
In this context it is also worth reflecting on if increased cooperation and data sharing will be followed by increased research demands for access to microdata? And if so, whether quality requirements on microdata then will impose a raised focus on other quality dimensions than statistical accuracy, timeliness etc.? Being a steward of microdata that are made available for public good research and analysis is a different role than the statistics producer’s. Highlighted perspectives on ‘data quality’ parallel to (or in parts of the organisation instead of) ‘statistics quality’ is likely to create new challenges of how to balance quality principles. Cooperation is the key to get access to new also privately held data sources, but this might also affect data protection. Cooperation partners might want to have access to data in return for providing their data. It also represents a challenge for the requirement of equal treatment.
Partnership has become a key word in the strategies of statistical authorities. In the ESS Vision 2020 [18] it is said that we will establish alliances and partnerships with data owners. Public sector normally does not own the new data sources, and the statistical authorities will have to negotiate access to the sources, and the stability of data deliveries depends on external parties.
But how could an NSI have a formal partnership with one or some private companies, for example telecommunication companies, and not with others? Are agreements with all necessary? The answer is probably that this kind of cooperation is OK if professional independence and statistical confidentiality are not compromised, and that competition between private companies is not affected. To prevent this, it is necessary that data used by the NSI is used solely for statistics and analyses benefiting and made available for the society, i.e. be subject to the quality principles of official statistics.
SSB is engaged in a few partnerships with both public, academic and private institutions where data are used only to investigate if and how they can be used for different purposes including new statistics.
For example, to investigate new ways of making the household budget survey, SSB have acquired data about purchase transactions from the private sector. To assist in the analysis of these data partnership with a research consortium of both private, public and academic entities are considered.
This is unproblematic in an experimental phase, but it must be clear that data acquired by the NSI is used only for experimenting aiming at and eventually producing official statistics. Access to data on these terms is included in the proposal for the new Norwegian Statistical Act.
6.Conclusions
A main message from the work on quality in statistics at the end of last century was that quality consists of several features reflecting user needs. This has been taken into account in the development of quality frameworks such as the European Statistical System Code of Practice, comprising principles related to both the statistical institutions, their production processes and the different dimensions of output quality required by the users.
It is difficult or in some cases meaningless to set minimum requirements for compliance with each principle across all statistics, since the principles inter-relate.
Compliance with quality frameworks implies finding the right balance between their principles and indicators. This even regards principles laid down by law which in practice may be challenged by users’ demand for relevant statistics. Ethical issues should be considered as well. New data sources and partnerships represent new challenges when balancing quality principles.
The existence of quality frameworks itself provides protection and safeguarding of public trust in official statistics. But assessing compliance with these frameworks is not enough. Comprehension of and reflection on balancing quality criteria is necessary to achieve a good total result. This is statistical professionalism in practice.
Means to ensure a satisfactory level of overall compliance comprise both internal and external quality reviews. International cooperation is crucial, and the European Peer Reviews are important in this context. A good example of this is the work on the new Norwegian Statistical Act, following a recommendation from last Peer Review and based on the Code of Practice.
Notes
2 Meso denotes middle, i.e. not micro and not macro.
References
[1] | European Statistics Code of Practice. Available from: http://ec.europa.eu/eurostat/web/quality/european-statistics-code-of-practice. The Quality Declaration of the European Sta- tistical System is included in the CoP preface. |
[2] | Quality Assurance Framework of the European Statistical System (QAF). Available from: http://ec.europa.eu/eurostat/documents/64157/4392716/ESS-QAF-V1-2final.pdf/bbf5970c-1adf-46c8-afc3-58ce177a0646. |
[3] | UN Fundamental Principles of official Statistics. Available from: https://unstats.un.org/unsd/dnss/gp/FP-New-E.pdf. |
[4] | European Statistical Law 223/2009 with its amendment Regulation 2015/759. Available from: http://ec.europa.eu/eurostat/documents/3217494/5719241/KS-31-09-254-EN.PDF/e82af841-615f-47d4-8c6d-647705de97a7?version=1.0, http://eur-lex.europa.eu/legal-content/EN/TXT/PDF/?uri=OJ:JOL_2015_123_R_0009&from=EN. |
[5] | International Statistical Institute Declaration on Professional Etics. Available from: https://www.isi-web.org/index.php/activities/professional-ethics/isi-declaration. |
[6] | Quality in the European statistical system – The way forward. Eurostat (2002). Available at: http://ec.europa.eu/eurostat/documents/64157/4373735/03-Quality-in-the-ESS-LEG-on-Quality-2002-EN-0-1.pdf/347a7417-679e-4feb-9d72-7151bd6330c7. |
[7] | Ljones O. Independence and ethical issues for modern use of administrative data in official statistics. Statistical Journal of the IAOS 27 ((2011) ). Available from: https://content.iospress.com/download/statistical-journal-of-the-iaos/sji00715?id=statistical-journal-of-the-iaos%2Fsji00715. |
[8] | Oppeln-Bronikowski S, Kronz C, Meinke I, Wirtzfeld H. How can professional and ethical frameworks strengthen statisticians in their practical work? Statistical Journal of the IAOS 31 ((2015) ). Available from: https://content.iospress.com/download/statistical-journal-of-the-iaos/sji941?id=statistical-journal-of-the-iaos%2Fsji941. |
[9] | Langkjær-Bain R. Trials of a statistician, Significance 14: (Issue 4) (August (2017) ). Available from: https://rss.onlinelibrary.wiley.com/doi/epdf/10.1111/j.1740-9713.2017.01052.x. |
[10] | Franks CES. The Munir Sheikh matter was no technicality. Globe and Mail July 29 2010, updated May 2 ((2018) ). Available from: https://www.theglobeandmail.com/opinion/the-munir-sheikh-matter-was-no-technicality/article1389761/. |
[11] | Norges Offentlige Utredninger, NOU (2017) : 2. Integration and trust – Long-term consequences of high immigration. Available from: https://www.regjeringen.no/en/dokumenter/nou-2017-2/id2536701/sec1. |
[12] | Georgiou A. The Production of Official Statistics Needs to be a Separate Branch of Government, Statistical Journal of the IAOS 34: (2), 149-160. Available from: https://content.iospress.com/download/statistical-journal-of-the-iaos/sji170399?id=statistical-journal-of-the-iaos%2Fsji170399. |
[13] | Rockmann U, Vaicenavičius RJ, Kárpáti J. Peer Review Report Norway (March (2015) ). Available from: http://ec.europa.eu/eurostat/web/quality/peer-reviews. |
[14] | Norges Offentlige Utredninger, NOU (2018) : 7. Ny lov om offisiell statistikk og Statistisk sentralbyrå. Available from: https://www.regjeringen.no/no/dokumenter/nou-2018-7/id2594104/?q=statistikklov%202018. |
[15] | Selzer W. Official Statistics and Statistical Ethics: Selected Issues, Proceedings of the International Statistical Institute’s55th World Statistical Congress, 6th-12th April (2005) , Sydney. Available from: https://unstats.un.org/unsd/WS%20%202005%20ISI%20paper.pdf. |
[16] | Crime among immigrants and children of immigrants in Norway. Statistics Norway Reports 2017/36. Available from: https://www.ssb.no/en/sosiale-forhold-og-kriminalitet/artikler-og-publikasjoner/crime-among-immigrants-and-children-of-immigrants-in-norway. |
[17] | Folketellingen i Norge 1 desember 1930, Fjerde hefte, Samer og Kvener – Andre lands statsborgere. Blinde, døvstumme, åndssvake og sinnssyke (Lapons et Quaines – Sujets étrangers. Aveugles, sourds-muets, idiots et aliénés). Statistics Norway, (1933) . Available from: https://www.ssb.no/a/histstat/nos/nos_ix_017.pdf. |
[18] | European Statistical System Vision (2020) . Available from: http://ec.europa.eu/eurostat/web/ess/about-us/ess-vision-2020. |
