Affiliations: [a] Statistics Netherlands, Division of Corporate services, IT and methodology, Sector Process Development and Methodology, 6401 CZ, Heerlen, The Netherlands | [b] Department of Mathematics and Computer Science, Section Stochastics, Eindhoven University of Technology, 5600 MB, Eindhoven, The Netherlands | [c] Statistics Netherlands, Division of Data collection, Sector Design, 6401 CZ, Heerlen, The Netherlands
Correspondence:
[*]
Corresponding author: Piet J.H. Daas, Center for Big Data Statistics, Room 0C13, CBS-weg 11, 6412 EX, Heerlen, The Netherlands. Tel.: +31 45 570 6964; Fax: +31 45 572 7440; E-mail: pjh.daas@cbs.nl.
Abstract: Producing an overview of innovative companies in a country is a challenging task. Traditionally, this is done by sending a questionnaire to a sample of companies. This approach, however, usually only focuses on large companies. We therefore investigated an alternative approach: determining if a company is innovative by studying the text on its website. For this task a model was developed based on the texts of the websites of companies included in the Community Innovation Survey of the Netherlands. The latter is a survey carried out every two years that focusses on the detection of innovative companies with 10 or more working persons. We found that the text-based model developed was able to reproduce the result from the Community Innovation Survey and was also able to detect innovative companies with less than 10 employees, such as startups. Model stability, model bias, the minimal number of words extracted from a website and companies without a website were found to be important issues in producing high quality results. How these issues were dealt with and the findings on the number of innovative companies with large and small numbers of employees are discussed in the paper.
Keywords: Innovation, webscraping, text analysis, Big data, concept drift
