Redesigning French Internet
Business data collection at Insee:
the Coltrane project
4th International Workshop on Internet survey
methods (Daejeon)
Béguin Jean-Marc
Business statistics director (Insee)
13/09/2013
Outline of the presentation

Thanks and context
1. The existing system in France
1. Surveys
2. Tools
2. The Coltrane project (four work packages)
1.
2.
3.
4.
2
Authentication
Contacts management
Portal of customized services
Data collection platform
4th International Workshop on Internet survey methods
09/13/2012
1.1 Business surveys and National Statistical
Authorities (NSA)
 NSA : Insee and 5 or 6 ministerial statistical Offices
(MSO) (In 2010 Sessi, the MSO for industry, & Insee have merged)
 80 Business surveys (of many kinds)
- 30 Insee and 50 MSO (+ others)
- About 500,000 enterprises (in over 3,5 millions) receive at
least one questionnaire
- The sum of all the samples sizes is  700,000
- Taking into account the periodicity of the surveys (12
questionnaires for a monthly survey), 1 800,000 forms are
sent each year by the NSA.
 "sent" can be
- By mail for paper forms
- Through a website for dematerialized forms
3
4th International Workshop on Internet survey methods
09/13/2012
1.1 The present situation regarding Internet Usage
 So far, put surveys on the web has not been a priority
 2012 situation:
Insee
Number of surveys
Web surveys
Ratio of Internet usage
Sum of the samples sizes
Web surveys
Ratio of Internet usage
30
MSOs
47
22
16
73.3%
34%
380,000
305,000
178,000
46.8%
80,000
26,2%
Number of questionnaires
895,000 875,000
(weighted by periodicity)
Web surveys
654,000
253,000
Ratio of Internet usage
4
4th International Workshop on Internet survey methods
73,1%
28.9%
09/13/2012
1.1 Factors explaning the ratio of Internet answers
 Heterogeneous from one office to another
 Heterogeneous from one survey to another
 Possible factors
- The age of the process
- The type of variables collected
- The number of variables and the length of the
questionnaire
- The frequency of the survey
- The size of the responding firms
- The renewal of the samples
- The ergonomics of the human interface
- the efforts made by Insee staff
- ...
5
4th International Workshop on Internet survey methods
09/13/2012
1.2 The tools to implement Internet Surveys
 First Internet survey : 2000 (Sessi)
- Presently a second-generation application (ASP)
- Now Insee took over the portal (since 2010)
 Beginning at Insee : 2004 (application called CRPI)
- This application is now 8 years old (JAVA)
- Designed for repetitive short term surveys
 For specific purposes, new ways of collecting
dematerialized data have been developed recently:
- Upload / download application (eg EXCEL sheets)(2009)
- Blaise IS (Dutch software) (2009)
- Voozanoo (a free PHP software from Epiconcept) (2008)
 Conclusion: many tools !
Complex for us and for enterprises
(e.g. one portal for each tool)
6
4th International Workshop on Internet survey methods
09/13/2012
1.2 An analysis grid to compare the tools (2010)
 Originally based on the GSBPM
 Within the sub-processes we have defined 23 functions
such as :
- Build data collection instrument (create the form)
-
Create a form on one web page…………………………
Create a form on several web pages or screens ……..
Create simple filters (from one question to another)….
Create complex filters (from page to page)…………….
- Build or enhance process components
enterprise)
- Generate both login & password……………………….
- Take into account the campaign dates ………………..
- Customize the questionnaire (e.g. with previous answers of the same
 This grid helped us to analyse the pros & the cons of the
tools and of the whole system
7
4th International Workshop on Internet survey methods
09/13/2012
1.2 There is a need for a new project
 CRPI is the best; but
-
old (9 years)
based on a proprietary software (to implement java)
its maintenance is costly;
Not designed for big annual surveys
 No common directory nor common governance nor
common portal
 Technically complex
 moreover: in april 2011, the government requested that
all statistical surveys should be put on the web; existing
tools couldn’t allow it (specially for the SBS survey)
8
4th International Workshop on Internet survey methods
09/13/2012
2 The Coltrane project
 Platform offering a range of technical & business
services
 Four independent work packages:
-
Authentication
Contact management
Portal of customized services (for contacts)
Data collection platform (divided into 4 blocks)
 Each WP is composed of several functions accessible
through services
 A specific batch is planned to transform the SBSsurvey into a web-based survey
- Innovative because forms should be generated from
their DDI description
9
4th International Workshop on Internet survey methods
09/13/2012
2.1 The Coltrane project: authentication
 Independent from the collection itself
 We do not think about using PKI (Public Key Infrastructure)
but a LDAP directory to authenticate all the contacts
 Any « contact » in an enterprise should have a unique
login & password
 We shall offer the possibility of using this service to
all the MSOs
- Enterprises do not make any difference whether the
survey is carried out by Insee or a MSO
- They receive questionnaires from both
 But each ministry has its own Internet policy and have
already built something for their own web-sites
 we are not sure to be successfull !
10
4th International Workshop on Internet survey methods
09/13/2012
2.2 The Coltrane project: contact management
 Independent from the collection itself, but linked to the
authentication service (to exchange credentials)
 Can be used to manage all kinds of contacts (for example,
people buying data)
 Insee staff will be able to:
- Create or delete a contact and its respective
characteristics
- Renew credentials
- Display the list of contacts of a given enterprise
- Display the list of the surveys of a given contact
- Assist the web respondents and inform them about the
coming surveys
- Communicate with all the contacts of a given survey
- …..
11
4th International Workshop on Internet survey methods
09/13/2012
2.3 The Coltrane project: portal of customized
services
 A unique Internet portal will present all the surveys
(different from today)
 Including the MSO surveys (as far as they have
accepted to use our portal)
 The portal will be customized for each contact so that,
once authenticated, he can:
- only see the surveys he is interested in
- modify his own personal parameters (phone number, email, address, etc.)
- transfer his “answering power” to a colleague
- display his previous answers
- look at the remaining surveys he still has to answer as
well as their deadlines
12
4th International Workshop on Internet survey methods
09/13/2012
2.4 The Coltrane project: Data collection platform
 Main part of the project
 Block 1: offering different collection modes
-
Classical web form
Paper form (exchanged by mail)
Electronic data Interchange (EDI)
Upload / download service
If several collection modes exist, the questionnaires
should be generated from the same metadata flow
 Block 2: running the collection (eg building management tools)
13
Opening/closing a campaign
Edit statistics for the collection
Organize reminders
Start litigations with non respondents
4th International Workshop on Internet survey methods
09/13/2012
2.4 The Coltrane project: Data collection platform
 Block 3: generating questionnaires
- Aims at maintaining the coherence between a questionnaire
& its description (metadata)
- We will use a DDI model to model the questionnaire
- Then the form should be generated from its DDI description
- We are testing some softwares helping either the generation
or the DDI description
- This could allow us to put the SBS survey on the web
 Block 4: managing the collected data
- In our view, centralizing all the collected data (whatever the
data collection mode) is of utmost importance
- The following functions will be developed : retrieve and
control the data, adapt the formats of data flows, keep an
image of the enterprise answer
14
4th International Workshop on Internet survey methods
09/13/2012
2.5 The Coltrane project: Some specific issues
 Give the possibility for one enterprise to keep a copy of
its answer and for several respondents in the same firm
to fill up a single form
 Find the best trade-off between
- Sending back quickly new credentials to a contact (when he
has lost them)
- Keeping a good level of security
 Use the dialogue possibilities of Internet
- to help firms to answer to questions with hundreds of
possible codes
- to implement dynamic controls (eg with previous answers)
or explanations (in case of “peculiar” answers)
 Make a proof an enterprise has received a web form and
has not replied (for mandatory surveys, to impose fines)
- Legal and technical issue
15
4th International Workshop on Internet survey methods
09/13/2012
The Coltrane project is still in a conceptual
phase and should be completed by 2014 or 2015.
Thanks for your attention !
Insee
18 bd Adolphe-Pinard
75675 Paris Cedex 14
www.insee.fr
Contact
M. Jean-Marc Béguin
Tél. : +33 1 41 17 50 41
Courriel : jean-marc.beguin@insee.fr
Informations statistiques :
www.insee.fr / Contacter l’Insee
09 72 72 4000
(coût d’un appel local)
du lundi au vendredi de 9h00 à 17h00