ILO - International Labour Organization

10/13/2021 | Press release | Distributed by Public on 10/13/2021 06:12

What to know when comparing data on women and men’s work

To improve the measurement of women's work and provide guidance on good measurement practices, the International Labour Organization (ILO) and the World Bank collaborated with the Sri Lanka Department of Census and Statistics (DCS) between 2017 and 2019 on a multi-stage pilot study. The study compared the outcomes of two types of household surveys: the labor force survey (LFS) and a multi-topic living standards survey (MLSS). LFSs are the primary data source for labor market indicators, including the unemployment rate and labor force participation rate. MLSSs are designed to allow broader measurement and analysis of living standards and poverty. Due to these different primary objectives, the two survey types differ in questionnaire length, structure, sample size, and other elements, meaning they do not always yield comparable results.

This difference in outcomes can be concerning for users who can often assume that an indicator carrying the same label - such as unemployment - should amount to approximately the same thing regardless of the source.

The Sri Lanka study, which was supported by Hewlett Foundation and Data2X under the Women's Work and Employment Partnership, yielded key findings (now published) that could help solve persistent measurement problems and generally improve our understanding of the issue of comparability across sources. In this blog, we highlight three findings:

  1. Different types of surveys won't always give the same results, but gaps between them can be reduced

An important insight from the Sri Lanka pilot study is that the size of measurement gaps can be surprisingly large.

Researchers completed two rounds of interviews for the LFS and MLSS to discover any key differences between the two survey types and test whether changes in questionnaire design and implementation could yield improvements. In the first wave of data collection, the MLSS questionnaire captured less employment than the LFS - with a gap of about 5.5 percentage points overall and 8.1 percentage points for women in the employment-to-population ratio (see Figure 1). The size of this gap could have important implications for the use of the data for countries that use different sources over time.

In the second wave of the study, a few additional questions were added to the MLSS labor module, for example: a question on contributing family workers in agriculture and "recovery questions" for small/casual jobs (i.e., additional questions worded specifically to target such jobs if not reported by respondents with the original set of questions). As a result of these and a few other changes, the gap between the two surveys was substantially reduced, with the MLSS now only reporting a 3.5 percentage points lower employment-to-population ratio than the LFS overall, and 2.5 percentage points for women.

Whether differences of this scale remain of concern depends on the intended use of the data. It's important to remember that the source of statistics does make a difference, something that should be kept in mind when planning and analyzing surveys.