The ILO Department of Statistics recently published a report on a global survey with national statistical offices on the impacts of the COVID-19 pandemic on labour force surveys and dissemination of labour market statistics. We have the pleasure to have a conversation with Mr. Kieran Walsh for some insights about the report and how it informs the work programme of ILO in the future.
ISWGHS: Kieran, so happy to have you with us! First of all, could you give us a quick introduction of the survey, as I understand this is a part of efforts of ILO Department of Statistics in supporting countries?
Kieran: Yes. This is our second survey on how COVID-19 has impacted on countries’ ability to compile labour market statistics and how countries have been reacting to the challenges. The first survey, carried out in March 2020, revealed how widespread the impact was and how much it varied across countries. We decided to do the survey a year later to get a more detailed understanding of how the situation had evolved during 2020. We hope this is valuable to countries in understanding how their peers have been reacting, maybe giving inspiration for actions they can take. We also hope that it can support those producing statistics to make a case to continue or increase investment in statistics to provide the data users need in a way which is resilient and sustainable for the future.
ISWGHS: Could you please give us three key findings that you would like our readers to learn from the results of the survey?
Kieran: Definitely.
First, the pandemic caused an unparalleled disruption to data collection. About half of all countries had to suspend their face-to-face interviewing for some time in 2020 and over 20% had to cancel a planned labour force survey entirely. This was a serious concern for many reasons, including the fact that data were urgently needed to understand the pandemic and how to emerge from it.
Second, how we collect information does matter during COVID. We have learned so much in recent decades about how to improve data quality while at the same time making data collection more efficient. Modernizing data collection can make the difference between us being able to produce data or not. Close to two-thirds of the countries responding to the survey (around 110) turned to or increased the use of telephone interviewing as a way to maintain data collection during 2020, and this lessened the impact of COVID-19 on data collection we would have felt. This means for example that countries were mostly able to maintain, or increase the range of data they published during 2020, despite the massive challenges they faced. However, we still cannot ignore the negative impacts which were ultimately felt, such as cancellation of publications, breaks in series or the need to stop publication of some indicators. If we learn the lessons from these experiences, we could avoid or lessen those impacts in the future if there are other crises.
Lastly, not all countries can maintain remote data collection while ensuring data quality While the pandemic did force a widescale change in data collection practices, it cannot be taken for granted that all countries can use or maintain remote data collection without addressing data quality concerns. For example, about half of the countries that introduced remote data collection in 2020 are considering reverting to the face-to-face collection mode for various reasons relating to data quality concerns. We have to be prepared to support countries in enabling appropriate use of remote data collection while still adhering to good statistical practices and achieving representative samples. This is far from easy but a challenge that should be taken on.
ISWGHS: Very important findings, thank you! So COVID-19 should be our wakeup call for not taking data for granted, data collection can be disrupted unless we keep evolving and support is necessary to keep the momentum. What would you say about the post-COVID LFS? Are we going back to business as usual?
Kieran: I think reality will be somewhere in-between but nothing is certain for the moment and the situation will vary across countries. As I mentioned earlier half of the countries that reported introducing remote collection in 2020 were not sure about keeping it due to a lack of survey infrastructure, such as databases of respondent contact details. In fact, quite a few have already reverted back to face-to-face interviewing.
One option worth assessing for countries that cannot fully rely on remote interviewing is multi-mode data collection, covering as much as possible by telephone or web with the rest face-to-face. Multi-mode also has its challenges but can be a path forward.
I also think the future holds more use of non-household survey data. This is not a new phenomenon but I think we have seen a leap forward in several countries and a general increase in interest. In doing so we need to ensure that people realise that different data sources should not be seen as competing with each other, rather supplementary.
ISWGHS: We have talked a lot about the disruptions and challenges from COVID-19, do you see any positive drives and changes in the field of household surveys because of COVID-19?
Kieran: I see a few related positives.
Firstly, the survey gives us a clear signal that, despite the challenges, countries did manage to maintain household survey data collection and generated a lot of useful information, in many cases even more than before. My hope is that this reinforces the critical role of household surveys in the data ecosystem. We have also seen the incredible flexibility and reactiveness of producers of statistics during the pandemic, but they need adequate resources and funding for this to be sustainable. We have to hope that the heightened awareness of the need for data will help them secure those resources.
On another note, we have seen that remote data collection can operate on a wider scale, bringing many potential benefits. So, while there are valid concerns, such as low network coverage in some areas, and loss in data quality, we can build on the experiences of the last two years to make sustainable improvements to data collection and dissemination practices, and leverage the positive aspects of remote data collection.
It is also positive to see increased awareness of the need for more data. For some time we, and many others, have been highlighting the need to supplement headline indicators with a wider range of estimates. This has been well illustrated during the past two years with countries and international organisations turning to a wide range of complementary indicators, sometimes from different sources, to provide a more complete understanding of the impact of COVID-19 on the labour market and many other topics. I hope this continues, even if having a small set of headline indicators will remain important for monitoring.
Another positive aspect that comes to mind is the increased collaboration across agencies and regions, supported by more online presence and events. I am hopeful that this will be carried forward and increased further, allowing us to provide more coherent support to countries to produce data, which is ultimately the objective for all of us.
ISWGHS: Thank you! Your office has been supporting countries in producing labour market statistics. I imagine that you had to react quickly when COVID-19 hit? Can you tell us a little bit of what the countries asked and how you reacted to their requests?
Kieran: It is fair to say the entire COVID experience has been a rollercoaster for all countries and it was challenging in various ways. We received many requests from countries in the early days of the pandemic but there were a few main themes.
The first was how to keep data collection going, particularly for those countries relying on face-to-face interviewing.
The second was on interpreting the international statistical standards and definitions during COVID-19 where massive restrictions on movement forced many people out of their workplaces. A key question in particular was whether those people should be considered as temporarily absent from work, and thus still employed, or not. This was covered by international standards but some additional clarity was needed.
Related to this, the third area was how to collect and disseminate additional information needed to understand the impact of the pandemic on the labour market. Existing key labour market indicators remained very important but users demanded additional information on issues such as remote working, working time, and job losses etc. Information on these topics were to varying degrees not available or not refined to illustrate what was going on during the pandemic.
Responding quickly to these questions was quite a substantial task for many of us in the Department of Statistics. We participated in many virtual sessions in collaboration with other agencies, regionally, globally or bilaterally with countries to try to spread guidance and advice as widely as possible. We also learned a lot from others on how they were dealing with the challenges. Many team members were engaged to quickly publish quite a lot of guidance reacting to the demands, for example to support the continuation of data collection through telephone interviews or other means, or to provide definitions for issues such as remote working.
ISWGHS: Would the covid lesson impact on how you support countries?
Kieran: Definitely and in different ways. Firstly, just how we engage with countries has changed a lot. Longer in-country visits have been replaced with shorter and more regular online contacts. This has significant benefits but we also feel the loss of in-country contact now. I think a hybrid mode of engagement will be the best approach.
Another point which I already mentioned but bears repeating is the need to work collaboratively across agencies, which we have experienced during the pandemic, and should definitely build on.
Otherwise, I really think it’s important to share practices across countries, as they are the ones directly facing the challenges and coming up with solutions – we can support that to some extent but we also can achieve a lot by making sure those solutions are shared.
ISWGHS: Now I would like to move to the work of the Inter-Secretariat Working Group on Household Surveys. Very happy to see that the report had mentioned the need for further guidance to assess survey quality, as you know we are working on a Guidance Note on this topic. Any other activities where you think our members could work together to support countries post-covid?
Kieran: That is a very good question. The ISWGHS has definitely played a role in the increased collaboration across agencies that I have mentioned earlier. The ISWGHS’ COVID-19 related outputs are a great example - information is being compiled across agencies to an extent we have not generally seen in the past. We absolutely should continue that. That could go in many directions, for example working together to develop guidance and support to enable telephone interviewing in more settings in the future is something which comes to mind – not particular to any agency but relevant to all, and clearly highly in demand.
We should be working collaboratively on how to plan a household survey system over a number years rather than on a survey-by-survey basis. Countries with more developed systems already do this generally, but those with less resources and less frequent surveys cannot always do this and miss out on the synergies this can create. As countries face major resource constraints it cannot be realistically expected that they will spend significantly more funds on household surveys in the future than they currently are. It is therefore important that funds already allocated for household surveys are well utilized. Coherent planning of household surveys across a period of years is one of the ways to make this possible. This is already under discussion by the ISWGHS and I really think it is crucial.
Resources:
Mr. Kieran Walsh is a senior statistician in the Department of Statistics of the ILO. He works on on labour force survey methodology, particularly to support the implementation of latest standards and good practices in labour force surveys. Kieran has been with the ILO for 7 years, before that he worked in the Central Statistics Office of Ireland for 13 years, covering various topics such as labour market and earnings statistics, income and poverty, price statistics and others.