Impact of Covid-19 on Survey Methods and Challenges

Introduction

Until 19 century the mainstream mode of collecting information was complete enumeration, which was replaced by sample surveys based on probability sampling in 1930s. Since then probability sampling became a standard approach for data collection and statistical inference in the area of survey research. Survey methods has been developed in chronological order, mail surveys in 1950s, telephone surveys in 1960s, computer-assisted surveys in 1980s, internet, web surveys in 1990s and mobile surveys in 2010s. In 2020, the outbreak of the COVID-19 occurred and survey research is in the midst of the COVID-19 pandemic.

The emergence of the COVID-19 has spurred the change of survey methods because the COVID-19 gave direct impact on field works, especially face-to-face surveys. Survey profession is faced with a challenge of changing data collection mode.

Another challenge is raised in the more use of survey data with other data sources like administrative data, transaction data, SMS data, web data and so on because a lot of massive data different from survey data have been produced rapidly in many areas of our society.

Impact of COVID-19 on Survey Methods

In May 2020, about 70% of face-to-face data collection was stopped and about 30% still stopped in July by the COVID-19. Many countries adapted quickly to challenges by changing data collection mode or use alternative source, adding COVID-19 related questions, reducing questionnaire content and so on [1].

The COVID-19 pandemic has disrupted the fieldwork of surveys in many countries. Face-to-face surveys has been affected especially, since interviewers usually conduct surveys in respondents’ home [2]. COVID-19 had a net negative effect on response rate and the cost were high of adapting to COVID-19 and providing paid timeoff benefits to staff affected by the pandemic [7]. The survey on ‘how understanding society in UK’ were transitioned to a different protocol without face-to-face interviews [4].

Transition to Web Surveys

Web survey is a survey method where questionnaires are sent over the web to respondents and they can respond to this survey over the web. Web surveys can generally be conducted using e-mail lists, internet user panels or publication of the survey’s link on websites [9].

Web surveys have advantages like lower cost, speed of data collection, ease of implementation, computerization of the questionnaire, multimedia mode and time and geographic flexibility [12]. Two advantages of lower cost and ease of implementation may explain the popularity of web surveys during the COVID-19 pandemic [9].

However, web surveys have also limitations such as lack of Internet-coverage, lack of a single registry of internet users, quantification of non-response, selection bias, the possibility of multiple responses of a single user, meaningless data [9,12]. To become a major mode of data collection in survey research, these limitations in web surveys should be overcome.

From the inferential viewpoint, selection bias especially in official statistics is a major drawback in web surveys [6,12,13]. To reduce selection bias, weighting adjustment methods may be used. Post-stratification or weighting class adjustments, raking or rim weighting, generalized regression modeling, and propensity score adjustment, pre-recruited panels of full population are sorts of adjustment methods [6].

Transition to Combining Data from Multiple Sources

Currently there are two trends of falling response rates and higher costs of data collection in survey research. Many survey statisticians have pointed out surveys may be unsustainable under such trends, even in the COVID-19 pandemic. Two directions may provide solutions, first one is transition of data collection methods to online surveys and second one is integrating survey data with other data sources.

Combining survey data and other data source like administrative data, transaction data, social media data etc. is to produce more efficient information [3,10,11,14]. Methodologically record linkage and statistical matching techniques can be used to combine multiple data sources. Record linkage is a method for linking two files A and B using quasi-identifiers such as name, address, data of birth, and other fields [8].

Statistical matching, also known as data fusion, data merging or synthetic matching, technique is a relatively new area of survey research which has been receiving attention in response to a lot of available data [15]. The goal of statistical matching is to provide joint information on variables and indicators collected through multiple data sources. In statistical matching, two main approaches can be used in terms of outputs that can be obtained through matching. The macro approach refers to the identification of any structure that describes relationships among the variables not jointly observed of the data sets. The micro approach refers to the creation of a complete micro-data file from multiple data [15].

For matched data related questions are still holds. How can we measure the quality of estimates based on matched data and can we develop a theoretical model for matched data? [5].

Summary

The Covid-19 has given great impact on field work especially face-to-face surveys in survey research. To overcome such difficulty, two directions are discussed: first one is transition to data collection methods like web survey and second one is integrating survey data with other data sources to produce more efficient information.

References

• United Nations and the World Bank (2020) Impact of COVID-19 on survey data collection and assistance from the international community. Presentation Report.
• Gummer T, Schmiedeberg C, Bujard M, Christmann P, Hank K, et al (2020). The impact of Covid-19 on field work efforts and planning in pairfarm and FReDA-GGS. Survey Research Methods 14(2): 223-227.
• Thompson ME (2019). Combining data from new and traditional sources in population surveys. International Statistical Review 87: S79-S89.
• Burton J, Lynn P, Benzeval M (2020). How understanding society: the UK Household Longitudinal Study adapted to the Covid-19 pandemic. Survey Research Methods 14(2): 235-239.
• De Waal T (2015). Statistical matching: Experimental results and future research questions. CBS discussion paper 2015-2019.
• Greenacre ZA (2016). The importance of selection bias in Internet surveys. Open Journal of Statistics, 6(3): 397-404.
• Sastry N, McGonagle K, Fomby P (2020). Effects of the Covid-19 crisis on survey fieldwork: Experience and lessons from two major supplements to the U.S. Panel Study of Income Dynamics. Survey Research Methods 14(2): 241-245.
• Winkler WE (2014). Matching and record linkage. Research report.
• De Boni RB (2020). Web surveys in the time of Covid-10. Cad Saude Publica.
• Lohr S, Raghunathan TE (2017). Combining survey data with other data sources. Statistical Science 32(2): 293-312.
• Rao JNK (2020). On making valid inferences by integrating data from surveys and other sources. Shankhya B 83: 242-272.
• Couper MP (2011). Web survey methodology: interface design, sampling and statistical inference. Eustat Report 6(3).
• Bethlehem J (2010). Selection bias in web surveys. International Statistical Review 78: 161-188.
• Conti PL, Marella D Scanu M (2016). Statistical matching analysis for complex survey data with applications. Journal of the American Statistical Association 111: 1715-1725.
• D Orazio M, Di Zio M Scanu M (2006). Statistical Matching. Wiley.