A Methodological Framework for the Analysis of Panel Conditioning Effects

Panel conditioning refers to the phenomenon whereby respondents’ attitudes, behaviour, reporting of behaviour and/or knowledge are changed by repeated participation in a panel survey. Uncovering such effects, however, is difficult due to three major methodological challenges. First, researchers need to disentangle changes in behaviour and attitudes from changes in the reporting of behaviour and attitudes as panel conditioning may result in both, even at the same time and in opposite directions. Second, the identification of the causal effect of panel participation on the various forms of change mentioned above is complicated as it requires comparisons of panel respondents with control groups of people who have not been interviewed before. Third, other sources of error in (panel) surveys may easily be mistaken for panel conditioning if not properly accounted for. Such error sources are panel attrition, mode effects, and interviewer effects. In this chapter the challenges mentioned above are reviewed in detail and a methodological framework for the analysis of panel conditioning effects is provided by identifying the strengths and weaknesses of the various designs that researchers have developed to address the challenges. The chapter concludes with a discussion of a future research agenda on panel conditioning effects in longitudinal surveys.

Measurement Error in Longitudinal Data

Understanding change is essential in most scientific fields. This is highlighted by the importance of issues such as shifts in public health and changes in public opinion regarding politicians and policies. Nevertheless, our measurements of the world around us are often imperfect. For example, measurements of attitudes might be biased by social desirability, while estimates of health may be marred by low sensitivity and specificity. In this book we tackle the important issue of how to understand and estimate change in the context of data that are imperfect and exhibit measurement error. The book brings together the latest advances in the area of estimating change in the presence of measurement error from a number of different fields, such as survey methodology, sociology, psychology, statistics, and health. Furthermore, it covers the entire process, from the best ways of collecting longitudinal data, to statistical models to estimate change under uncertainty, to examples of researchers applying these methods in the real world. The book introduces the reader to essential issues of longitudinal data collection such as memory effects, panel conditioning (or mere measurement effects), the use of administrative data, and the collection of multi-mode longitudinal data. It also introduces the reader to some of the most important models used in this area, including quasi-simplex models, latent growth models, latent Markov chains, and equivalence/DIF testing. Further, it discusses the use of vignettes in the context of longitudinal data and estimation methods for multilevel models of change in the presence of measurement error.

Panel Conditioning in the U.S. Consumer Expenditure Survey

The U.S. Consumer Expenditure Interview Survey asks many filter questions to identify the items that households purchase. Each reported purchase triggers follow-up questions about the amount spent and other details. We test the hypothesis that respondents learn how the questionnaire is structured and underreport purchases in later waves to reduce the length of the interview. We analyze data from 10,416 four-wave respondents over two years of data collection. We find no evidence of decreasing data quality over time; instead, panel respondents tend to give higher quality responses in later waves. The results also hold for a larger set of two-wave respondents.

Data Collection as Disruption: Insights from a Longitudinal Study of Young Adulthood

Research disrupts the social world, often by making respondents aware that they are being observed or by instigating reflection upon particular aspects of life via the very act of asking questions. Building on insights from the first Hawthorne studies, reflexive ethnographers, and methodologists concerned with panel conditioning, we draw on six years of research within a community in southern Malawi to introduce a conceptual framework for theorizing disruption in observational research. We present a series of poignant-yet-typical tales from the field and two additional tools—the refresher-sample-as-comparison and study-focused ethnography—for measuring disruption empirically in a longitudinal study. We find evidence of study effects in many domains of life that relate directly to our scope of inquiry (i.e., union formation, fertility) and in some that extend beyond it (i.e., health). Moreover, some study effects were already known and discussed in the broader community, which was also affected by our research in unintended ways. We conclude that the assumption of non-interactivity in observational research is shaky at best, urging data-gatherers and users to think more seriously about the role of disruption in their work.

Item Nonresponse Rates and Panel Conditioning in a Longitudinal Survey among Youth

The article examines two important aspects of data quality in self-completion surveys of young people, taking advantage of a unique data source: Understanding Society: the United Kingdom Household Longitudinal Study. Young persons aged 10–15 are asked to complete a self-administered paper questionnaire at annual intervals. The number of completed interviews varies over waves from 4,049 to 5,020. Data are also collected from parents, providing important explanatory covariates for our analysis. Stronger parent–child relationship and higher mother’s involvement in education were associated with lower item nonresponse rate and lower inconsistency throughout waves. We also found some evidence for a negative panel conditioning effect with an increase of social desirability bias and measurement errors in the subsequent waves. There was a higher level of inconsistent responses and a higher probability of social desirability bias throughout waves in more sensitive items.