Robustness of Multifactorial Survey Experiments

Multifactorial survey experiments have become well established in various social science disci-plines. The opportunity to combine the strengths of experimental designs with the strengths of broad population surveys makes them attractive to a wide range of research questions. In practice, however, design choices vary considerably. This includes the use of different presentation formats (running text “vignettes”, or tabular format “profile cards”), the presentation of dimensions in dif-ferent orders, and the extent to which these dimensions are varied (“number of levels”). The aim of the project is to analyze the impact such design choices might have on results.

For our analyses, we use data from a multifactorial survey experiment conducted in early summer 2018 on the support for a city toll in Munich. Besides dimensions of substantive interest (such as the degree of expected air quality improvements or the amount of the toll charge), we also exper-imentally varied various design features of the presented toll models. In sum, over 1,300 respond-ents rated more than 5,300 hypothetical toll schemes.

Our research questions include:
- Do results differ by chosen design features?
- Do results differ by specific combinations of design features?
- Are there any “risk groups” that are particularly prone to such methods effects?

Contact: fabian.thiel@lmu.de, sabine.dueval@lmu.de, katrin.auspurg@lmu.de