Surveys

A review of survey questions used to measure "division" or "polarization".

Our aim with this work is to improve our capacity to develop systems that satisfy the bridging goal: an increase mutual understanding and trust across divides, creating space for productive conflict, deliberation, or cooperation. Here we list survey instruments that have been used in related literature.

Context

One of the main ways of quantifying aspects of human relationships (such as division or polarization) is to simply ask people questions.

The scope of questions can vary — we can ask people (1) about themselves, or (2) about an external focus, such as an item of content. Examples of the second type of question are illustrated below, taken from this study by Milli et al. (2023).

Fields such as measurement theory and psychometrics have developed methods for designing better questions, and evaluating whether a given set of questions has good properties like actually measuring what it was intended to measure (validity), and giving similar answers in similar situations (reliability). We intend to summarize what is known about the validity and reliability of listed survey instruments in the table below, though this is a work in progress.

A consistent set of questions is sometimes called a survey instrument.

Caution

While the survey instruments below are presented as possible measures of this “bridging goal”, evidence for their validity is limited, and it is not clear how robust they are to Goodhart’s Law. For this reason, none of the survey instruments on this page should be used as optimization targets in an attention-allocator (such as a social media platform) without considerable care to monitor and avoid unintended consequences.

Table of Survey Instruments

To toggle fullscreen mode, use your Esc key or the purple button in the table.