Using the Bias Detection app

What does this app do?

This app checks whether the formulation of survey questions might induce biased responses. For questions where it suspects a bias, it will suggest a reformulation of the question.

The app is currently trained to detect and reformulate leading, double-barreled, loaded, overgeneralized, ambiguous, negatively framed and gendered questions.

Small changes in the formulation of a survey question can lead to radically different answers. When developing a survey, that you’ve looked at a hundred times, it is sometimes difficult to spot issues with the formulation of questions.

This app provides you with an additional layer of checks. It’s not perfect, and can make mistakes, but it can also help you identify issues with your survey that you might otherwise have missed.

What do I need to know about the input and the output?

The app works with a toggle, allowing you either to enter one question at a time through a text box, or multiple questions in one go, using a file upload.

If you chose the “Input text” option, the input and the output will be text. The app will only work if you input one question at a time. For example if you enter a leading question, like “Don’t you think this policy is a good idea?”, the app might respond: “The original question is biased due to its leading nature. It has been reformulated to be more neutral and unbiased. Suggested reformulation: What do you think of this policy?”.

If you chose “File upload”, both the input and the output will be a file. To prepare the input, you will need to put together an Excel sheet with two columns. Column 1 should contain a unique identifier for each question, while column 2 should contain the corresponding questions or survey items. Be sure to provide a title for each column.

The app will then loop through each question, check whether any of the questions might induce a biased response, and suggest a correction if they are. You can visualize the output in the app directly or by downloading the output sheet, which will only list questions with a suspected issue.

Please remember that this app can make mistakes. It might not capture all types of baises in questions and it might also identify questions as being biased when they are not. We do think this app can be very helpful, especially when you’re reviewing many questions and would like to quickly identify and correct potential issues.