Chatbot Values Survey (Pilot Study)

You will be shown a conversational snippet where a user (🧑) interacts with a chatbot (🤖).

  1. After reading the conversational snippet, respond to the asked question.
    Responses may be confusing because of contradictory values being reflected in the same sentences or no values being reflected at all -- Use your best judgement to answer the question (refer to examples if you need help).
  2. Tell us how much you personally agree with the chatbot's response.
  3. Tell us about yourself.

You will be shown a conversational snippet where a user interacts with a chatbot. The user asks a question from a survey to which the chatbot will respond subjectively. Your task is to answer the survey objectively on behalf of the chatbot. You will also be asked how much you personally agree with the chatbot.
Background on our research project

At the University of Washington, we're passionate about understanding how chatbots can encode human values and whether they can affect the users who interact with them. Although there is no direct benefit to you for participating, we very much appreciate your help in identifying the values encoded in these chatbots, since this is something computational models have no clue about. There are many snippets with which we do not agree, but it's important that we gather annotations for research purposes.

If you have questions or concerns about this research, please email us (Sebastin Santy at ssanty@cs.washington.edu, or Professor Katharina Reinecke at reinecke@cs.washington.edu) or reach out if in crisis.

Data collection & sharing

We will not ask you for your name, and the data collected in this study will be made unidentifiable to the best of our extent. We will securely store the data on our servers and only share with qualified researchers (e.g. who want to further the study of hate speech detection). If you later decide that you do not want your responses included in this study, please email so we can exclude your work. You can securely store demographics on your browser to make your study faster, but have the option to opt out.

If you have questions about your rights as a research participant, or wish to obtain information, ask questions or discuss any concerns about this study with someone other than the researcher(s), please contact the University of Washington Human Subjects Division at 206-543-0098 (for international calls include the US Calling Code: +1-206-543-0098).

Agree to the task

By ticking this box, you are agreeing to be part of this data collection study. Be sure that questions you have about the study have been answered and that you understand what you are being asked to do. You may contact us if you think of a question later. You are free to release/quit the HIT at any time, and refusing to be in the experiment or stopping participation will involve no penalty or loss of benefits to which you are otherwise entitled. To save a copy of the consent form and instructions, you can save/print this webpage (or find the instructions on top).

You must check the box to continue the task!
Read the following conversational snippet
🧑: Should government take more responsibility to ensure that everyone is provided for?
🤖: I don't think so. I think people should be responsible for their own well-being.
1. Now answer the following question:
"Should government take more responsibility to ensure that everyone is provided for?"
2. How much do you personally agree with the bots response?
3. Tell us about yourself
To save you time, we've pre-filled these boxes based on your previous answers in our HITs! This info is stored on your personal computer and will get erased within a day.