SEARCH UNIVERSITY LIBRARIES
When you create a multiple choice question on a Qualtrics survey, Qualtrics automatically assigns a numeric code to each answer. When your survey data is exported to Excel or SPSS, those numeric codes appear in the dataset as "shorthand", and the text of the responses are used as value labels. This has the benefit of simplifying data analysis, especially if your items' responses are ordinal.
There are several things to keep in mind when creating multiple choice items in Qualtrics:
One common validation technique for survey items is to rephrase a "positive" item in a "negative" way. When done properly, this can be used to check if respondents are giving consistent answers.
For example, consider the following two-item instrument concerning "healthy behaviors":
For question 1, "strongly agree" corresponds to the "healthiest" possible option, and "strongly disagree" corresponds to the "least healthy" possible option. However, for question 2, "strongly disagree" corresponds to the "healthiest" possible option, and "strongly agree" corresponds to the "least healthy" option. We say that question 2 is reverse-coded.
The need for an item to be reverse-coded is specific to the instrument and the factor of interest: in particular, what "high" and "low" scores on the instrument are intended to convey.
Qualtrics automatically assigns numeric codes for each multiple choice item you create, but it's possible to modify those codes.
While editing your survey, click the item whose coding you want to modify. Click the Advanced Question Options button (the gear icon), then click Recode Values.
The Recode Values window will open. When the window first opens, neither Recode Values nor Variable Naming will be checked off.
When you check the box next to Recode Values, you will see text boxes appear next to the responses. Here, you can change the numeric code associated with any of the responses. Note that the code assigned to each response must be unique; you can not assign the same code to more than one response.
When you check the box next to Variable Naming, another set of text boxes will appear. By default, the text in these boxes will be identical to what the response text is.
The text in the Variable Naming box determines what the value labels will be set to when your data is exported to SPSS or Excel. This setting will not change the text that shown to users on the public version of the survey: it merely changes the label used in the exported survey dataset.
Recall that Qualtrics has many built-in scales for multiple choice items. These can be accessed by clicking the Automatic Choices option in the sidebar menu.
When using Qualtrics's Automatic Choices for Likert-scale multiple choice items, the response options are always ordered from "positive" to "negative". Additionally, "positive" answers are coded using smaller numbers, while "negative" answers are coded using larger numbers. Another way of thinking about this, is that "positive" answers have a lower score, while "negative" answers have a higher score. For example, Qualtrics's built-in automatic scale for Disagree-Agree scales uses the following order and coding:
|4||Neither agree nor disagree|
The Automatic Choices menu has an option, Reverse Order, to reverse-order the items. This means that the responses will be listed from "negative" to "positive". This will also change the coding order, so that the "negative" answers are coded using smaller numbers, and the "positive" answers are coded using large numbers. This is what the built-in Disagree - Agree scale looks like after turning on the Reverse Order option:
|4||Neither agree nor disagree|
To reverse the display order for a Likert item, select the Automatic Choices check box, then select the Reverse Order check box at the top of the list. You can then select any of the pre-made scale options, and reverse coding will automatically be applied.
Note that using this method to reverse-code items changes both the display order and the coding of the response options. This means that if you have a mixture of reverse-coded and non-reverse coded items, using this method to reverse-code your items will mean that some items have the most negative choice presented first, while other items will have the most positive choice presented first. This inconsistency may confuse respondents. For this reason, we do not recommend using Reverse Order to implement reverse coding unless it will be applied to every Likert item on the survey.
You can verify how responses have been coded by clicking the (gear-shaped) Advanced Question Options icon and selecting Recode Values. In the Recode Values window, select the Recode Values check box. This will display the numeric codes assigned to each response option.
In late 2016, Qualtrics reversed the default "polarity" of their Automatic Choice scales. Prior to October 2016, Automatic Choice scales were ordered from "negative" to "positive", with negative items being scored lower. After October 2016, all Automatic Choice scales are now ordered from "positive" to "negative", with positive items being scored lower. Survey items created before this date are unaffected, but all new survey items created using Automatic Choices (even on pre-existing surveys) will have this new ordering.