Qualtrics Tutorials: Item Coding

When you create a multiple choice question on a Qualtrics survey, Qualtrics automatically assigns a numeric code to each answer. When your survey data is exported to Excel or SPSS, those numeric codes appear in the dataset as "shorthand", and the text of the responses are used as value labels. This has the benefit of simplifying data analysis, especially if your items' responses are ordinal.

There are several things to keep in mind when creating multiple choice items in Qualtrics:

• When creating the responses to a multiple choice question, numeric codes are automatically assigned to each response in increasing numeric order (1, 2, 3, ...)
• If responses to a multiple choice question are added, deleted, and replaced with new options, the numbering will continue to increase sequentially.
• If you change the order of the responses for an item, the numeric codes associated with each response will stay the same. This may mean that the numeric codes may not be ordinal with respect to the order that the responses appear in.
• You can safely change item coding anytime before your survey launches. However, after you begin collecting data, DO NOT change the item coding or alter/delete existing answer options. (Doing so risks losing or altering the meaning of the items' codes, making your results uninterpretable.)

A reader asked: How can I apply the information in this tutorial to the Slider question type in Qualtrics?

As of this writing (May 2022), Qualtrics's Slider question type does not include the same type of recode functionality that Multiple Choice questions have. Part of this has to do with the design philosophy of the Slider question type: while it can be made to capture ordinal-scaled answers, it is generally intended to be used to capture "true" numeric data (i.e., interval- or ratio-scale measurements). By contrast, Multiple Choice questions are generally intended to be used with nominal- or ordinal-scaled answers.

If reverse-coding is desired for Slider questions, it must be applied using alternative means, such as using embedded data variables in the Survey Flow to compute a new variable using a formula, or performing the calculation in a statistical software after data collection.

One common validation technique for survey items is to rephrase a "positive" item in a "negative" way. When done properly, this can be used to check if respondents are giving consistent answers.

For example, consider the following two items concerning "extraversion" [1]:

For question 1, "agree strongly" corresponds to the "most extraverted" option, and "disagree strongly" corresponds to the "least extraverted" option. However, for question 2, "disagree strongly" corresponds to the "most extraverted" option, and "agree strongly" corresponds to the "least extraverted" option. We say that question 2 is reverse-coded.

The need for an item to be reverse-coded is specific to the instrument and the factor of interest: in particular, what "high" and "low" scores on the instrument are intended to convey.

[1] John, O. P., Donahue, E. M., & Kentle, R. L. (1991). The Big Five Inventory - Versions 4a and 54. Berkeley, CA: University of California, Berkeley, Institute of Personality and Social Research.

Qualtrics automatically assigns numeric codes for each multiple choice item you create, but it's possible to modify those codes manually.

1. While editing your survey, click the item whose coding you want to modify. Depending on whether you are using the old survey editor interface or the new editor interface:

   Old Survey Editor
   (April 2021 and earlier):

   Click the Advanced Question Options button (the gear icon), then click Recode Values.

   

   New Survey Editor
   (May 2021 and later):

   In the Edit Question menu, locate the Question Behavior section, then click Recode Values.

   

2. The Recode Values window will open. When the window first opens, neither Recode Values nor Variable Naming will be checked off.

   

3. When you check the box next to Recode Values, you will see text boxes appear next to the responses. Here, you can change the numeric code associated with any of the responses. Note that the code assigned to each response must be unique; you can not assign the same code to more than one response.

   

4. When you check the box next to Variable Naming, another set of text boxes will appear. By default, the text in these boxes will be identical to what the response text is.

   

   The text in the Variable Naming box determines what the value labels will be set to when your data is exported to SPSS or Excel. This setting will not change the text that shown to users on the public version of the survey: it merely changes the value label used in the exported survey dataset.

As of May 2021, the new survey editor interface includes a new option to automatically update the item codes for all questions on the survey. This tool is especially helpful if your process of setting up your survey resulted in "messy", non-sequential item codes!

To update the item codes for all questions on the survey, click Tools > Reset Recode Values:

The Reset Recode Values window will appear. There are two choices:

• Clear recode values for all questions will "throw out" any item codes you've manually entered. They will return to the "original" item coding that was assigned when you first created the question.
• Reset recode values to sequential numbers for all questions will automatically update the item coding to be sequential for all applicable questions on the survey. The first option will always have the code 1; the second option will always have the code 2; and so on. (If you have manually specified any item codes, this option will over-write them! So if you plan to manually assign any item codes, we recommend applying this reset first.)

See also: Auto-Number Questions (Tools > Auto-Number Questions).

Accessing Built-In Answer Scales

Qualtrics has many built-in answer scales for multiple choice items. These built-in answer scales include commonly used unipolar and bipolar scales, as well as answer categories for common demographic questions.

In the old survey editor, these built-in options for multiple choice questions can be accessed by clicking the Automatic Choices option in the sidebar menu. In the new survey editor, these can be accessed by clicking the Use suggested choices option in the sidebar menu.

Default Item Coding for Built-In Scales

When using Qualtrics's built-in Likert items, the response options are always ordered from "negative" to "positive". (The meaning of "positive" or "negative" will vary between scales, but in general, "positive" categories correspond to agreement or high frequency, while "negative" categories correspond to disagreement or low frequency.) Additionally, "negative" answers are coded using smaller numbers, while "positive" answers are coded using larger numbers. Another way of thinking about this is that "negative" answers have a lower score, while "positive" answers have a higher score. For example, Qualtrics's built-in automatic scale for Disagree-Agree scales uses the following order and coding:

Using the Reverse Order option for Built-In Scales

When you have enabled "automatic choices"/"suggested choices" option, the Reverse Order option will be available to reverse-order the items. This means that the responses will be listed from "positive" to "negative". This will also change the coding order, so that the "positive" answers are coded using smaller numbers, and the "negative" answers are coded using large numbers. This is what the built-in Disagree - Agree scale looks like after turning on the Reverse Order option:

To reverse the display order for a Likert item:

Note that using this method to reverse-code items changes both the display order and the coding of the response options. This means that if you have a mixture of reverse-coded and non-reverse coded items, using this method to reverse-code your items will mean that some items have the negative choices presented first, while other items will have the positive choices presented first. This inconsistency may confuse respondents. For this reason, we do not recommend using Reverse Order to implement reverse coding unless it will be applied to every Likert item on the survey.

You can verify how responses have been coded by looking at the Recode Values section (described in the Manually Changing Item Coding within Qualtrics section above). In the Recode Values window, select the Recode Values check box. This will display the numeric codes assigned to each response option.

New Suggested Choices Options: "Show All" versus "Show First and Last"

As of May 2021, the new survey editor interface includes new options for suggested choices (formerly "automatic choices"): Show All and Show First and Last.

• Show All displays a descriptive text label for all answer choices.
• Show First and Last displays a descriptive text label only for the first and the last answer choices. No text labels are displayed for the choices in between.

For example, here's what the built-in Disagree-Agree scale looks like with Show All:

All five scale points have text labels attached.

Here's what the built-in Disagree-Agree scale looks like with Show First and Last:

Only the first and last scale points have text labels attached; all points in between are unlabeled.

These differences in labeling can have an effect on how respondents answer the questions. If you are implementing an existing, validated survey instrument in Qualtrics, you should follow the text labeling protocol that was used when the instrument was validated. For example, only use the Show First and Last option if the original, validated instrument used that labeling.