[Adapted from General Accounting Office (1991). "Using Structured Interviewing Techniques." Washington, DC: Program Evaluation and Methodology Division Report 10.1.5.]
One key element in conducting useful research is gathering reliable information. And the basis for doing that is designing questions and questionnaires that get the kind of information from which the researcher can draw valid conclusions.
When one looks at a completed questionnaire or the results of a valid study, he or she is often tempted to say, "That's not difficult to do," but that temptation quickly passes after a few minutes of closer analysis. That's about all it takes to realize that designing a good question and good questionnaire requires more thought and time than one might originally think. There are an almost infinite variety of things to think about and to do correctly to avoid the kinds of errors that can make scores of hours of work worthless.
This digest looks at some of the basic building blocks of a structured interview, points out potential pitfalls, and suggests ways for the researcher to avoid them, in order to produce a set of questions that have the best possibility of generating reliable, accurate data on the topics of interest.
Data-collection instrument (DCI)--A document containing questions presented in a systematic, highly precise fashion. The DCI's purpose is to enable the evaluator to obtain uniform data that can be compared, summed, and, if it is quantitative, subjected to additional statistical analysis.
Structured interview--One that uses a DCI to gather data, either by telephone or face to face. In a structured interview, the evaluator asks the same questions of numerous individuals in a precise manner, offering each individual the same set of possible responses. (In contrast, an unstructured interview contains many open-ended questions that are not asked in a precise, structured way.)
Computer-assisted telephone interview (CATI)--A form of telephone interview where the DCI is stored in a computer and the interviewer records responses directly into the computer.
Face-to-Face Interview
- Enables the interviewer to establish rapport with the respondent
- Allows the interviewer to observe as well as listen
- Permits more complex questions to be asked than in other types of data collection
- An effective method of gathering data when the DCI is lengthy
- Some uses: to get before-and-after data about a lesson module or a change in administrative procedure; to gather opinions on a specific learning or teaching technique
Telephone Interview
- Less costly than personal interview
- Takes less time than a personal interview
- Simplifies recording of data if CATI is used
- Most effective when the number of questions is relatively small and time available to gather data is short
- Some uses: gathering information on parent satisfaction with the educational program; determining awareness of programs and tools available in the school district
Mail Questionnaire (not a structured interview method)
- Least costly but slowest method of collecting data
- Requires precise question design to match reading comprehension of respondents
- Some uses: to get demographic information on district residents; to get opinions on potential new programs, if they can be clearly described in the questionnaire
DESIGNING A STRUCTURED INTERVIEW The first step in designing the interview is to formulate the broad overall questions that the survey is intended to answer, i.e.,
- Why is the study being undertaken?
- What does the study aim to learn or determine?
- - To describe something that has occurred; e.g., How do parents view the value of computer use as an aid to teaching math in elementary grades?
- - To compare results to some standard (a normative study); e.g., Has the use of computers as a teaching aid matched the goals of the teachers using these tools?
- - To determine if a procedural change has made a difference (a cause-and-effect study); e.g., Has the use of computer-assisted instruction changed student attitudes about learning math?
The second task is to translate the broad overall questions into measurable elements as hypotheses or more precise questions. The descriptive question above, for example, would require developing measures such as parents' relative knowledge about computer use in math instruction, the importance parents place on math learning, whether parents believe their children learned more with or without the use of computer technology, to name a few.
Then, the target population needs to be identified. If computer-assisted math instruction is new only to a certain grade or school, care needs to be taken to interview only the parents whose children have been affected by the program change. It may sound basic, but innumerable studies have proceeded to unsatisfactory completion because of insufficient consideration of this aspect.
Now, the study can proceed to the development of a pool of specific questions designed to elicit the desired information. The number of questions developed should be more than the number to be asked; then, the most appropriate and useful can be selected from those available.
- Relevance--Questions should be directly related to the purpose of the study and have a good probability of yielding the kind of data desired.
- Selection of respondents--Even though a question may be relevant to the study, it may not be answerable by the people to whom it will be asked.
- Ease of response--Questions need to be relatively easy to answer and should not create embarrassment for or an undue burden on the interviewee.
Among the types of questions that should be avoided are those that require respondents to consult records or other information sources, would make them uncomfortable for any reason, would reflect negatively on them, would make the interview confrontational, or have no specific answer.
- Open-ended--Because open-ended questions provide no structure for the answer, they should be tightly focused to elicit the kind of information the researcher wants to get. And, because they require accurate and time-consuming transcription, their use should be limited to initial research where the number of respondents is small and the object is to refine the research direction and determine more precise questions that can be structured another way.
- Fill-in-the-blank--This type of question has a simple answer, usually a name, frequency, or quantity, which is the kind of information these questions are good at obtaining.
- Binary--These are good for obtaining factual information that falls into the yes-no, true-false category of answer.
- Scaled-response--These consist of a list of alternative responses that increase or decrease in intensity in an ordered fashion. These kinds of questions can be further defined as balanced, unbalanced, and rating and ranking.
- Unscaled-response--With this type of question, the respondent is asked to choose one or more options from a list; this is the type of question that should include the ootheroe category so that the responder is not forced to select an answer with which he or she is not completely satisfied.
The reader should be alerted to the fact that this digest provides only an introduction to structured interviewing. Success in developing and conducting structured interviews requires consulting and studying additional references and reliance on the assistance of experts in the field.
Hollwitz, J. & Wilson, C.E. (1993). "Structured Interviewing in Volunteer Selection." Journal of Applied Communication Research, 21, 41-52.
General Accounting Office. (1991). Using Structured Interviewing Techniques. Washington, D.C.: Program Evaluation and Methodology Division (http://www.gao.gov/policy/10_1_5.pdf).
Pawlas, G.E. (1995). "The Structured Interview: Three Dozen Questions to Ask Prospective Teachers." NASSP Bulletin, 79, 62-65.
Watts, G.E. (1993). "Effective Strategies in Selecting Quality Faculty." Paper presented at the International Conference for Community College Chairs, Deans, and Other Instructional Leaders. Phoenix, AZ.
This publication was prepared with funding from the Office of Educational Research and Improvement, U.S. Department of Education, under contract RR93002002. The opinions expressed in this report do not necessarily reflect the positions or policies of OERI or the U.S. Department of Education. Permission is granted to copy and distribute this ERIC/AE Digest.
###