Volume 2, Issue 2
March-April 2001
Page Navigation Panel
Page 1 2
Inside this Issue 1
Survey Sampling 1
The Focus Group 2
Inside the next issue
Juliette Poulson, RN, MN.
Director, HCA
Douglas Barton
Deputy Agency Director,
Behavioral Health Services
Behavioral Health Services
Quality Improvement and Program Compliance
405 W. 5 th St., Suite 410
Santa Ana, CA. 92701
714-834-5601
Survey Sampling and Administration-Jonathan Rich, Ph. D.
In a previous article, I talked about designing useful survey questions. Survey questions should be easy to understand and should not bias the participants. However, the method of choosing participants and administering the survey is just as important as the questions themselves.
A survey is always administered to a group of individuals. However, the group taking the survey rarely includes everyone of interest. The participants in a survey are usually considered to be a sample of a larger population. The population is everyone of interest to you – all the people about whom you would like information. The population of interest is often very large, and so a sample, or subset, of the population is surveyed to save time and money. It is also likely that you will not be able to access some members of your population. Some people may refuse to participate and there may be others whom you are unable to contact. A random sample is generally considered to best represent a population. This is a sample
that is drawn in such a way that every population member has an equal chance of being included. If the name of every population member is written on a slip of paper, and then the slips of paper are drawn from a hat, this would be a random sample. While the concept is simple, in practice it is very difficult and sometimes impossible to draw a random sample. Since all of the population members are often not known, one cannot simply drop their names in a hat. Furthermore, the ones who are willing to participate and are reachable may have different attitudes and opinions than the ones who do not participate or can not be reached. As an unfortunate rule of thumb, more effort and time is required to draw a good sample than a bad or biased sample. Below I have listed some sampling methods and their potential pitfalls:
"If you survey, they will come."
One way to obtain survey participants is by posting a notice or advertisement, leaving survey forms in a publicly accessible area, or by setting up a web site. Since the participants then come to you, relatively little effort is needed to obtain a sample.
HIPAA-Health Insurance Portability and Accountability Act of 1996 Using Statistics in Mental Health
1
1 Page 2 3
2
Volume 2, Issue 2
March-April 2001
However, this sample is unlikely to represent the population you want to know about. The participants will be taken only from people who notice your solicitation, not the larger population of interest. The volunteers are also likely to be unusually interested in the survey subject matter, and may be motivated to participate because they have unusually strong opinions, either positive or negative. If you offer a small amount of money to participate, persons with limited finances are more likely to respond. An Internet survey is likely to be very atypical, since persons without computer access and knowledge will be excluded.
"With a little help from my friends…"
Another readily available group of participants can be found among people you know personally. However, you have unique personal characteristics that bring people into contact and into relationships with you. The sample you draw is a sample of your population of friends, but is unlikely to be generalizable to any population of general interest.
"Thanks for coming…"
A common method of administering satisfaction surveys is to give them to persons after a specified number of visits. This is not a bad sampling method; you are likely to get a reasonable cross-section of program attendees. However, if surveys are administered only to persons who have been involved in a program for two months, it must be noted that the results can't be generalized beyond this group. Disgruntled clients may drop out before the survey is given. Persons who are unhappy with the way they are treated when first contacting a clinic may never come in and never be surveyed.
"Don't mind me."
It is important that participants feel free to express their opinions. If they feel that negative reports could be used against them or could harm their relationship with a service provider, they may slant their reports in a positive direction. The last time I bought a car, the salesman handed me a satisfaction survey, stood next to me as I filled it out, and remarked, "We live and die by these surveys. If you're going to rate anything less than excellent, let me know so I can fix it." Needless to say, the salesman's methodology is an excellent way to generate high ratings, but a poor way to accurately measure opinions. The best sampling methods make an attempt to contact every member of the population. This often requires extra effort and expense. One must be careful not to systematically eliminate any subsets of the population. Thus, if you define your population as "every client, who has sought treatment at XYZ Clinic over the last three months," persons should not be eliminated because they do not read, do not have a phone, do not speak English, or do not come to the clinic on Friday mornings. Every member of your population should have an equal chance of being surveyed, even if persistent attempts to contact them are needed. The survey should be conducted in a way that elicits honesty and openness, and participants should be given assurances of confidentiality.
The Focus Group- Cort Curtis, Ph. D.
Focus groups are a cost effective means to find out what is truly in the mind of a consumer. The focus group is a relatively new phenomenon for gathering valuable data compared to surveys and polls, which have been around for years. Polls and surveys typically elicit quantitative data to determine trends and percentages while focus groups are more of a qualitative research tool that elicit the experience of the individual. If you really want to know what is in the heart and mind of a consumer, you ask them and you ask the kinds of questions that elicit an open rather than a forced choice response. Forced choice responses limit the total experience of the individual and in fact may have very little to do with the individual's actual experience of the product or service involved.
2
2 Page 3
3
Volume 2, Issue 2
March-April 2001
Forced choice surveys are typically for the convenience of the researcher since it gives easily analyzable data that can be quantified and manipulated. Qualitative research instead demands a little more skill from the interviewer to fully inquire into the consumer's viewpoint and feelings without unconsciously directing the consumer toward a certain type of answer.
Participants in a focus group share some common characteristics relevant to the evaluation in question. Originally used as a market research tool to investigate the appeal of various products, the focus group technique has been adopted in a variety of other fields, such as education and politics. It is a well-known fact that the Clinton Administration relied heavily on focus groups to determine individual attitudes and it may be the way he has established a reputation (by some) for being "in-tune" with the heart of so many Americans.
What happens in a focus group session?
The focus group typically involves six to ten individuals who are guided by a skilled moderator through a series of open-ended questions meant to elicit a person's opinions, knowledge, perceptions, concerns and emotional responses in regard to the topic at hand. The focus group session is not a discussion group, problem-solving session, or decision-making group. Yet the results obtained from the focus group capitalize on group dynamics. The hallmark of focus groups is the explicit use of the group interaction to generate data and insights that would be unlikely to emerge without the interaction found in a group. The technique inherently allows observation of group dynamics, discussion, and firsthand insights into the respondents' behaviors, attitudes, language, etc.
Focus group participants are typically asked to reflect on the questions asked by the moderator. Participants are permitted to hear each other's responses and to make additional comments beyond their own original responses as they hear what other people have to say. The objective is to get high-quality data in a social context where
people can consider their own views in the context of the views of others, and where new ideas and perspectives can be introduced.
When to use focus groups
Focus groups can be used in any situation where the researching entity is interested in understanding the inner reactions of a research population such as a consumer or a client. Some examples of appropriate research questions might be:
How does the consumer feel about this product or service? How does the consumer perceive us? Are we responding to the consumer's needs? What is really important to the consumer? What is the consumer looking for? Are we responsive to the consumer? When asking these kinds of questions, the researcher is viewing the subject of the research more as a collaborator in a particular project or context instead of merely a number. It is the relationship between the company and the consumer that is considered important. Whatever the context of the research, whether it is for a product or program evaluation the results obtained from the focus group research can help identify problem areas, strengths, weaknesses and generate new ideas that were previously not considered.
3
Volume 2, Issue 2
March-April 2001
1 2 3
