How does a survey strengthen a case?
Without hard data, the judge or jury will base much of its decision on the credibility
of witnesses, rather than easier-to-assess facts. And the opposition may likely find an
equally credible expert to present a very different opinion.
Surveys allow an expert to base his or her opinion on the kind of easy-to-understand
information that newspapers and television use every day. They enable experts to bring real world
information into the courtroom and can force the opposition to expend considerable
resources to understand the technical details of the survey and collect its own data.
How are legal surveys conducted?
Legal surveys are conducted using the same methods as other market research surveys. Survey data can be collected by phone, at a research facility, at a shopping mall, or over the web. Factors influencing the choice of which method to use include:
- The population you are trying to reach (if the survey is about Internet use, for example, it may be appropriate to administer the survey over the web)
- Whether there is a need to conduct the survey in person (respondents may need to see and touch products related to the survey questions)
- Geographical requirements
- Cost effectiveness
When should a survey expert be retained?
It is always best to involve the survey expert early. This way, he or she can become familiar
with the case, and Counsel can get the full benefit of the expert’s point of view. When
involved early, experts can assist Counsel in formulating questions for deposition and
determining which items and facts to obtain during discovery.
The decision to use a survey, however, can become part of the legal strategy at any point
in time. An expert should be prepared to work with the legal team under tight but realistic
timelines. This is frequently the case when hiring a survey expert to rebut or critique a
survey presented by the opposing side.
What are the key things to consider when choosing a survey expert?
When hiring a survey expert, the most important criteria to consider (in no particular
order) are testifying skill, credentials, expertise, and availability. Look for a survey expert
who can provide an admissible opinion under the Daubert-Joiner-Kumho gate-keeping tests
and remain cool when testifying under oath despite intensive questioning. The expert
should have prior court and deposition experience as well as strong academic and/or survey
practitioner credentials. Ideally, survey research experience should cover both survey
development and survey rebuttal. It is also helpful for an expert to understand economic
issues and the process for calculating damages.
Last but not least, an expert must be available and able to meet the demands of the
case in a timely and thorough manner. Experts often work in partnership with a market
research firm, and it is important to ensure that this firm is familiar with legal processes,
documentation, and communications, and can provide guidance to attorneys when needed.
How long does a survey take?
A survey project for litigation can take anywhere from one or two weeks to a few months to complete. You should expect to receive a firm time estimate from your survey expert or survey research firm up-front. This estimate is typically based on factors that affect the time it takes to design, execute, and analyze a survey. Examples of these factors include:
The type of exploratory research that may be needed to better understand:- The subject matter
- The language consumers use to describe their experience
- The characteristics that define the population of interest
- The best ways to reach the population of interest
- Ease of finding people who meet the qualifications for participation
- Level of cooperation (how busy are the qualified people and how interested are they in taking the survey). Survey panels, which pre-recruit people who are willing to participate in surveys, can help shorten the recruiting cycle.
- Time requirements can vary depending on whether data is collected over the phone,
at a research facility, at a shopping mall, or over the web. For example:
- Shopping malls may be limited by the number of eligible people they can recruit in a given day
- Many people can take a web survey simultaneously, making this a relatively quick method
- Telephone methods may depend on whether people are at home or in their office when called
How large a sample is needed?
Many people incorrectly assume that the number of interviews required for statistical
significance is a function of the size of the population of interest — the larger the population,
the larger the sample needed. In reality, relatively small samples, carefully
selected from large populations, can yield precise estimates of how the total universe of
potential respondents would respond. For example, political surveys of less than 1,000
respondents can predict the behavior of one hundred million voters with an accuracy of
plus or minus 3%.
While the “confidence interval” (the “plus or minus 3%” quoted above) does get
smaller and the precision of the answer improves as the sample size increases,
appropriately chosen samples can provide statistically significant results with far fewer
respondents. Aside from statistical significance, however, is the intangible factor of
having a sample size large enough to appear credible to a judge or jury. This desire for
“face validity” often leads to samples of 200 or 300 respondents. Another factor leading
to larger sample sizes is the particular analytical techniques used, as some may
require larger samples.
What is the main challenge in defining the survey population?
Survey researchers and experts are often challenged to define a population of interest
that will not be attacked as being either too narrow or too broad. While legal strategy
can guide survey experts in determining a population of interest, there are often a
number of subtle choices that experts make which lead to the ultimate definition of the
respondent population.
A good strategy for guiding these choices is to consider the allegation(s) in light of two
groups of people: those who may have been harmed and those who may potentially be
harmed. It is then helpful to list as many dimensions as possible characterizing people
within this spectrum (for example, how they are exposed to a product, how they
use a product, how they get information, etc.). This exercise is beneficial because
it exposes potentially subtle, yet important differences between segments, and then
enables the expert to create specific screening questions to target and capture the
relevant population.
How much do surveys cost?
To a great extent, survey costs are driven by the ease of reaching a respondent population,
the method of conducting a survey, and the sample size. Since these factors are quite
variable, typical surveys can range between $20,000 and $100,000. Factors that can
contribute to higher costs include the need to expedite a schedule or conversely, needing
to spend a lot of time during the planning stage to address complex issues.
While the cost for some surveys can be reliably estimated, others require additional
information in order to develop an accurate estimate. To lessen this uncertainty and
reduce costs, survey experts can conduct exploratory research to better understand
the respondent population and the potential issues in recruiting them. Another good
way to estimate and manage costs is to conduct a small-scale pilot research study and
then decide about further investment. Your survey expert can advise you as to the best
approach for your particular study.
