NCSC: Project Initiatives: Race and Ethnic Fairness Awareness

The major methods used in court opinion research are random surveys of the general public, focus groups, and exit-surveys. Random surveys give each adult in a state or locality an equal likelihood of being included. A properly conducted survey of 500 or 1000 people can accurately reflect the opinion of the entire state, county or city. Surveys of the general public are usually conducted by telephone.

Focus groups bring together a dozen or so individuals to participate in a structured and intensive discussion of specific issues. Participants can be selected at random or purposefully chosen because of their experiences or background. In some applications a focus group is brought together several times (Morgan, 1988; Krvegen, 1988).

Exit-surveys are distributed to jurors, litigants, or others at the conclusion of a case or as they leave the courthouse. Sometimes the surveys are completed on the spot; in other uses the individual completes the survey at home and returns it to the court by mail.[1]

Strengths and Weaknesses of Information Gathering Methods

Each method for surveying the public has strengths and weaknesses. The strength of public opinion polls is that the results represent the views of the public at large, at least if the persons interviewed are selected at random.[2] Each participant hears the identical questions and interviews can be conducted in languages other than English.

Weaknesses of opinion surveys include the costs involved and the length of time needed to draft the survey questionnaires. Opinion surveys tend to under-represent members of racial or ethnic minorities. Public opinion as derived from surveys is driven more by emotion than careful reflection. Opinions measured through one or two questions often change if the respondent is provided with policy-relevant information. National media influences tend to produce a stereotypical view of the courts and judges, one not tied to the circumstances of individual states or cities. Simple questions are easiest to answer in a telephone survey, but the responses can be superficial.

Focus groups do a good job at building an understanding the way people approach a topic, presenting their thinking in the round. The methodology is dynamic, revealing opinions as sharpened by debate, new information, and reflection.

Disadvantages of focus groups include their short duration and the influence of the focus group facilitator. Like public opinion surveys, the quality of the information is proportional to the time and care taken in selecting the questions. A professional focus group facilitator will tend to produce more informative results than using a court employee in that role. Paying a fee to participants is expensive, but tends to ensure that participants do not stray from the questions at issue.

Exit surveys are targeted to individuals with relevant court experience. Their views are grounded in how they were treated and how they saw others treated by the courts. Exit surveys can be relatively inexpensive to administer. Numerous examples of exit surveys are available for modification as needed.

Exit surveys can suffer from low response rates. Surveys handed out to court participants may not be completed on site or returned if taken from the courthouse. Care in designing the survey form and a strategy for handing them out in the courthouse will increase the response rate.

Quality Control

Survey research about the courts should be evaluated by the standards set for the conduct of opinion research generally. Primary resources easily accessed include the National Network of State Polls and the American Association for Public Opinion Research (AAPOR). Advice to court administrators on how to be savvy consumers of opinion research is also available (see Rottman 2000). In addition, a concise and authoritative guide to survey research written for use by judges and lawyers is available from the Federal Judicial Center (Diamond, 2000).

Quick checks on the quality of a survey include whether (a) the selection criteria for interviewees were specific in which household member is to be interviewed (avoiding biases associated with systematic differences in which household members typically answer the telephone), (b) several attempts to contact the chosen respondent were made, and (c) the survey was carried out by a university of commercial polling outfit. University and commercial research outfits follow standards that make the survey respondents representative of the group of interest, ask clear questions, and adhere to practices that raise the likelihood that people contacted will answer the survey. [3]

Making Opinion Research Affordable

Cost is always an issue when examining public opinion, especially when using telephone surveys. Rather than designing and fielding a special court survey, sets of questions can be added at reasonable cost to surveys of the general public carried out within regularly scheduled omnibus surveys conducted by universities and marketing research firms. Purchasing a fixed number of questions in such an on-going survey is typically affordable. A list of ongoing state-level opinion surveys is available at http://survey.rgs.uky.edu/nnsp, along with an archive of previous surveys. University professors and classes can undertake the research as part of a project in which they are allowed to use the findings for academic research.

Integrating Survey Findings into Policy and Improvement of Practice

Effective use of opinion research requires that courts integrate the timing and interpretation of findings with a process of planning and\or systematic improvement of court operations. Examples of well-integrated programs of research on the general public and court users include the state-level planning by the Supreme Court of Virginia, the quality assessment program by the Hennepin County, Minnesota ( Minneapolis ) District Court, and the community surveys carried out by the Red Hook Justice Center .

The Supreme Court of Virginia has a coordinated process for feeding survey and other information into an ongoing planning process.[4] The Judiciary’s Strategic Planning and Management System operates as follows:

Four key sources of research and information fuel the development of the court system’s plans. An extensive environmental scan provides insight and data on expected developments in the areas of demographics, technology, consumer trends, law and government to highlight what lies on the horizon. A telephone survey registers the public’s perceptions on how well the courts are performing and indicates where citizens perceive that improvements are needed. An extensive issue survey of judges, clerks of court, chief magistrates and bar organization representatives gathers information on their needs, concerns, and suggestions for improving the courts. The mission of the courts and the vision statements adopted by the Judicial Council articulate the underlying values on which the judicial system is founded. Each of these sources yields different, but often intersecting, data upon which to gauge the potential demands for court services in the years ahead.

Once this data is gathered and compiled, the search for both innovative and pragmatic solutions to the identified issues begins by analyzing the crosscutting themes suggested by the data. For each theme, a focus group or “venture team” is convened. Participants include a broad base of citizens, businesspersons, representatives of other government agencies, judges, clerks, magistrates, technologists and attorneys. Based on the individual and collective experience of its members, each team is asked to contribute ideas and solutions for how the courts can manage the repercussions that may flow from one of the themes.

At the trial court level, the Hennepin County District Court uses survey methods to answer the following questions on an on-going basis:

The survey work, which was designed in partnership with some of the national academic experts in measuring fairness, is part of an ambitious evaluation process, the Baldrige Assessment (Fourth Judicial District, 2002).

Annual community surveys were important components of the process of planning the Red Hook Justice Center ( Brooklyn , New York City) Community Survey (Paik, 2003). Face-to-face interviews were conducted with local residents for five consecutive years. The interviewers were other local residents serving in an AmeriCorps community service program.

[1] Many judicial performance evaluation programs use a variation on the exit survey to rate courtroom decorum, fairness, punctuality, etc. Such surveys can be undertaken to allow judges to assess their conduct in the courtroom, and thus kept confidential, or be used for decisions on retention in office, whether by court officials, the legislature, or the public at election time.

[2] In practice, the goal is that each person’s probability of being included in the sample is known. The likelihood of being included varies among different population groups.

[3] Surveys about the law and courts require care when questions and instructions are translated into Spanish and other languages. Legal and court terminology need to be carefully rendered to capture the appropriate meaning. Involving court interpreters in the process of drafting a survey is advisable.

[4] The Minnesota Judicial Branch acted to ensure that their 1999-2000 opinion survey would be “more than a snapshot of perceptions, it will be a detailed blueprint for long-term goals.”

History of Use/Replication

The 1977 “Public Perceptions of the Courts” survey, carried out for the National Center for State Courts, was the first opinion survey directed specifically at measuring the American public’s opinion on the state courts. Three surveys with a common set of core questions were carried out: the general public, judges, and opinion leaders. This effort remains the most ambitious attempt to measure public opinion about the courts.

Twenty-five states subsequently carried out statewide opinion surveys, often to meet the information needs of a court’s futures or fairness task force. Some surveys were joint efforts by the state bar and the judiciary. A number of states have repeated their surveys over time.

Several state surveys (MN, NM, VA, WA) have largely replicated questions first asked in the 1999 national survey, “How the Public Views the State Courts,” offering standard questions that can be compared across jurisdictions and over time.

Recent opinion surveys on the courts have been influenced by the social psychological study of what constitutes fairness. People see themselves as having been treated fairly when they experience respect, neutrality, an opportunity to express their viewpoint, and trustworthy decision-makes (Tyler, 2001; Warren, 2000).

Contacts

David B. Rottman, Principal Court Research Consultant, National Center for State Courts, 757-259-1856, drottman@ncsc.dni.us.

Pamela Casey, Principal Court Research Consultant, National Center for State Courts, 757-259-1508, pcasey@ncsc.dni.us.

Alan Tomkins, Director, Public Policy Center , University of Nebraska , 402-472-5688, atomkins2@unl.edu.

Further Reading

Diamond, Shari Seidman (2000) Reference Guide on Survey Research in Reference Manual of Scientific Evidence (2^nd Edition). Washington D.C. : Federal Judicial Center.

Minnesota State Court System, Minnesota Public Trust and Confidence Survey, 1999-2000, Minnesota Public Trust and Confidence Strategies http://www.courts.state.mn.us/?page=519

National Center for State Courts (1999). How the Public Views the State Courts: A 1999 National Survey. Williamsburg, VA. Funded by The Hearst Corporation and carried out by the National Center for State Courts.

Paik, Leslie (2003) Surveying Communities: A Resource for Criminal Justice Planners. Washington DC : Bureau of Justice Assistance.

Rathjen, G.J. (2001) The Role of Survey Research in the Administration of Justice. Justice System Journal 22 (No. 3).

Rottman, D. (2000) Public Perceptions of the State Courts: A Primer. The Court Manager Vol. 15 (No. 3).

Tyler, T. R., (2001). Public Trust and Confidence in Legal Authorities: What do Majority and Minority Group Members Want from the Law and Legal Institutions. Behavioral Sciences & the Law 19 Number 2, 215-236.

Warren, Roger (2000). Public Trust and Procedural Justice, Court Review 37, 3, 12-16.

Yankelovich et al., (1978). Highlights of a National Survey of the General Public, Judges, and Community Leaders, National Center for State Courts, State Courts: A Blueprint for the Future, 21.

Effective Practice

Issues

Selecting Methods for Measuring Public Opinion

Strengths and Weaknesses of Information Gathering Methods

Quality Control

Making Opinion Research Affordable

Integrating Survey Findings into Policy and Improvement of Practice

History of Use/Replication

Contacts

Further Reading