NCSC: Research: Trial Court Performance Standards

Trial Court Perforrmance Standards & Measurement System

Standard 3.3: Court Decisions and Actions

Trial courts give individual attention to cases, deciding them without undue disparity among like cases, and upon legally relevant factors.

Commentary. Standard 3.3 requires that litigants receive individual attention without variation due to judge assignment or legally irrelevant characteristics of the parties, such as race, religion, ethnicity, gender, sexual orientation, color, age, handicap, or political affiliation. Persons similarly situated (e.g., criminal defendants faced with or found guilty of similar offenses and having similar criminal histories) should receive similar treatment. The standard further requires that court decisions and actions be in proper proportion to the nature and magnitude of the case and to the characteristics of the parties. Variations should not be predictable due to legally irrelevant factors, nor should the outcome of a case depend on which judge within a court presides over a hearing or trial. The standard refers to all decisions, including sentences in criminal cases, the conditions of bail, the amount of child support ordered, the appointment of legal counsel, and court-supervised alternatives to formal litigation.

Measurement Overview. One of the most fundamental problems confronting a democratic society is discrimination on the basis of race, ethnicity, gender, religion, or any other factor. The undesirable nature of discriminatory conduct becomes truly odious when the source of the conduct is a governmental institution. Hence, not surprisingly, the performance of courts is scrutinized closely for the presence of discriminatory policies, procedures, and practices.

Virtually every State court system has tried to identify whether it is contributing to discrimination, where discrimination occurs, and what can and should be done to eliminate it. The formation of racial, ethnic, and gender bias commissions is a recent and prominent example of these concerns.¹⁶ The purpose of many of the commissions is to determine the extent of perceived bias in the courts among the citizenry, to evaluate the reality of that bias, and to recommend ways to remedy both the perceptions and any actual biases discovered during the inquiry. In doing this work, these commissions have drawn attention to the problem and have heightened the consciousness of State judicial leaders, prompting them to remove bias where it exists.

Discrimination and bias are antithetical to underlying legal and constitutional principles and thus are crucial to eliminate. Standard 3.3 reiterates these principles by asserting that the court is to treat every case with individual attention in a consistent manner on the basis of legally relevant factors.

Because the topic of bias is extremely sensitive, courts will want to measure their performance in this area very carefully. Courts will want to know that measures of fairness, equality, and integrity are valid and that conclusions concerning their performance are not open to misinterpretation. However, determining the scope, location, and magnitude of bias requires considerable court resources. With a desire for more precise conclusions about the court’s policies and practices pertaining to the race, gender, ethnicity, or age of courtroom participants, the evaluation process requires more time-consuming and costly methodological skills. As a result, courts should begin with the most simple approach to determining court performance with regard to bias and move on to more complex measures as the court desires or requires more precise or complex answers.

Courts with limited experience in the area of fairness, equality, and integrity may want to begin by compiling information, literature, and readily available data. A court may choose to limit its compilation to specific topics it considers most relevant or to materials that discuss issues in similarly sized and situated jurisdictions.¹⁷ Has the topic ever been investigated in this court? How broad in scope and how detailed were these studies? Did they cover the treatment of litigants, witnesses, and jurors in both civil and criminal cases?

For a court already familiar with the general topics of equality and fairness, the initial approach might be to focus on the opinions of experts, court users, and the community. Following the model of many bias commissions, the court may convene focus groups to reveal attitudes toward the court from various points of view. This opinion gathering should have an agenda that structures the discussion. For instance, invited participants might be limited to representatives of selected groups or the topic might be limited to a specific aspect of the legal process such as sentencing criminal defendants. Court organizers should emphasize that the discussion is about general opinions toward the court and should not focus on any particular person or case. Following the exercise, the court can then evaluate issues such as: Is the court commonly viewed positively or negatively? Or, is the general outlook one in which bias is thought to be an exception and limited to particular circumstances?

A knowledge of the literature and data about bias and an awareness of the opinions toward the court will be useful. If the general picture reveals areas of potential problems, the court can decide whether to pursue a more systematic inquiry into the possible sources of bias and discrimination. Even if the general picture is almost entirely favorable, the court may decide to confirm this view with more systematic information.

The gathering of more specific and detailed information demands more resources, time, and skills to complete. As explained in detail in the following pages, the implementation of quantitative measures requires more complex methodologies (e.g., inquiry into individual case files, data manipulation, or a systematic survey of a random group of individuals) than those required by the first two approaches.

Measures 3.3.1, Evaluations of Equality and Fairness by the Practicing Bar, and 3.3.2, Evaluations of Equality and Fairness by Court Users, focus on the views of practicing attorneys and court users toward the decisions and actions of courts through a survey of a random sample of these individuals.

Measure 3.3.3, Equality and Fairness in Sentencing, focuses on the extent to which legally relevant factors account for the court’s sentencing decisions in criminal cases. To document whether any perceived problems exist, a statistical approach is described. This approach, however, is likely to require technical assistance from the research community.

Measure 3.3.4, Equality and Fairness in Bail Decisions, focuses on the extent to which legally relevant factors account for the court’s bail decisions in criminal cases. Systematic information is gathered to answer this question through a review of closed case files.

Finally, Measure 3.3.5, The Integrity of Trial Court Outcomes, examines the integrity of court decisions and actions as indicated by the outcomes of civil and criminal appeals.

The measures described for Standard 3.3 are challenging. The reason for this complexity, however, is to ensure that any findings regarding the presence or absence of bias are valid. These measures may be beyond the scope of some courts’ available expertise and resources. Other courts may choose to implement only one or two of the suggested measures based on their own resources.

______________________________________
¹⁶ Approximately 20 States at this writing have undertaken efforts to establish a racial/ethnic bias commission or task force. Similarly, nearly every state has established a gender bias commission or task force.
¹⁷ Suggested sources for the literature published in this area can be accessed in the Index to Legal periodicals or automated databases. The Information Service at the National Center for State Courts also can provide information on articles or reports published, particularly in court-related publications. The court also may be able to access actual data on the topic from such bodies as a State sentencing commission or race and ethnic bias task force.

Go to Performance Area 3

Measure 3.3.1: Evaluations of Equality and Fairness by the Practicing Bar

The purpose of this measure is to ascertain the practicing bar’s perceptions of the equality and fairness of the court’s decisions and actions. Members of the bar who appear in court will be asked, through a survey questionnaire, to assess the fairness and equality of the court’s actions and decisions. A consensus among them that the court provides attention to litigants, produces similar outcomes among like cases, and relies upon legally relevant factors in making decisions will be another indication that the court complies with Standard 3.3.

Planning/Preparation. The first step is to construct a set of questions that measure the extent to which attorneys believe that the court is treating individuals fairly and equally. Questions can be drawn from both previous pools of judicial performance¹⁸ and basic research studies.¹⁹ These two bodies of literature have been consulted to design a form for use by the court. (See Form 3.3.1, Illustrative Questionnaire Concerning the Practicing Bar’s Views of the Court’s Equality and Fairness.)

The questionnaire that follows is divided into four sections. Section I seeks to establish the experience of attorneys with the courts. For example, attorneys who have had many cases heard before the court (question 1) may have different responses than attorneys who have had only a few cases heard.

Section II focuses on the views of attorneys regarding whether the court’s decisions are affected by characteristics of litigants or attorneys. Following Standard 3.3, the court should not be affected by legally irrelevant factors such as the gender or race of the attorneys or the litigants (questions 4, 7). Attorney views on court practices also can be gauged by asking them if the court shows favoritism (question 5) or antagonism (question 6) to any of the participants. Because there are many possible situations in which the court might demonstrate such undesirable practices, an open-ended question (question 8) is included to describe those situations.

The answers to the questions in Section II will most likely determine the answers to the questions in Section III, which asks attorneys for their overall judgments concerning fairness and equality (questions 9 and 10).

Finally, Section IV seeks to establish the profile of the attorneys. This information is helpful for comparing the responses between different categories of attorneys (e.g., male versus female).

Data Collection. This step involves asking members of the bar to complete the questionnaire. Because there are many attorneys who have no direct contact with the court, a portion of them will not return the questionnaire. Hence, a preferred method is to send questionnaires only to those attorneys who have appeared before the court at least once during the past year. Names of these persons may be obtained by a canvas of dockets during the period. This approach has the advantage of identifying in advance attorneys who are heavy, medium, and light users of court resources. A court may wish to target one set of users or to sample attorneys in proportion to usage ratios. For both methods, however, followup mailings of reminder postcards should be used to ensure a good response rate.

Data Analysis and Report Preparation. Most responses on the survey instrument are associated with a specific number code (e.g., "strongly agree" equals 1). For each survey form that is returned, attorney responses are recorded by entering these number codes into a computer file and then tabulated using a computer software program.

For the first analyses, each question should be examined to determine whether the attorneys consider the court to be a source of unfair or unequal decisions. For example, what percentage of the attorneys believe that the court sets higher bail for particular racial/ethnic groups (question 7)? That is, how many respondents circled options 1 and 2?

In general, the higher the percentage of attorneys that agree that the court acts without bias, the more the court meets Standard 3.3. That principle should guide the interpretation of individual questions. For example, if at least a majority of the respondents circle options 1 and 2 in question 10, it appears that the court, in general, is performing positively on this indicator.

Conclusions, however, should not be drawn without first analyzing the responses of various subgroups of respondents. These analyses are important for determining whether the opinions of some groups are underrepresented. For example, if most respondents are white males, the general analyses will reflect the opinions of this group. If white males do not see the favoritism or hostility experienced or perceived by other groups, the general analyses will not give the whole picture. It is important, then, to determine how the responses of other groups compare with general responses.

Finally, the responses to different questions can be examined in relationship to one another. Specifically, what issues explain the attorneys’ overall reactions (questions 9 and 10)? As an illustration, it may be the case that the more a respondent believes that the court does not sentence defendants of particular racial/ethnic groups more severely question 7d), the more likely he or she is to agree that the court is fair (question 10). In considering such relationships, questions 4 through 8 can be regarded as potential criteria for determining attorneys’ reactions regarding fairness and equality in the court.²⁰

It is important to note that this measure examines perceived bias among practicing attorneys. It does not consider the accuracy of those perceptions. It is up to the court to determine the level at which the perception of bias by practicing attorneys is sufficient to warrant further action.

_________________________________________
¹⁸ See, for example, D. Maddi, Judicial Performance Polls (Chicago: American Bar Foundation, 1977); and C. Philip, How Bar Associations Evaluate Sitting Judges (New York: Institute for Judicial Administration, 1976).
¹⁹ See, for example, T. Tyler, "What is Procedural Justice? Criteria Used by Citizens to Assess the Fairness of Legal Procedures," Law and Society Review 22 (1988):103.
²⁰ One method of determining the association between the survey items is correlational analysis. A statistical measure called the gamma coefficient can be used to test the extent to which the response to one question is associated with the response to another question. Statistical software packages routinely provide the statistic when cross tabulations of items are requested.

Go to Form 3.3.1

Go to top of page

Measure 3.3.2: Evaluations of Equality and Fairness by Court Users

All individuals (litigants, jurors, witnesses, and victims) who are involved in a court case form impressions of the way they and others are treated in the courthouse. Even members of the public who only observe the court proceedings form impressions. This measure is designed to collect information about their impressions of the court’s ability to provide fair and equal treatment.

Planning/Preparation. The first step is to construct a set of questions that measure the extent to which court users believe the court is treating individuals fairly. Many of the questions can be drawn from previous pools of judicial performance²¹ and basic research studies.²² These bodies of literature have been consulted to design two forms that can be used to gather information on the experience and perceptions of two groups of court users: (1) a courtroom group consisting of civil and criminal jurors, witnesses, and litigants involved in court proceedings; and (2) an administrative group consisting of persons coming to court to pay a fine, meet with a probation officer, or to check a court record. (Please refer to Form 3.3.2, Illustrative Questionnaire Concerning the Users’ View of the Court’s Equality and Fairness, for an example of the questionnaire.)

The questionnaires are divided into three sections. Section I asks each respondent to comment on his or her general views of court policies, procedures, and practices. Section II asks each respondent to comment on his or her experiences. Section III asks for information on the respondent and the nature of their contact with the court. This information will provide a profile of the respondents that may help to explain their answers.

Data Collection. Administration of the questionnaire is different for each group. The distribution strategy for each group is presented next.

Courtroom group: Lists of civil and criminal case jurors, witnesses, and litigants who have been involved in court proceedings during the past year are compiled. A questionnaire is mailed to each individual on the list.
Administrative group: Employees of each administrative office or section of the court distribute a questionnaire to each individual with whom they have contact. Employees ask each respondent to complete the questionnaire and return it in the envelope provided. Questionnaires should be distributed for a specific time period to ensure that a sizable number have been given out.

Data Analysis and Report Preparation. Most responses on the survey instrument are associated with a specific number code (e.g., "strongly agree" equals 1). Responses are recorded by entering these number codes into a computer file and then tabulated using a computer software program.

Analysis is conducted in two steps. First, each question should be examined to determine whether the respondent considers the court to be a source of unfair or unequal decisions. In general, the higher the percentage of court users that agree that the court acts without bias, the more the court meets Standard 3.3.

Conclusions should not be drawn, however, without first analyzing the responses of various subgroups of respondents. These analyses are important for determining whether the opinions of some groups are underrepresented. For example, if most of the respondents are white males, the general analyses will reflect the opinions of this group. If white males do not see the favoritism or hostility experienced or perceived by other groups, the general analyses will not give the whole picture. It is important, then, to determine how the responses of other groups compare with general responses.

The responses to different questions also can be examined in relationship to one another. Does the respondent’s personal experience correlate with his or her views of how social groups are treated? For example, do those individuals who feel they were treated on the basis of their race (options 1 and 2 in question 2a) tend to see the court favoring or showing hostility toward a particular racial/ethnic group?²³

It is important to note that this measure examines perceived bias and not the accuracy of the perceptions. It is up to the court to determine the level at which perceived bias among court users warrants further attention.

________________________________________________
²¹ See note 18.
²² See note 19.
²³ One technique for determining the association between the survey items is correlational analysis. A statistical measure called the gamma coefficient can be used to test the extent to which the responses to one question are associated with the responses to another question. The technique is available in most computer software packages.

Go to Form 3.3.2

Go to top of page

Measure 3.3.3: Equality and Fairness in Sentencing

One application of Standard 3.3 is sentencing in criminal cases. Because the imposition of criminal sanctions deprives individuals of their liberty, the fairness of the process and corresponding outcomes is an important topic for the measurement of court performance. In fact, some courts might regard fairness in sentencing to be among the most critically important goals that it should strive to meet. However, fairness in sentencing is understandably very difficult to measure.²⁴ Even the most refined measurement will produce results more suggestive than definitive, which is not astonishing given the difficulty of sentencing for trial judges. Just as the trial judge must weigh, balance, and take into account many factors, the court researcher must identify, measure, and interpret the effects of many complex factors, including some that are difficult to express as a precise scale of measurement.

Hence, trial courts take on a very daunting task by attempting to measure fairness in sentencing. Why? Because of the sensitive nature of conclusions about fairness, a court will want to know that the conclusions are valid to the greatest extent possible. However, sound conclusions require a rigorous methodology, which requires a substantial commitment of time, quantitative skills, and resources. Thus, without intending to deter courts from applying this measure, honesty requires acknowledging the labor-intensive aspect of the measurement process necessary to reach the kind of conclusions the court is likely to want to draw. (Note: The same point applies equally to the measurement of fairness in bail decisions, Measure 3.3.4.)

What does fairness in sentencing mean? According to Standard 3.3, "trial courts give individual attention to cases deciding them without undue disparity among like cases and only upon legally relevant factors." Translated into more operational terms, the standard is saying that the imposition of punishment should not be on the basis of a defendant’s race or gender. For example, African Americans should not receive longer sentences than non-African Americans simply because they are African American. Different sentences should be the product of differences in criminal backgrounds, offense severity, circumstances surrounding the offense, and other legally relevant factors.

Finally, while equality and fairness are positive standards, they are observed in the negative. Courts are urged to be equal and fair in their treatment, but their performance is measured in terms of outcomes that are not supposed to occur—inequality, disparity, and inconsistency.

Planning/Preparation. Courts should consider four steps in planning to undertake the measure. First, some familiarity with the literature on sentencing might prove useful. The most comprehensive volume, Research on Sentencing: The Search for Reform, is published by the National Academy of Sciences and available in most public and college libraries. The volume is written from the researcher’s perspective, however, and contains some articles of a technical nature. A complementary article, "Racial Discrimination" by Rose Matsui Ochi, which appeared in 1985 in The Judges’ Journal, illustrates how research results are interpreted and used by practitioners who seek to eliminate bias in sentencing. This article is also useful because it references additional readings on the topic that are readily available.

A second step is for the court to examine its capacity for conducting a rigorous measurement process. If the court lacks a staff person skilled in quantitative analysis, it might find it helpful to ask for guidance and assistance from a staff member of a State sentencing commission, State administrative office of the courts, or local university to assist in designing a plan of data collection, analysis, and interpretation.

A third step is to set some boundaries on the scope of the measurement process. Despite the fact that researchers construct very complex quantitative models of sentencing, the proposed measure is intended to help a court assess itself and not necessarily to advance the state of knowledge. Hence, it permits the court to limit the scope and detail of its inquiry without sacrificing the validity of the results. As an example, the court needs to decide what aspect of sentencing is of greatest importance. Is it more important to determine fairness in the types of sentences that defendants receive (e.g., incarceration versus probation) or in the length of sentences imposed (e.g., are men incarcerated for longer periods of time than women)? Are both aspects equally important?

Finally, before applying the measure, the court should discuss how it plans to interpret the results. The results will be in the form of numbers called coefficients that are based on the application of quantitative techniques to information gathered from individual case files. There will be a coefficient for each legally relevant (e.g., prior record and offense committed by the offender) and each extra-legally relevant factor (e.g., race of offender). The coefficient measures the impact of a particular factor, controlling for the effects of all other factors. If the legally irrelevant factors are not influencing outcomes, the numerical value of their respective coefficients will not be statistically different from zero. For example, knowing that an offender is a man will not predict the sentence any better than knowing that the offender is a woman. Additionally, the coefficients of all legally relevant factors should be significantly larger than those of irrelevant factors. If they are, one reasonably can draw the conclusion that there is limited bias in sentencing and that sentencing is primarily a product of legally relevant factors. If the court knows what to look for in advance, it will be more prepared to interpret and use the results both internally for self-improvement and for presentation to interested groups outside the court.

Defining the Data Elements. Although the exact delineation of legally relevant and legally irrelevant factors may vary somewhat across States because of differences in substantive and procedural law, some distinctions likely will be valid in almost all situations. For the purposes of demonstrating the utility of the measure, therefore, it is assumed that legally relevant factors include offense seriousness, quality of the evidence, prior criminal record, and current legal status. Irrelevant factors include demographic, socioeconomic, and social stability attributes, and case processing attributes.²⁵ Based on that assumption, a court meeting Standard 3.3 has sentencing outcomes that can be explained more on the basis of those legally relevant factors than on factors deemed irrelevant.

In addition to identifying a set of determinants of sentencing outcomes, the initial measurement step involves specifying the outcomes of sentencing. Two related outcomes are especially important:

In/Out Decision. Is the offender sentenced to a term of institutional incarceration? Or is the offender given some alternative such as probation, restitution, community service, or fine?
Length of Sentence. How long is the period of institutionalized incarceration?

The first outcome distinguishes between convicted offenders who are sentenced to prison or jail and those who are given a sentence outside these institutions. The second outcome focuses on the length of the sentence in years, months, or days imposed on individuals sentenced to jail or prison.

Legally relevant factors: Concerning the seriousness of the offense, a basic judgment must be made to focus on either a broad range of offenses or to isolate particular offense categories (e.g., robbery, burglary). The first option is to consider a large set of offenses and to rank them according to severity (e.g., homicide, robbery, rape, assault, weapons, drug sale, drug possession, burglary, forgery, and theft).²⁶ Additionally, other indicators may be used to gauge the more specific degrees of severity, such as the use of a dangerous weapon, the extent of injury to the victim, the amount of property taken, and whether the offender was a principal or accessory to the offense.

Although some version of the first approach is highly recommended, a second option is to focus on selected offenses separately. If particular offenses are deemed of such importance to the court and the community that they merit special attention, this approach may be appropriate. However, this option lacks the representativeness of the first option, which encompasses the full range of offenses. Hence, we generally recommend some version of the first option.

The quality of the evidence is extremely difficult to measure and may be known fully only by the participants involved in each individual case. As a result, retrospective reliance on case records for information only approximates the complete and correct picture of the strength of the evidence. Possible indicators include the number of prosecution witnesses, the number of expert witnesses, the number of exhibits, the submission of laboratory tests, and so forth. A limitation to these indicators, of course, is that they relate primarily to the few cases that go to trial.

Prior criminal history is usually information presented to the court from State law enforcement records. Although some law enforcement information systems are more detailed than others, criminal history generally is measured in terms of the number of prior adult felony convictions, the elapsed time since the last conviction, whether the last conviction was for the same offense as the current charge, and the current legal status of the individual at the time of arrest (e.g., on parole or probation).

Legally irrelevant factors: Demographic, socioeconomic, and social stability factors are a combination of quantitative indicators such as age (years), income (earned income per month), education (number of years) and categories such as gender (male versus female), race (white versus nonwhite), employment status (employed versus unemployed), and marital status (married versus nonmarried).

The case processing characteristics are all categories. Pretrial release status may be divided between those offenders on bail, those detained at least part of the time between arrest and final disposition, and those detained all of the time. Disposition similarly can be separated among those offenders who pled guilty, those convicted by a bench trial, and those convicted by a jury trial.

A final factor is the judge presiding over a sentencing decision. Each judge need only be identified by an alphabetic character (e.g., Judge A, Judge B, Judge C, and so forth). The measure is intended to determine if any judge has an influence on sentencing that is greater than generally accepted legal factors. Sentencing outcomes involve the distinction between institutional incarceration and some alternative to incarceration. This distinction captures the in/out decision. For the length of the sentence, a standardized measure is the percentage of the statutory maximum imposed in the actual sentence. Because some sentences may involve a range, the minimum of the sentence imposed should be used in calculating the percentage. This standardization permits different offenses to be compared despite their differences in severity.

Data Collection. In most jurisdictions, virtually all of the factors and sentencing outcomes can be measured against information contained in presentence investigation reports and closed court case records. A court can use these sources by drawing a random sample of approximately 1,000 closed cases and selecting from that pool those cases in which a conviction was obtained by guilty plea or trial. (The remaining cases should not be discarded because they can be used as part of the data set for Measure 3.3.4, Equality and Fairness in Bail Decisions.) Of this pool, 70 percent are likely to involve some sort of conviction, which means that these 700 cases can be used to examine the factors associated with the in/out sentencing decision. Of these cases, approximately half will involve a sentence of institutional incarceration, providing the basis for assessing the factors associated with the length of the sentence.

The measurement of sentencing and sentencing outcomes described above needs to be translated into a more specific and detailed form prior to the review of court case records and presentence investigation reports. A data collection form should be constructed for the purpose of applying the sorts of indices suggested for the different factors. (Please see Form 3.3.3, Illustrative Sentencing Data Collection Form.)

Data Analysis and Report Preparation. The question of whether legally relevant factors are more powerful predictors of sentencing outcomes than are irrelevant factors is addressed by the use of statistical models. These models are available in many software computer programs that are likely to be familiar to sentencing commission staff, court researchers in a State administrative office, or university professors. One or more of these individuals will likely know how to use an appropriate software program to analyze the data collected on the data collection form. Specifically, the expert will know what particular quantitative techniques should be applied to determine the independent impact of each legally relevant and irrelevant factor on the two types of sentencing decisions.

In the case of the in/out decision, an appropriate technique is logit analysis. Logit analysis is designed to indicate the independent effects of various factors on different categories (e.g., a sentence of institutional incarceration versus one of nonincarceration). The numbers generated by logit analysis include coefficients for each factor. The sign (±) of the coefficient indicates whether there is a positive (e.g., the more serious the offense, the more likely the sentence will involve incarceration) or inverse (e.g., the longer the length of time since the last conviction, the less likely the sentence will involve incarceration) relationship between each factor and the outcome. A comparison of the magnitude of the coefficients will indicate the relative importance of each factor in determining whether an offender is sentenced to prison as opposed to some alternative sentence.

The issue of the length of sentences for incarceration is examined appropriately through the use of regression analysis. Regression analysis is designed to indicate the independent effects of factors on an interval factor such as the number of months to be served. Similar to the logit analysis, coefficients are generated by regression analysis. They indicate if there is a positive (e.g., the older the offender, the longer the sentence) or inverse (e.g., the higher the offender’s level of education, the shorter the sentence) relationship between each factor and the length of the sentence.²⁷

The coefficients bear upon the central purpose of the measure in two ways. First, if the legally irrelevant factors are not influencing outcomes, the coefficients associated with them should not be statistically different from zero.²⁸ Second, the coefficients of all legally relevant factors should be significantly larger than those of irrelevant factors.

Looking at the coefficients associated with the different factors, the court can begin to assess their implications for fairness in sentencing. Do the results signal that legally irrelevant factors are having undue influence on the likelihood of incarceration or the length of sentences? Or do the results signal that irrelevant factors fail to account for the court’s decisions to sentence offenders to prison or the length of prison sentences? In sum, do the results indicate that sentencing decisions are the product primarily of legally relevant factors and that irrelevant factors are of limited significance?

Depending on what the results indicate, the court can use the information as a guide to reviewing its sentencing policies, practices, and procedures. The results might suggest the need for special training programs for newly appointed judges, especially those who come from private civil practice backgrounds. Or, the results might suggest the need for a courtwide training program on current developments in substantive and procedural criminal law.

________________________________________
²⁴ The measure proposed outlines a statistical approach to assessing whether there is undue disparity and bias in a court's proceedings. However, it is not a complete treatment of every aspect of particular techniques and their interpretation. For this reason, the court may wish to consult outside experts when applying the measure.
²⁵ The definition of the data elements and the proposed methods of data analysis reflect the input and advice of academic sentencing experts and former staff of the U.S. Sentencing Commission. Their opinions were solicited to achieve maximum statistical validity, although future research is likely to use even more refined methods in this growing area of research.
²⁶ An offense severity scale can be developed by assigning numerical weights to different offenses. The U.S. Sentencing Commission has constructed such a scale.
²⁷ For discussion of parallel applications of this technique to case processing data, see R. Flemming, P. Nardulli, and J. Eisenstein, "The Timing of Justice in Felony Trial Courts," Law & Policy 9 (1987); and M. Luskin and R. Luskin, "Why So Fast, Why So Slow: Explaining Case Processing Time," Journal of Criminal Law & Criminology 77 (1989).
²⁸ A coefficient may be greater but not statistically greater than zero because the factor under consideration (e.g., race) does not have consistent, uniform effects on what is being measured (e.g., sentence length). However, a statistical test performed by the software will indicate whether each coefficient is significantly greater than zero.

Go to Form 3.3.3

Go to top of page

Measure 3.3.4: Equality and Fairness in Bail Decisions

The purpose of this measure is to provide information to the court concerning the nature of the factors associated with bail, bond, and release on recognizance decisions.²⁹ In making these decisions, a court should focus on factors permitted by law. One way to measure the court’s reliance on appropriate factors is to determine whether differences in bail decisions are linked more to factors recognized in law or to extra-legal factors such as the defendant’s race or gender, the judge assigned to the case, or the geographic location of the court. According to Standard 3.3, the greater the degree to which the differences in the bail status of defendants are consistent with factors permitted by law, the better the court is performing on this measure. The remainder of this discussion outlines a step-by-step procedure that courts can use to measure and assess factors associated with bail decisions.³⁰

Planning/Preparation. The initial step is to identify the factors permitted by law to shape the court’s bail decisions. Because States have different bail guidelines, the list of factors will differ somewhat across jurisdictions. However, most courts use a core set of factors in deciding whether to release the defendant on recognizance and in setting the dollar amount of the required surety bond if the defendant is not released.

Legally relevant factors are as follows:

Defendant’s Criminal and Court History

(1) Prior record—Does the defendant have prior felony convictions? If so, how many and for what offenses? The notion is that it is rational for the court to set stiffer bond requirements for a more extensive prior record, especially if the defendant has recent convictions for the same offense. Some States may incorporate this rationale explicitly into bail guidelines by limiting the release of "dangerous offenders." Finally, did the defendant intimidate witnesses while on release for prior offenses? Such behavior also is grounds for imposing a more restrictive bond.

(2) Prior court appearances—Has the defendant missed prior court appearances? How many times? Did the defendant leave the area on those occasions? Because a rationale of bail is to ensure court appearance, previous failures-to-appear also are reasonable grounds for imposing a more restrictive bond.

(3) Current legal status—Is the defendant on parole or probation? Are there outstanding warrants? Parole or probation violations are considered sound reasons for imposing stricter bond conditions. Similarly, an outstanding warrant justifies stricter bond conditions.

Current Offense

Is the defendant charged with a violent offense? Was there alleged bodily harm caused to a victim? What is the length of the sentence on conviction of the charged offense? It is often deemed appropriate to place more constraints on individuals who are believed either to pose serious threats to the community or face the possibility of severe sanctions.

Community Ties

Is the defendant a resident of the jurisdiction? For how long? With whom does the defendant live? Do family members live in the area? Is the defendant employed? What is the defendant’s monthly income? Individuals with close ties to the community are considered likely to appear in court when required and are, therefore, regarded as appropriate candidates for release on recognizance or low surety bonds.

Defendant’s Character

Is the defendant currently using drugs? Could the defendant’s mental or physical condition be impaired by detention? Defendants free of drugs or likely to suffer under detention should receive less restrictive bonds.

To determine whether legally irrelevant factors affect bail decisions, data also must be collected on these factors. Legally irrelevant factors include:

Demographic Characteristics

These characteristics include race and gender.

Legal Counsel

Was counsel available to the defendant? If so, when? What type of attorney represented the defendant?

Judge Assigned to the Bail Hearing

Each judge need only be identified by an alphabetical character (e.g., Judge A, Judge B, and Judge C, and so forth). The measure is included to see if the composite effect of judge identity is greater than the effect of legally relevant factors.

In addition to identifying possible determinants of bail decisions, the decisions themselves need to be outlined. Three of the most fundamental issues are as follows:

Is the defendant released on recognizance? Because of limited incomes, many defendants cannot post even modest surety bonds. For these defendants, release on recognizance may be the only avenue to pretrial release. It is important therefore to see the relative frequency with which the court decides to use this option.³¹
If the decision is made not to release the defendant on recognizance, what is the amount of the surety bond?

If a different bail decision is made after the first appearance, should the initial or subsequent decisions be counted? That is, if a surety bond is set but the defendant is later released on recognizance, should the defendant be considered to be released on recognizance? If the amount of the surety bond is lowered or raised at a later proceeding, which figure should be recorded? One approach to this question is to record bail status at the initial appearance separately from the decision in place at 15 or 30 days after the first appearance. This strategy captures more of the legal process without elevating one decision over another.

Data Collection. The information necessary for this measure is available in closed court case records and the records of local bail agencies, pretrial release organizations, or probation departments. The process of selecting cases for analysis involves drawing a sample of 1,000 closed court cases and tracing those cases back to the other organization’s files. (The sample of cases can be drawn from the pool of cases used for Measure 3.3.3, Equality and Fairness in Sentencing).

Measurement of bail decision determinants consists of a combination of quantitative scales and classification schemes. An illustrative data collection form is offered as a way of measuring the factors that determine bail decisions and the decisions themselves. (See Form 3.3.4, Illustrative Bail Decision Data Collection Form.)

Generally, the factors included on the data collection form are the same as those used by researchers in the field.³² However, in some jurisdictions, the court may never receive information on specific aspects of the defendant’s community ties, character, or socioeconomic status. Instead, these factors may be taken into account by a bail agency that recommends bail decisions to the court. If this is the situation, the bail agency’s recommendation should be considered a surrogate for those factors.³³

Data Analysis and Report Preparation. Quantitative techniques can be applied to the data and each data element assessed for its effect on bail decisions after taking into account the influence of all other factors. The results of the analyses will tell the court whether and to what extent each legally irrelevant factor influences bail decisions. Results also will tell the court whether and to what extent legally relevant factors are more influential in decisionmaking than legally irrelevant ones.

The two types of bail decisions require different types of analyses: logit analysis and regression analysis. They are discussed below.

In the case of the decision to release or not to release on recognizance, an appropriate technique is logit analysis. The basic results of logit analysis are numbers, called coefficients. A coefficient is associated with each factor. The sign (+) of the coefficient indicates whether there is a positive relationship (e.g., the longer the defendant has lived in the community, the more likely his or her release on recognizance) or inverse relationship (e.g., the greater the number of past failures to appear, the less likely his or her release on recognizance) between each factor and the bail decision. A comparison of the magnitude of the coefficients will indicate the relative importance of each factor in predicting the likelihood of a defendant being released on recognizance.

The issue of what factors predict the amount of surety bonds is examined by another technique called regression analysis. Regression analysis is designed to indicate the independent effects of variables on an interval measure such as the dollar amount of bonds. Similar to logit analysis, in regression analysis coefficients are generated by the technique. The sign (+) of a coefficient indicates if there is a positive (e.g., the greater the number of prior felony convictions, the larger the amount of the bond) or inverse (e.g., if the offender has family members in the community, the lower the bail amount) relationship between each factor and the bail amount.

The coefficients bear upon the central purpose of the measure in two ways. First, if legally irrelevant factors are not influencing decisions, the coefficients associated with them should not be significantly greater than zero.³⁴ Secondly, the coefficients of all legally relevant factors should be significantly larger than those of irrelevant factors. An inspection of the coefficients should address these issues.

After the data have been gathered and analyzed, a key task is to present the results to the court. Do the results make sense? For example, should the court be concerned if the results indicate that having family members in the community decreases a defendant’s chances of personal recognizance? What does it mean if the presence of family members decreases the average surety bond by a certain amount?

In addition to reviewing the intuitive soundness of the results, the court should assess their implications for court performance. Do the results signal that irrelevant factors are not having undue influence? Or, do the results confirm that irrelevant factors have emerged as unacceptably powerful predictors of the court’s release decisions?

Finally, the court must decide what to do with the results. Regardless of the level of performance, what should be done? What sort of action is appropriate to improve performance? For example, is a courtwide review of bail policies, procedures and practices needed? Would special training programs for newly appointed judges and programs on current developments in substantive and procedural criminal law help? Although the court must make its own judgments as to what is necessary and desirable, the empirical evidence should inform the making of that judgment.

____________________________________
²⁹ In most jurisdictions the majority of bonds are released on recognizance and surety bonds. However, in some courts, case bonds also are prominent. In this event, the court should consider what factors account for the amounts of different cash bonds.
³⁰ The measure proposed outlines a statistical approach to assessing whether there is undue disparity and bias. However, it is not a complete treatment of every aspect of particular techniques and their improvement. For this reason, the court may wish to consult outside experts when applying the measure.
³¹ The majority of defendants released on recognizance typically have nonfinancial conditions placed on them, such as third-party custody, prohibitions against returning to the scene of the crime, and restrictions on residence, travel, associations, drug and alcohol use, and weapons possession. These conditions are not crucial to determining whether legally relevant or irrelevant factors explain who is released and who is not. Hence, the court should collect data on these matters only if it seeks to pursue other research questions concerning bail decisions.
³² See, for example, J. Goldkamp and M. Gottfredson, Guidelines for the Pretrial Release Decision: Superior Court of Arizona, Maricopa County; Circuit and County Courts, Dad County; Boston Municipal Court; and Suffolk County Superior Court, Bail Guidelines Project (Philadelphia: Temple University, 1985).
³³ See, for example, I. Nagel, "The Legal/Extra-Legal Controversy: Judicial Decisions in Pretrial Release," Law and Society Review 17 (1983):481.
³⁴ A coefficient may be greater but not statistically greater than zero because the factor under consideration (e.g., race) does not have consistent, uniform effects on what is being measured (e.g., to release or not to release on recognizance). However, a statistical test performed by the software will indicate whether each coefficient is significantly greater than zero.

Go to Form 3.3.4

Go to top of page

Measure 3.3.5: The Integrity of Trial Court Outcomes

Measures 3.3.3 and 3.3.4 address adherence to laws or procedures, which can be ascertained explicitly and objectively. A complementary approach, which looks at adherence more broadly, involves the examination of appeals taken from trial court judgments. The analysis of the outcomes of appeals in terms of affirmance and reversal patterns will uncover where problems may exist and point to areas where trial court performance can and should be improved. Such an examination will shed light on where problems (i.e., reversible errors) occur. Do problems more frequently arise in particular areas of civil law such as property and commercial litigation and not in other areas such as torts? Are problems more common in appeals taken from certain trial court proceedings such as pretrial motions and not from nonjury trials? Or, are problems associated with particular issues? For example, in criminal appeals, how often are suppression issues successful on appeal?

Information on the nature and rate of reversals will enable individual trial courts to identify where problem areas exist.³⁵ It will also be useful in identifying problem areas for all trial courts within a State and in examining performance over time.³⁶ A step-by-step procedure for examining the decisions of first-level appeals courts is described next.³⁷

Planning/Preparation. An examination of appeal outcomes should include all subject areas (e.g., civil and criminal). However, because of the constitutional nature of the issues involved, if a jurisdiction does not have the resources to conduct an examination of all outcomes, first attention should focus on criminal cases. The number of cases to be examined will depend on the scope of the inquiry. In an examination of only civil or criminal appeals, for example, 250 to 300 appeals resolved on the merits will be sufficient in each category to see broad patterns.

The information to be collected from each appeal will depend on the subject matter of the case, which may include:

An area of law or criminal offense (e.g., for civil appeals: tort, commercial/contract, domestic, property; for criminal appeals: homicide, other crimes of violence, property crimes).

A trial court proceeding (e.g., jury trial, nonjury trial, pretrial motion, agency review).
The nature of each issue raised on appeal and its outcome.
The outcome of the appeal.

Additional information may be included as measures of case complexity (e.g., severity of the sentence, number of parties, or type of counsel). An example for collecting the data is presented on Form 3.3.5, Illustrative Outcomes Data Collection Form for Criminal Appeals.

Data Collection. The information needed to conduct this measure is available in the case records of the appeals court, although different sources may have to be checked. The docket should be the first source consulted to identify the appeals (e.g., trial court, subject matter, and resolved on merits). The docket also may be a source of other information (e.g., type of counsel). The court’s decision document/opinion is a key source of information on the issues raised and their treatment. The notice of appeal or docketing statement is a useful source for background information (e.g., sentence in a criminal appeal) that may not be provided in the decision document. Finally, it may be necessary to check the briefs if the court does not file a written decision or if the decision does not identify the issues the court considered.

Data Analysis and Report Preparation. A variety of basic analyses can shed light on the pattern of appeal outcomes and the frequency and distribution of error. For example:

The relative frequency of appeals by subject matter, by underlying trial court proceeding, and by other measures of case complexity.
The relative frequency of outcomes by subject matter, by underlying trial court proceeding, and by other measures of case complexity.
The relative frequency of issues raised by issue disposition.

These tabulations help jurisdictions determine whether and the extent to which cases involving certain areas of law, raising particular issues, and being resolved in particular trial court proceedings are more likely to pose problems for trial judges than are other appeals.

The analysis can be expanded to include other questions of interest. In addition to the quantitative analyses, a qualitative examination of the circumstances surrounding the errors can be undertaken. From a qualitative perspective, it is important to know whether an error occurred because of one of three basic circumstances: (1) the error arose in a new area of law or litigation, (2) the error resulted from the misinterpretation or misapplication of applicable law, or (3) the error was caused by a failure to follow established or appropriate procedures.

The results should be reviewed to identify areas of difficulty for trial courts that need to be improved. For example, if the relative frequency of error is strongly related to the area of law/offense, the trial court proceeding, or other measures, the court should focus its corrective measures (e.g., educational programs) on such areas. A disproportionate "error rate" for particular issues would also indicate the need for educational attention.

This analysis can be used over time both to identify areas in need of corrective measures and to indirectly measure the effectiveness of such programs. In addition, the use of a common data collection system and a common set of data elements across jurisdictions can highlight the existence of alternatives. For example, a jurisdiction that has a high incidence of error on jury instructions can and should learn how and why other jurisdictions have fewer instruction errors.

Finally, the collection and analysis of information on the outcomes of appeals should provide trial courts with a concrete starting point for establishing the acceptable and unacceptable frequency of reversible error. Because most estimates of reversal patterns are based on impressions and personal observations, the data can help courts construct meaningful standards that combine both the frequency of errors and the circumstances under which the errors occur.

_________________________________________
³⁵ Another important question is: Are there differences in the rates of reversals across individual trial courts within the same State? To address this issue, there must be a sufficient number of appeals from each court. Because few courts generate more than 50 appeals each year, the data requirements are difficult to satisfy. Hence, as a first effort, this measure is most profitably aimed at statewide patters or patterns with a regional appellate district.
³⁶ For an investigation of reversible error in criminal appeals, see J. Chapper and R. Hanson, Three Papers on Understanding Reversible Error in Criminal Appeals (Williamsburg VA: National Center for State Courts, 1979). The authors present evidence from a study of five appellate courts and discuss the implications of the results for judicial education.
³⁷ Defining trial court error by the decisions of first-level appeals is not conclusive, of course. Trial court decisions overturned on first-level review may be reinstated by a higher court. Such subsequent review is uncommon, however. In 1987, for example, State courts of last resort granted review in only 14.1 percent of the discretionary petitions filed. As a result, first-level appeals courts are, in fact if not in law, the final arbiter for most appeals.

Go to Form 3.3.5

Go to top of page

Go to Table of Contents