Measure 3.1.2 complements the panels’ assessments. It requires surveying court employees and practicing attorneys to assess their views on the extent to which legal requirements are met. For both Measures 3.1.1 and 3.1.2, the greater the extent to which requirements are met, the higher the court’s performance in this standard area.

Integrity is essential to court performance. To maintain their position as independent and fair arbiters of disputes, courts must be faithful to the laws they are expected to apply. The court’s integrity in upholding the law can be measured by the extent to which its actions are in accordance with the requirements specified in substantive and procedural laws. If the requirements are met, the court is performing well. Whether the court adheres to legal requirements often can be determined empirically. For example, if a court by statute must advise convicted offenders orally of their appeal rights, empirical data can be gathered by observing several adjudication or sentencing hearings to determine if the offenders actually are advised of their appeal rights. Similarly, if a statute requires that all decrees of divorce include a finding on the subject of medical insurance for children, the presence or absence of the finding is ascertainable by examining the order.

A recommended approach to identifying areas of law to be examined is to organize panels on basic areas of law such as civil, criminal, juvenile, and domestic relations. The local trial court is in a position to suggest the names of relevant practitioners from the bar and other justice system agencies. The administrative office of the courts (AOC) in each State may be particularly helpful in facilitating this task. The AOC can assist in organizing the panels and providing guidance for the panels’ deliberations. A panel approach is suggested because the measure requires detailed knowledge in several areas of the law (e.g., criminal, juvenile, domestic, civil torts, and contracts).¹

Planning/Preparation. When more than one jurisdiction in a State participates in an evaluation using the measurement system, a sponsoring agency (e.g., a State AOC) designates a coordinator for the measurement effort in this area. The coordinator should be capable of taking charge of a panel of professionals and leading them through a measurement process. If a single trial court in a State is using the measurement system, an individual at the local level coordinates the effort.

The coordinator assembles panels of individuals knowledgeable about the State’s laws and practices relating to particular types of cases. These case types will include some or all of the following: general civil, juvenile offender, juvenile dependency (neglected/abused), domestic relations and mental health, and criminal. As an example, the panel of criminal experts might include defense and prosecution attorneys, a probation official, a corrections department official, a staff member employed either by a judiciary committee of the State legislature or the judicial council, and a trial judge.

Each panel convenes to identify 5 to 10 requirements of law. The coordinator should use specific examples to help focus the discussion and the process of selecting the laws. Some States and jurisdictions have special requirements that should be considered. In Ohio, for example, the coordinator might point out that State law prohibits the use of a probation sentencing alternative for juveniles with certain offense histories. Also in Ohio, State court rules call for specific oral advisements by the judge regarding appeal rights for criminal defendants on the occasion of sentencing. In other States, orders on matters of child custody must address health insurance coverage.

The following list identifies areas of law that are applicable to virtually every State. Panels should use these categories as a guide in developing their own lists of laws. Other areas of law can be added or substituted, depending on local circumstances.

• Reviewing and deciding motions—extent to which required documentation (e.g., briefs) is met for particular motions (e.g., summary judgment), whether the deadlines for filing motions and responses to motions are met, whether motions are ruled on in a timely manner.

• Imposition of sanctions—extent to which sanctions are imposed when attorneys request them and the court’s rule clearly states that they shall be imposed (e.g., costs are to be imposed on the losing party in a discovery-related motion).

• Enforcement of continuance policies—extent to which the court adheres to its own policy of granting extensions of time (e.g., no continuance is granted simply because all parties agree to it or no more than two continuances are granted, except in exceptional cases).

• Required documents—documents, for example, summarizing income in divorce matters.

• Enforcement—specific requirements for court activity (e.g., issuing wage withholding orders and reviewing guardianships and conservatorships).

• Jury instructions—whether, when, and how instructions to juries are given.

• Awards of costs and attorney fees—statutes requiring awards to parties.

• Juvenile detention—statutes, case law, rules governing administration of detention services.

• Setting bail—extent to which guidelines are followed.

• Process for appointment of counsel—eligibility, timeliness of appointment, participation at critical stages.

• Required proceedings—whether, when, and how proceedings are conducted (e.g., advisement of rights and alternative dispute resolution hearings).

• Content of orders—mandated elements such as length and terms of incarceration.

• Appeal process—extent of notification of the opportunity to appeal.

An overlap between the laws the panels will select for this measure and the laws of interest in other standard areas likely exists in areas such as expedition and timeliness, and equality, fairness, and integrity. Hence, each panel should consider data collected for other measures and utilize them as appropriate to simplify the data collection effort.

Data Collection. The laws selected must be measurable using one of the following data collection methods:

• Records search—data collected about (1) presence or absence of documents in case files, (2) form of documents in case files, (3) content of case file documents and summary records, and (4) filing date of documents.

• Observation of proceedings—data about whether and how required proceedings are conducted.

• Interviews with judges, court employees, and the local bar—see Measure 3.1.2 for an example of this method.

The data collection methods require the use of relatively straightforward measurement instruments. Forms that employ a series of questions can be designed for use in reviewing files, documents, and other court materials. Examples of such questions are:

• Does the case file, document, or other form contain a record of whether there was adherence to the required law or procedure?

• If not, is there evidence available elsewhere?

• If so, what does the record indicate? Was there adherence?

These questions should be applied to a sample of each type of case. It is not necessary to draw a separate sample of cases for each area of law under consideration. If areas of law address similar types of cases (e.g., civil or criminal), data may be gathered on each area from the same sample of cases.

Data Analysis and Report Preparation. Each panel will review the results of the examination of the sampled cases and proceedings to determine (1) the percentage of cases in which there was a clear indication of whether the law’s requirements were followed, (2) the percentage of cases in which there was no evidence of whether the requirements were or were not followed, (3) whether the degree of fidelity was uniform across each panel’s set of laws, and (4) whether some areas of law exhibit higher fidelity than others.

After addressing these questions, each panel will rate the court’s performance. Their assessments will then be forwarded to the court, which will decide what appropriate action should be taken to improve performance.

Integrity is a matter of perception as well as objective adherence to substantive and procedural laws. Courts should be viewed as faithfully applying the requirements of substantive and procedural laws. Practicing attorneys and employees of the court are in a position to provide useful information about the integrity of the court’s procedures. This information will be collected in a survey administered to members of the bar and court employees. The attorneys and court employees will be asked whether they are aware of specific requirements of law not observed in the court. If they are, they will be asked to cite underlying statutes or procedures as closely as they can.

Planning/Preparation. Different techniques will be used to select each set of respondents. For the purpose of this measure, "court employee" designates staff members of the clerk of court as well as persons employed by the judges. A court employee payroll list is requested from the court administrator and court clerk. The list should be annotated, using the most expedient means possible, with each employee’s position and duties. In some cases, the court’s personnel records may be organized in such a way that no extra effort will be required for annotation. If this is not the case, a system of codes will be provided to the court to make annotation as simple as possible. One group of code values will indicate the kinds of cases with which the employee is familiar; another will indicate the kinds of duties the employee performs. For example, classification of duties will include activities such as courtroom services, document processing at the public counter, recording documents in docket records, managing records, working with judgments and judgment dockets, scheduling matters for calendars of court sessions, supervising probation, preparing pretrial release evaluations and reports, conducting presentence investigations, and handling cashier duties, cash bookkeeping or accounting, jury management, and so forth.

The court employee list will be used to identify employee respondents for the survey. Part-time employees and maintenance staff should be excluded from the sample group. Of those remaining, all employees should receive the survey. (If some groups of employees performing similar duties are very large, a sample of these groups could be drawn rather than sending the questionnaire to all of the employees in each group.)

For each division of the court included in the evaluation (i.e., general civil, criminal, juvenile, domestic, and so forth), docket entries for cases filed during the previous year is sampled, and a list of attorney names for all of the sampled cases is compiled. Attorneys who practice often in the court are likely to appear on the list more than once, and this frequency should be noted. As a result, if the court chooses, it can sample according to the attorneys’ degrees of practice.

Data Collection. For examples of possible questions to be used in data collection, refer to Form 3.1.2, Illustrative Questions for Measuring Court Employees’ and Attorneys’ Assessments of Fidelity to the Law. The questionnaires for court employees will be distributed via interoffice mail. The questionnaires for attorneys should be mailed with a cover letter signed by the president of the local bar association and the presiding judge of the local court.

Questionnaires will be returned in sealed envelopes coded to match the "master" list of individuals to whom questionnaires were given. Because the respondents are employees of the court and attorneys who regularly practice before the court, confidentiality is important, especially in smaller jurisdictions where it may be relatively easy to determine the identity of respondents to questionnaires. Care must be taken to avoid identifying respondents by their handwriting or how they return the questionnaires.

Individuals who return a questionnaire should be noted on the master list. When 10 days have elapsed from the date respondents received the questionnaire, a reminder should be sent to those individuals who have not yet returned a questionnaire. A second notice should be sent after 10 days have elapsed. At any time after 30 days, the data may be tabulated.

Data Analysis and Report Preparation. The laws listed by attorneys and employees should be compiled, noting the frequency of each citation. The percentage of respondents who list one, two, or three laws also should be calculated. For the purpose of measuring court performance, the larger the percentage of respondents who believe that legal requirements are not being followed, the lower the level of performance on this measure. The results of this measure should be submitted, if possible, to the panels described in Measure 3.1.1. The panels should comment on the results and assist in drafting the evaluation report to the court.