Go to Performance Area 1

The Federal Child Abuse Prevention and Treatment Act of 1974 requires all States to appoint an individual to represent the interests of children involved in judicial proceedings regarding child abuse and neglect. The individual appointed for this purpose is usually called a guardian ad litem (GAL). The States employ various models for providing the services of a GAL. In some States the GAL must be an attorney, while in others a trained volunteer (most often a court-appointed special advocate) may serve as the GAL or may work in conjunction with an attorney. The model used by individual jurisdictions within States also may vary from one another. In addition, the roles and responsibilities of GALs vary across the States, and many State statutes offer little guidance on the GAL’s specific duties. In most States, however, the GAL is expected at a minimum to act as an independent investigator of the facts related to the abuse or neglect, an advocate of the child’s interests, and a case monitor.¹⁴ Proponents of the rights of children and guidelines on GAL representation recommend that the GAL perform other duties as well.¹⁵

This measure determines the effectiveness of legal representation of the child in child abuse and neglect proceedings. An evaluator (or court staff) reviews the state statutes and court rules relevant to the appointment and responsibilities of GALs in child abuse and neglect proceedings, compares the statutes or rules to recommended practices for GALs, and obtains data from court records and surveys or interviews with GALs, judges, and child protective services caseworkers.

Planning/Preparation. Planning and preparation for conducting this measure includes four steps. First, court staff review the relevant statutes, court rules and policies, and case law on the appointment of guardians ad litem and their roles and responsibilities. Second, court staff modify the sample case data collection form (see Form 1.3.1a, Evaluation of Legal Representation of Child Abuse and Neglect Proceedings: Case Data Collection Form) and the survey forms (see Forms 1.3.1b, Judge Survey, 1.3.1c, Guardian ad litem Survey, and 1.3.1d, Caseworker Survey) to conform to the court’s procedures, practice, and terminology. Forms 1.3.1a through 1.3.1d include items related to practices recommended in the literature on GAL representation and in guidelines developed in a few States.¹⁶ Unless these items bear no relationship to local practice or are contrary to State law or court rule, they should not be eliminated because they are an important gauge of the effectiveness of legal representation.

The third step is the selection of the case sample, which should include 20 current child abuse and neglect cases that have reached a disposition hearing and 20 current review cases that have had a review hearing. The measure requires current cases to ensure that the judges, GALs, and caseworkers have fresh memories of their experiences in the sample cases. The sample should include a broad representation of the pool of individuals who serve as GALs in the jurisdiction. Staff also should determine if they must have approval to access the case files and obtain any approval that is required.

Fourth, as staff select the case sample, they create a list of judges, GALs, and child protective service caseworkers involved in the sample cases. The list should match the judges, GALs, and caseworkers to the specific case in which they were involved. These individuals will be surveyed to obtain information about GAL performance that is not available from the case record. In some instances, court staff may also need to interview the judges, GALs, and caseworkers to clarify their responses to the survey. If interviews become necessary, staff may need to request assistance in scheduling interviews with GALs and caseworkers.

Data Collection. Data collection from the case records and from the judges, GALs, and caseworkers may proceed simultaneously to reduce the time required to complete this measure. The surveys should be distributed with a cover letter from the chief or presiding judge of the division of the court that has jurisdiction over child abuse and neglect cases. The letter explains the purpose of the survey and that all responses are and will remain confidential. Provide the name of the specific case on each of the survey forms distributed to the judge, GAL, and caseworker. In some jurisdictions, one GAL, judge, or caseworker may have been involved in several cases and therefore will receive multiple surveys. In those cases, only one of the surveys should include Part III, which calls for the respondent’s general opinions about training and practice issues related to GAL representation rather than his or her views about GAL representation in a specific case. As the surveys are being prepared and distributed, court staff complete Form 1.3.1 for each of the sampled cases. As the surveys are returned, court staff should review them to determine if calls to the respondents will be needed to clarify responses.

Data Analysis and Report Preparation. Data Analysis and Report Preparation. Data Analysis and Report Preparation.

Case records: The analysis of case record data provides information on the timeliness of GAL appointments, the level of participation of GALs in court proceedings, and the degree to which GALs contribute to case dispositions. For each case, determine whether the appointment of the GAL was made within the time limit set by statute or court rule. Calculate the percentage of cases that fall within the time limit. Also calculate across all cases the average time (mean) in days between the appointment of the GAL and the filing of the petition, the emergency removal order, or other initial court action in the case. The quality of representation is likely to be higher when appointments are made within the time limit and in cases in which appointments are made shortly after the first court action taken because the GAL will have greater opportunity to assess the child’s environment and the need for placement outside the home.

Next, calculate the number and percentage of hearings in which the GAL participated. The higher the rate of GAL participation in hearings, the higher the effectiveness of representation is likely to be. To assess the level of GAL preparation, calculate the average number of required reports submitted by the GAL. To determine the extent to which GAL performance creates delays in child protection proceedings, calculate the number of continuances of hearings because the GAL was not prepared, the percentage of GAL reports filed on time, and the number of days past the deadlines reports were filed. The extent to which reports from involved agencies are in the case record indicates whether adequate information is available for the GAL to review, making preparation for the case more efficient and effective. Finally, calculate the percentage in both new and review cases in which the GAL made recommendations regarding the placement of the child. The higher the percentage of cases in which the GAL offers the court recommendations, the greater the likelihood that GALs are aggressively representing the child’s interests.

Surveys: For each GAL activity, calculate the percentage of judges, GALs, and caseworkers who reported that the activity was undertaken. To calculate these percentages, the number of "x’s" for each activity are summed across all cases and divided by the total number of cases. If an activity was marked with a "0" or an "I", that case is not counted as part of the numerator or the denominator of the percentage. In addition, calculate the mean rating of each group surveyed (judges, GALs, and caseworkers) for all cases for the items in Parts II and III of Forms 1.3.1b through 1.3.1d. The results should be calculated separately for each group so that the perceptions of the different players can be compared.

Court officials should review the average "overall ratings" in Part II first. The higher the average ratings, the better the court is performing on this measure. Is the quality of legal representation generally good (average rating of 4 or 5), or is it considered better in some areas than in others? To determine why the ratings of the quality of representation vary, court officials can examine the responses in Parts I and III. Do GALs undertake certain tasks more frequently than others? Do they demonstrate greater competence in fulfilling particular responsibilities than in completing others? Do review cases receive adequate attention? Are some important activities in representing a case neglected? Do judges, GALs, and caseworkers believe that GALs and judges receive sufficient training? Answers to questions such as these will help court officials determine whether children are being represented properly and, if not, what kinds of improvements are needed. These improvements might include additional GAL or judicial training, clearer definition of the roles and responsibilities of the GAL, implementation of compensation policies that encourage GALs to spend more time on the case, and the development of standards of practice.

Go to Form 1.3.1

Go to top of page

This measure involves observation and evaluation of the work of court interpreters by individuals who are skilled in foreign language and sign language interpretation.¹⁷ The experts observe interactions in which interpreters are involved, make an assessment of the interpreter’s proficiency, and record interpretation problems or violations of interpreter standards. This measure is only appropriate when courts can predict with reasonable certainty that interpreters will be used in specific locations during predictable timeframes.

Before arranging for this measure, court personnel should first inform themselves of the qualifications that "skilled individuals" used as observers should possess. For example, if the proposed observers are very proficient in both English and the other language but not familiar with the code of professional responsibility for court interpreters, they should not be used. "Certified" professional interpreters would make the best observers. However, they are not available for many languages nor are they available in many parts of the country.¹⁸

Planning/Preparation. Individuals skilled in foreign languages and sign communication should be identified and recruited to evaluate the court’s interpretation services. These individuals, serving as paid professionals or as volunteers, will provide an independent viewpoint of the quality of the court’s interpreter services.

It is essential for this measure that the language experts understand the requirements for interpreting in court settings. If the observers are not themselves certified court interpreters, they need to be thoroughly familiarized with the professional responsibilities of court interpreters. In addition to any State or local rules governing appropriate professional conduct, the observers should be provided with the following material from Court Interpretation: Model Guides for Policy and Practice in the State Courts:¹⁹

• Chapter 2: Interpreting Terminology

• Chapter 6: Judges’ Guide to Standards for Interpreted Proceedings

• Chapter 9: Model Code of Professional Responsibility for Interpreters in the Judiciary

Experts may be located by contacting national and state interpreter associations;²⁰ the State’s office of social services that is responsible for services to deaf or hearing impaired individuals; universities; or community agencies that serve foreign language or handicapped citizens.

Experts should be informed that what they see or hear in open court should be discussed only with court officials and that they should not attempt to intervene in any way in the cases they observe.

The next step is to select a sample of scheduled court proceedings to observe. Ideally, this sample includes both nonevidentiary and evidentiary hearings. High-volume calendars that likely will include interpreters are good choices for observation scheduling. Examples include traffic court sessions, misdemeanor arraignment and plea dockets, and child support calendars. Felony arraignment and plea calendars should be included if possible.

Observations of evidentiary hearings in which interpreters are used for witness testimony are also important to include in the sample. Pending cases should be examined to obtain a list of cases in which interpreters will be needed. When arranging for these observations, identify several proceedings that observers could go to in the same day. The key to scheduling is to ensure that if some of the scheduled proceedings are continued or delayed, other observation opportunities are available.

If a court uses interpreters infrequently, this measure should not be attempted.

Data Collection. Evaluators observe short procedural hearings in their entirety, striving to achieve as much variety in languages as possible and as many different interpreters as possible. Observations of interpreters working during witness testimony should last at least 5 minutes but not longer than 30 minutes.Data Collection. Evaluators observe short procedural hearings in their entirety, striving to achieve as much variety in languages as possible and as many different interpreters as possible. Observations of interpreters working during witness testimony should last at least 5 minutes but not longer than 30 minutes.

Using Form 1.3.2, Evaluation of Interpreter Services, the evaluator records observations regarding the quality of interpreter services. The observer first identifies the session of court and the type of proceeding observed. The specific case number, date, and time should be noted, but this may not be possible in high-volume court sessions. If different interpreters are used during a session of court, a separate form should be used for each interpreter. If one interpreter is used for several different cases, a separate form should be completed for each case.

For each interpreted session recorded on the form, the observers should rate the overall performance of the interpreter on three dimensions, as shown on the form. These dimensions are:

• Language proficiency

• Interpreting skills

• Professional conduct

If problems with the interpreter’s performance are noted during the proceeding, these should be briefly recorded on the form.

Data Analysis and Report Preparation. After the data collection is complete, the observer should prepare a brief report summarizing the observations. The report should include the following: (1) the number of individual cases that were observed, (2) the number of different interpreters that were observed, by language, and (3) a summary of the evaluation results for all of the cases observed, by language (e.g., the percentage of all cases observed where problems were noted). A summary qualitative assessment should also be provided informing the court of any problem areas that are severe in the observer’s opinion, with examples included in the narrative.

Interpreters cannot adequately perform their job without knowledge of the principles of appropriate professional conduct and basic legal and justice system terminology and concepts. Research has shown, however, that many interpreters used by the courts have not mastered these fundamentals. This measure involves administering a written test that is used to determine whether interpreters have acquired this knowledge. Other essential job requirements—language proficiency and interpreting skills—must be measured independently.

Measures 1.3.3 and 1.3.4 may be unnecessary if courts already have a valid and reliable testing process for interpreters used in their courts, including freelance interpreters. If freelance interpreters are not tested prior to employment, use of the measure should be considered.

Planning/Preparation. Preparation for this measure involves reviewing and modifying the attached model written test (see Form 1.3.3, Court Interpreter Terminology, Procedure, Protocol, and Ethics Fundamentals Test) to ensure that it reflects local terminology and concepts. After the revisions are complete, the instrument should be pretested by giving it to at least three experienced local court personnel or practicing lawyers (legal and justice system terminology) and to at least two professional court interpreters (questions related to professional conduct). No time limit should be imposed during the pilot test.

All pilot test takers should agree on which answer is correct for each test item and that there is only one correct answer. If there is disagreement, the question should be eliminated or replaced with a test item that is agreed upon by the test takers. Each test taker should also be asked to suggest cutoff scores for "excellent," "good," "acceptable," "poor," and "very poor" levels of performance on the exam. It is recommended that the test then be reviewed by at least one judge (preferably two) before setting the final criteria that will be used to evaluate individual test performance.

Data Collection. Data collection involves administering the test to all or a majority of the individuals who the court uses as interpreters, and then scoring the test using a standardized scoring guide. It is useful to prepare a scoring template to add greater speed and reliability to the scoring process.

Data Analysis and Report Preparation. Results should be analyzed using a standard statistical analysis and reporting software package, if possible. This method allows greater speed and flexibility of analysis. Every time the test is administered, the new scores should be added to the database. The analysis should include, at a minimum, a frequency report showing the number and percentage of test takers in each ranking group (i.e., "excellent," "good," etc.). It is recommended that the analysis also include frequency reports of score rankings by language, and, within each language group and overall, breakdowns by years of experience and educational level.

This measure allows the court to make an assessment of a person’s knowledge of a language other than English. The procedure can be used for virtually any language and can be applied by an examiner who speaks only English.

Measures 1.3.3 and 1.3.4 are unnecessary if courts already have a valid and reliable testing process for interpreters for the language in question, including freelance interpreters. If freelance interpreters are not tested prior to employment, the measure should be used.

Back translation is a technique in which a candidate interprets or translates English into the foreign language in question and, after the passage of time, interprets or translates her or his own foreign language version back into English. The interpreted or translated English version is then compared to the original English to determine how faithfully the original message has been preserved.

Planning/Preparation. Before undertaking this measure the court should acquire the textbook Fundamentals of Court Interpretation: Theory, Policy and Practice.²¹ The textbook includes a detailed description of the proper procedure for administering and scoring the back translation exercise, including 10 sample questions and statements with underlined scoring units. The measure also requires the use of two audiotape recorders, one for playing a recorded script and one into which the interpreter records her or his interpretation of the script.

A written script in English is then prepared in a form identical or similar to the script suggested in Fundamentals of Court Interpretation. The written script is read aloud into a tape recorder in the same way that an attorney would pose a question to a witness or a witness would answer a question. Between each prerecorded question or statement there must be a pause long enough for the interpreter to complete the interpretation.

To conduct the measure the court identifies all interpreters who work regularly in the court and plans a testing schedule. The schedule should require the interpreters to report to the testing room on two separate occasions. On the first occasion the interpreter listens to a tape-recorded passage in English and interprets it aloud in the foreign language, using a second tape recorder to record the foreign language rendition. On the second occasion the interpreter will listen to her or his own recorded foreign language rendition of the original script and interpret it back into English. The interval between the first occasion and the second occasion may be as little as one hour. However, separating the occasions by one or several days is not only acceptable but may result in a better test because the passage of time reduces the opportunity for the interpreter to rely on memory of the original English. Approximately 15 minutes should be allocated for each interpreter for each test session.

The final preparation step is to select one or two individuals to score the test results. These individuals should have highly developed language skills in English and be able to discern the difference between substitution of words and distortion of meaning.

Data Collection. Data collection consists of administering the test to the candidates as summarized above and as described in more detail in Fundamentals of Court Interpretation (pp. 196–199). Test raters listen to each interpreter’s back-translated English version of the script and compare it to the original. The script will contain approximately 40 underlined scoring units that are used to determine the individual’s score. The resulting data sources are scoring sheets prepared for each interpreter by the test rater showing the number of scoring units on the back translation that match the meaning of the original English script.

Data Analysis and Report Preparation. After all of the tests are scored, a listing of their scores should be prepared. The analysis should then report the summary results in terms of percentiles, as shown in the following table.

Research and experience with court interpreter testing suggests that analysis of test results should examine test scores of interpreters by language groups. One obvious way to do this in most States is to prepare a report that distinguishes the test results for Spanish language interpreters from other languages. In interpreting the results, the court’s policymakers should draw their own conclusions about what is an acceptable level of performance. The mathematics speak for themselves in terms of performance: an interpreter who gets 20 correct items is only rendering one-half of the questions or testimony accurately; a score of 30 correct implies that 25 percent of the "message" is changed, distorted, or lost altogether in the process of being rendered from one language to another.

This measure examines access to courthouse facilities by persons with physical disabilities. The measure produces two kinds of information: general information about accessibility in the courthouse as a whole and more specific information on the ease or difficulty with which individuals with disabilities conduct business transactions with the court. (The volunteers who complete this measure may also be included in the samples for Measure 1.2.6, Evaluation of Accessibility and Convenience by Court Users, and Measure 1.4.2, Observers’ Assessment of Court Personnel’s Courtesy and Responsiveness.)

Planning/Preparation. The first step is for court staff to complete Part I of Form 1.3.5, Access to Courthouse Facilities by Individuals with Disabilities. Part I is based on a checklist developed by the National Center for State Courts to assess court compliance with the Americans with Disabilities Act.²² These questions will provide the court with general information about the accessibility and usability of courthouse facilities and services as a whole for persons with disabilities.

Next, court officials should prepare a list of routine activities that citizens engage in while using the court’s services. These should include: (1) transacting business in the clerk’s office, (2) appearing for jury duty, (3) observing a domestic relations calendar, (4) observing a criminal arraignments calendar, (5) observing a trial or simulating the experience of being a litigant during a trial (e.g., visit the courtroom, sit in the litigation area), (6) accessing facilities for special services such as ADR program offices, child support complaint and payment offices, and bail payment windows, and (7) using general courthouse facilities such as cafeterias, restrooms, attorney-client conference rooms, and public telephone areas.

Local service agencies or advocacy associations for individuals with physical disabilities should be contacted to obtain the names of individuals who may be willing to participate in a simulation exercise. At a minimum, two individuals confined to wheelchairs and two individuals with a visual impairment should be asked to visit the courthouse.

Data Collection. A list of the simulation activities and a copy of Part II of Form 1.3.5 is given to each volunteer. (The information is provided verbally as well.) Each volunteer should attempt each activity and note the results of each simulation on Form 1.3.5. (Volunteers with visual impairments will need assistance in recording the results of the simulations.) Form 1.3.5 allows 15 simulations to be recorded. If more than 15 simulations are conducted, the form should be modified to accommodate the additional simulations. Each simulation should be started from outside the courthouse. Volunteers should record the length of time it takes to conduct each simulation, the ease with which the activities are accomplished, and any specific problems encountered.

After the volunteers have completed the simulations, court officials should schedule a meeting with them. During the meeting, the volunteers can compare their experiences with one another and discuss possible improvements for making the court more accessible to individuals with disabilities. Problems encountered in obtaining the resources to make improvements should be described, and both court officials and the volunteers should discuss possible options for overcoming the problems. Court officials may also choose to interview regular users of the court who have disabilities (such as court employees or attorneys) to discuss problems they encounter while working in the courthouse, as well as suggestions they have for improvements.

Data Analysis and Report Preparation. Court staff first review the answers to Part I of Form 1.3.5 and consider whether the court has adequately addressed the issue of access for persons with disabilities. To what extent are employees knowledgeable about policies and procedures related to accommodating persons with disabilities? Did the answers to these questions provide a favorable or unfavorable impression of the courthouse’s accessibility for all persons?

Next, court staff review the results of the simulation exercises. The average rating for "ease of conducting business" for all simulations is calculated. (If 4 volunteers each rated 15 simulations, the average would be based on 60 simulations.) The closer the average rating is to "1," the better the court is performing on this measure.

The average length of time needed to conduct each simulation also is calculated. Did some activities take longer than others? If so, what specific problems were encountered? Does an examination of all the simulated activities reveal that some areas of the courthouse are less accessible than others?

Using both the general information gathered in Part I and the more specific information gathered from the simulations in Part II, court officials should summarize where the most serious problems exist and develop an action plan (incorporating the volunteers’ suggestions, if possible) for alleviating the problems. If court officials have not already done so, they should refer to the Americans with Disabilities Act for suggestions and requirements when developing their plan.