(This syllabus is not part of the opinion of the Court. It has been prepared by the Office of the Clerk for
the convenience of the reader. It has been neither reviewed nor approved by the Supreme Court. Please
note that, in the interests of brevity, portions of any opinion may not have been summarized).
Argued February 4, 1997 -- Decided May 22, 1997
HANDLER, J., writing for a majority of the Court.
In this appeal, the Court addresses the extent of access to test materials and grading scales given a
civil-service candidate in the review of his test scores.
In October 1992, James Brady, who is currently a police sergeant in the Atlantic City Police
Department, took a civil-service exam to attain the rank of captain. The exam was administered by the
Department of Personnel (“DOP”) and is designed to test a candidate's ability to respond to specific
situations that may arise in the course of duty. The exam consists of both a written and oral part, both of
which evaluate various categories of behavioral characteristic or “dimensions.” Brady's overall score,
although passing, reduced his relative eligibility for promotion.
Brady, who was unhappy with his overall score, appealed through the DOP's administrative
channels. He was permitted to review a portion of his written test materials, including his answers, brief
summaries of the questions, brief comments by the grader, and an explanation of the scoring process. He
also received an audio tape of the oral component of his exam. Pursuant to its internal policy, however, the
DOP had placed significant limitations on Brady's ability to review those materials. Specifically, Brady was
allowed only one hour to review all the materials provided and could not copy any of the materials. He was
not given access to the actual test questions or to the answer key, which identified several “possible courses
of actions” (“PCAs”) upon which the grading was based.
Based on his review of the materials, Brady wrote to the DOP Selection Appeals Unit, expressing
his disagreement with his scores and requesting that the exam be regraded. Nine months later, a supervisor
in the Selection Appeals Unit replied to Brady's request with an analysis of his score. In that analysis, the
supervisor addressed the concerns that Brady had raised and broke down his score for each dimension. She
further concluded that Brady's assigned scores were accurate and appropriate.
After expressing her conclusion, the supervisor informed Brady that he could appeal the decision to
the Merit System Board (“the Board”), but that the Board would only consider the proofs, arguments and
issues presented at the previous level of appeal. Apparently relying on that information, Brady appealed to
the Board but did not advance any new arguments, including his belief that the supervisor erroneously had
relied on information to which plaintiff had not been given access (i.e., the PCAs). The Board subsequently
denied Brady's appeal, noting that he had provided no arguments, submissions or issues in support of his
appeal, other than those raised and already considered in the appeal below.
Brady appealed the Board's determination to the Appellate Division, which ordered production of
all test materials. The court based its decision on the need of both a court reviewing and a party challenging
an administrative determination to have access to the record upon which the agency has acted. The
Appellate Division subsequently refused to stay its order, thus allowing Brady immediate access to the
materials, subject to a protective order.
The Supreme Court granted the DOP's petition for review.
HELD: The DOP's provision for partial or limited access to civil-service examination materials is a valid
exercise of the agency's regulatory authority and represents a reasonable balance between its interest in the
confidentiality of the exam process and an examinee's interest in reviewing the grading of examinations.
1. This case's technical mootness is not a bar to the Court's exercise of jurisdiction. (p. 8)
2. In keeping with the New Jersey Civil Service Act's general policy of encouraging employment that
focuses on merit, the Act vests the DOP with the authority to devise a fair, secure, merit-based testing
process by which candidates are selected for employment and promotion. (pp. 9-11)
3. Brady's contention that he was entitled to greater access to his exam materials must be considered
against the standard of review of whether the DOP's limitation of access was arbitrary, capricious, or
unreasonable. (pp. 11-12)
4. An agency decision may not be based on undisclosed evidence. (pp. 12-13)
5. Courts may not routinely review the contents of civil-service examinations and answers and determine
whether the questions were well or poorly answered, as such an inspection and review would involve a
challenge to the substantive validity of the examination. (pp. 13-15)
6. The DOP has not abused its discretion in deciding to recycle test questions or in making a determination
to limit access to test materials in order to ensure confidentiality and security. (pp.15-17)
7. Given the general prohibition against judicial regrading of examination, full disclosure would confer little
or no administrative or litigational benefit on the examinee. (pp. 17-20)
8. To the extent that Martin v. Educational Testing Service suggests a requirement of full disclosure of civil-service examinations without regard to security and confidentiality concerns, it is overruled. (p. 20)
9. A candidate may be able to make a prima facie showing of arbitrariness or discrimination in grading that
is so obvious and rises to such a high level that the full exam materials must be produced. (pp 20-24)
10. The supervisor's potentially erroneous statement that the Board would not consider new arguments was
harmless and provides no basis for a reversal of the Board's denial of Brady's appeal.
Judgment of the Appellate Division is REVERSED.
JUSTICE STEIN filed a separate opinion, dissenting in part and concurring in part. Justice Stein
did not disagree with the Court's determination that the Appellate Division erred in holding that all persons
challenging their test scores must be provided with copies of the questions, their answers, and the grading
standards. However, he believed that the Court's opinion tipped the balance too far toward the interests of
confidentiality when it precluded disclosure of relevant test materials to the reviewing court and applicant,
unless the applicant makes a prima facie showing that the test results are arbitrary. Rather, he believed that
the Court could strike a more fair balance by requiring the DOP to furnish the reviewing court, in camera,
with the complete materials to enable it to make a preliminary assessment of arbitrariness and determine
whether further disclosure or other relief may be appropriate.
CHIEF JUSTICE PORITZ and JUSTICES POLLOCK, O'HERN, GARIBALDI and COLEMAN
join in JUSTICE HANDLER's opinion. JUSTICE STEIN filed a separate opinion, dissenting in part and
concurring in part.
SUPREME COURT OF NEW JERSEY
A-
103 September Term 1996
JAMES T. BRADY,
Appellant-Respondent,
v.
DEPARTMENT OF PERSONNEL,
Respondent-Appellant.
Argued February 4, 1997 -- Decided May 22, 1997
On certification to the Superior Court,
Appellate Division, whose opinion is reported
at
289 N.J. Super. 557 (1996).
June K. Forrest, Senior Deputy Attorney
General, argued the cause for appellant
(Peter G. Verniero, Attorney General of New
Jersey, attorney; Mary C. Jacobson, Assistant
Attorney General, of counsel).
D. William Subin argued the cause for
respondent (Mr. Subin, attorney; Mr. Subin
and Steven P. Scheffler, on the briefs).
Paul L. Kleinbaum argued the cause for amicus
curiae New Jersey State Policemen's
Benevolent Association (Zazzali, Zazzali,
Fagella & Nowak, attorneys).
Dennis J. Alessi submitted a brief on behalf
of amici curiae New Jersey State Firemen's
Mutual Benevolent Association and Newark
Firefighters Union (Fox and Fox, attorneys).
The opinion of the Court was delivered by
HANDLER, J.
In this case, a municipal police sergeant, seeking to attain
the rank of captain, took a State promotional civil-service
examination. His examination score reduced his chances of
promotion. The officer challenges the results of his
examination, claiming that in order to determine the accuracy of
his score, he must have complete access to his testing materials,
including the actual exam questions and the standards used to
grade his answers. He contends that without full discovery of
the examination materials, he will not be able to obtain
meaningful administrative and judicial review. The State allowed
the officer only limited access to the materials. It based this
limitation on the need to preserve the integrity of the civil-service examination process and the security and confidentiality
of civil-service examinations, which it believed would be
compromised by permitting full access to the materials.
In October 1992, plaintiff James Brady, who is currently a police sergeant in the Atlantic City Police Department, took a civil-service examination to attain the rank of captain. The examination is administered by the Department of Personnel ("DOP") and is designed to test a candidate's ability to respond to specific situations that may arise in the course of duty. It consists of two parts, a written component, known as the "in-basket" exercise, and an oral component based on a video exercise. Candidates first take the written portion, which lasts
for three hours, and those who exceed a particular score then
qualify to take the oral portion, although scoring highly enough
on the written part to proceed to the oral part does not
guarantee that the examinee has performed exceptionally on the
written portion.
The written portion, consisting of ten questions, is
designed to evaluate candidates based on nine behavioral
characteristics or "dimensions." The dimensions include
analysis, judgment, decisiveness, ability to delegate, community
sensitivity, leadership, planning and organizational ability,
management capability, and written-communication skills. These
characteristics are not tested in isolation; instead, each of the
ten questions tests multiple characteristics.
Although written answers respond to issues raised in the
context of each of the ten questions, the actual grading of the
written portion does not proceed question by question, but rather
dimension by dimension, with each dimension receiving a score
ranging from one (lowest) to five (highest). In assessing each
dimension, graders compare examinee responses to a list of
"possible courses of actions" ("PCAs"), crediting responses for
the PCAs that examinees recognize and marking off for the PCAs
that they miss.
The oral portion of the examination tests six categories:
analytic ability, judgment, decisiveness, leadership ability,
community sensitivity, and oral-communication skills. Like the
written portion, the oral portion employs a grading system of one
(lowest) to five (highest).
Once a candidate has taken both parts of the examination, he
or she receives an overall score that ranks the examinee among
all exam-takers. This ranking determines a candidate's relative
eligibility for promotion.
Plaintiff performed sufficiently well on the written portion
of the examination to proceed to the oral component, but his
written score was not outstanding.See footnote 1 In contrast, he excelled on
the oral portion.See footnote 2 His overall score, although passing,
reflected his lackluster written performance, thus reducing his
promotion opportunities.
Unhappy with his overall score and with his written score in
particular, plaintiff appealed through the DOP's administrative
channels. He was permitted, approximately one year after he had
taken the test, to review a portion of his written test
materials, including his answers, a very brief summary of each
question, brief comments by the grader about the PCAs that
plaintiff had missed, an Orientation Background Guide (which he
previously had received), and an explanation of the scoring
process. He also received an audio tape of the oral component of
his examination.
Pursuant to its internal policy, however, the DOP placed
significant limitations on plaintiff's ability to review the test
materials. First, he received only one hour to review them,
which was actually fifteen minutes longer than DOP guidelines
provided. Second, he did not gain access to the actual test
questions or the answer key containing all potential PCAs.
Finally, he could not copy any of the materials, although he
could take notes. Those restrictions apparently derive from the
DOP's "Examination Review Policy for the Police Promotional
Assessment Process," issued in Fall 1992, which allows examinees
"to review their own responses to the examination (written/tape),
the examination instructions and the scores they achieved on each
dimension for each examination exercise, along with a brief
descriptive statement of their performance on each dimension" and
which provides for a forty-five-minute review period.
Based on his brief review of the limited materials,
plaintiff wrote a letter to the DOP Selection Appeals Unit on
September 13, 1993, in which he expressed his disagreement with
his scores on several of the written dimensions and requested
that the examination be regraded. Nine months later, Anita
Mathes, a supervisor in the Selection Appeals Unit, replied by
letter, dated May 31, 1994, with an analysis of plaintiff's
score. In the analysis, she addressed the concerns that he had
raised and broke down his score for each dimension. She also
noted many of the PCAs that he had missed on the written
component. Mathes then concluded:
In reply to your appeal of the scoring of
your in-basket examination, a review of your
in-basket dimension scores revealed that in
each case, significant opportunities to
demonstrate behaviors associated with
specific dimensions were missed. As stated
previously, these possible courses of action
were generated by a panel of subject matter
experts consisting of senior command
personnel from police departments. After
reviewing your responses in light of the
guidelines and parameters established, we
conclude that your assigned scores are
accurate.
After expressing this conclusion, the letter informed
plaintiff that he could appeal the decision to the Merit System
Board ("the Board"). The letter, however, contained the caveat
that "the Board will only consider the proofs, arguments and
issues presented at the previous level of appeal. No new or
additional proofs, arguments or issues will be considered at the
next level of appeal." Apparently relying on that advice,
plaintiff appealed to the Board but did not advance any new
arguments, including his belief that the supervisor erroneously
had relied on information (i.e., the complete list of PCAs) to
which plaintiff had not had access. The Board subsequently
denied his appeal, noting that "Mr. Brady provides no arguments,
submissions or issues in support of his appeal, but relies on
proofs and arguments presented below" and concluding that because
the supervisor had "addressed all arguments and contentions
raised by appellant[,] . . . [t]he present record fails to
provide a basis to disturb [the Supervisor's] decision."
Plaintiff then appealed the Board's determination to the
Appellate Division, which ordered production of all test
materials,
289 N.J. Super. 557 (1996), concluding that
"procedural fairness requires that a person who is entitled to
appeal from the grade awarded him or her on a written civil
service examination must be furnished with a copy of the
questions and of his or her answers." Id. at 565. The court
based its decision on the need of both a court reviewing and a
party challenging an administrative determination to have access
to the record upon which the agency has acted:
The limited disclosure afforded [plaintiff]
was not sufficient to satisfy the due process
requirement that all of the evidence on which
an agency bases its decision must be revealed
to the appellant. Without custody of the
questions and answers, he was not in a
position to obtain the advice of an attorney
or of an expert on the subject matter of the
examination. Without that advice and some
reasonably complete explanation of the basis
for the grader's evaluation of his answers,
he is not able to intelligently dispute or
acquiesce in the grade he was awarded.
The Appellate Division recognized that its "appellate duties [did] not include grading civil service examination questions," ibid., but nevertheless reasoned that
after we have accorded the utmost deference to the Department of Personnel and to its technical expertise, . . . [we must] judge whether there is a reasonable basis for its determinations. . . . We cannot perform that review function without the questions, the answers, and some expert explanation to tell us in what respects the questions were well or poorly answered.
The court left the initial "good faith determination" to the DOP of "how or subject to what conditions" the information was to be provided. Id. at 565. The Appellate Division subsequently refused to stay its order, thus allowing plaintiff immediate access to the materials subject to a protective order.
We granted the State's petition for certification. 146 N.J. 565 (1996).
As an initial matter, we reject plaintiff's suggestion that, because the DOP already has provided him with the test materials, the case is moot. Although the controversy between the parties about access to these particular materials is technically moot, the issue presented is of sufficient importance that review by this Court is appropriate. See In re Geraghty, 68 N.J. 209, 212 (1975) ("[W]e have often recognized that courts may hear and decide cases which are technically moot where issues of great public importance are involved."). Moreover, as evidenced by the Appellate Division's refusal to stay its decision in this case and the pendency of similar cases, the issue presented is capable of repetition yet likely to evade review. See In re J.I.S. Indus. Serv. Co. Landfill, 110 N.J. 101, 104-05 (1988); In re Conroy, 98 N.J. 321, 342 (1985). Consequently, the case's technical mootness is not a bar to our exercise of jurisdiction.
New Jersey's Constitution and the New Jersey Civil Service Act ("the Act"), N.J.S.A. 11A:1-1 to 12-6, express a general policy that selection for State employment should be dependent on considerations of merit. N.J. Const. art. VII, § 1, ¶ 2; N.J.S.A. 11A:1-2c; see Connors v. City of Bayonne, 36 N.J. Super. 390 (App. Div.), certif. denied, 19 N.J. 362 (1955). In order to carry out this policy, the Act creates the DOP and endows it with broad power over the selection and retention of State employees. N.J.S.A. 11A:2-6.
Regarding employee hiring and promotion, the Act vests the DOP with substantial authority over the civil-service examination process. It states:
The commissioner shall provide for:
a. The announcement and administration of
examinations which shall test fairly the
knowledge, skills and abilities required to
satisfactorily perform the duties of a title
or group of titles. The examinations may
include, but are not limited to, written,
oral, performance and evaluation of education
and experience;
b. The rating of examinations;
c. The security of the examination process
and appropriate sanctions for a breach of
security;
e. The right to appeal adverse actions relating to the examination and appointment process . . . .
Thus, in keeping with the Act's general policy of encouraging a
system of state employment that focuses on merit and its
delegation of authority to the DOP to further that end, the Act
vests the DOP with the authority to devise a fair, secure, merit-based testing process by which candidates are selected for
employment and promotion.
In exercising its authority under the Act, the DOP has
promulgated several regulations dealing with access by examinees
to testing materials. N.J.A.C. 4A:4-2.16 requires retention of
the bulk of test materials for a set time period after an
examination. Moreover, N.J.A.C. 4A:4-6.4 provides for a degree
of access by examinees to test materials, but it provides the DOP
with substantial discretion in determining the appropriate level
of access. N.J.A.C. 4A:4-6.4(e) ("In order to maintain the
security of the examination process, the Commissioner may, on a
particular examination, modify or eliminate the review of
examination questions and answers.").
The DOP has exercised its discretion under N.J.A.C. 4A:4-6.4(e) to preclude examinee access to the actual test questions
and the PCA scoring key. It bases those restrictions on the need
for security and confidentiality. Apparently, the DOP reuses
significant numbers of exam questions from test to test, either
in identical or similar form. Its reasons for this practice are
the expense involved in formulating entirely new questions for
each test, the consistency that recycling questions achieves
among exams given at different times, and the ability to evaluate
the effectiveness of questions that results from providing the
questions to a large pool of examinees. Given the massive number
of civil-service examinees and the hundreds of tests administered
each year, the DOP has concluded that its reuse of test questions
would render it impossible to ensure the integrity of the testing
process if candidates were to have full access to exam materials.
Moreover, according to the DOP, full access would lead to
significantly increased costs in propounding test questions and
would impair the DOP's ability to contract with private testing
firms to provide this service.
Plaintiff's basic contention is that he was entitled to greater access to his examination questions and to the specific testing criteria that were used to evaluate his examination. That contention must be considered against the standard of review of whether the DOP's limitation of access was arbitrary, capricious, or unreasonable. Public Serv. Electric & Gas Co. v. Department of Envtl. Protection, 101 N.J. 95, 103 (1985). In applying that standard of review, we are mindful of the deference that courts must accord agency action that purports to effectuate statutory and regulatory authority. Id. As we recently stated:
Courts have only a limited role to play in reviewing the actions of other branches of government. In light of the executive function of administrative agencies, judicial capacity to review administrative actions is severely limited. . . . Courts can intervene only in those rare circumstances in which an agency action is clearly inconsistent with
its statutory mission or other state policy.
Although sometimes phrased in terms of a
search for arbitrary or unreasonable action,
the judicial role is generally restricted to
three inquiries: (1) whether the agency's
action violates express or implied
legislative policies, that is, did the agency
follow the law; (2) whether the record
contains substantial evidence to support the
findings on which the agency bases its
action; and (3) whether, in applying the
legislative policies to the facts, the agency
clearly erred in reaching a conclusion that
could not reasonably have been made on a
showing of the relevant factors.
[In re Musick,
143 N.J. 206, 216
(1996) (citations omitted).]
The deference that courts accord an agency's actions that
fall within its delegated authority is not without limits. One
of those limits is embodied in the general principle of judicial
review that an agency decision may not be based on undisclosed
evidence. See High Horizons Dev. Co. v. Department of Transp.,
120 N.J. 40, 53 (1990) ("[A]n agency is never free to act on
undisclosed evidence that parties have had no opportunity to
rebut."); Brotherhood of R.R. Trainmen v. Palmer,
47 N.J. 482,
487 (1966) ("The [administrative] determination cannot rest upon
undisclosed evidence which the parties have had no opportunity to
test for trustworthiness or to explain or rebut."); see also In
re Dep't of Ins. Order,
129 N.J. 365, 383 (1992) ("One of the
core values of judicial review of administrative action is the
furtherance of accountability.").
In conducting judicial review of the DOP's administration of
the civil-service system and its determinations regarding civil-service testing processes, we note its broad regulatory
authority and the limited role allotted to the judiciary. We
long have recognized that
[t]he preparation and administration of civil
service examinations is an administrative
function delegated most liberally to the
authorized examiners of the Department . . .
by the Legislature. . . . The fulfillment of
that function is a matter requiring special
expertise, involving as it does the
determination of what job knowledge, skills
and abilities are necessary or desirable in a
candidate for a particular position, and the
highly technical problem of devising suitable
examination questions which will demonstrate
as accurately as possible whether an
applicant possesses those requirements
sufficiently to qualify for the position. . .
. In view of the above, the courts cannot
intervene to nullify a civil service
examination unless it is clearly shown that
the Department has abused its discretion.
[Zicherman v. Department of Civil
Serv.,
40 N.J. 347, 350-51 (1963)
(quotations and citations
omitted).]
See also Artaserse v. Department of Civil Serv.,
37 N.J. Super. 98, 105 (App. Div. 1955) ("[T]he judiciary is not disposed to
engage in a critical supervisory examination of the composition
of the questions to be propounded to candidates in promotional
tests . . . or to exercise an appellate review of the scoring of
the answers . . . .").
In this case, the gravamen of plaintiff's complaint is that
he did not receive a fair score. The question, then, is what
evidence relating to his examination is plaintiff entitled to
have disclosed in order to substantiate that claim. The
answer to that question is that plaintiff is entitled to obtain
only such evidence that reasonably may enable an examinee to
assess the correctness of his or her answers and to demonstrate
that the DOP's grading of his or her examination constituted an
abuse of discretion. Zicherman, supra, 40 N.J. at 351.
Consistent with that standard for determining the validity
of the DOP's grading process, courts in only very rare cases have
invalidated agency scoring, namely, cases in which the person
challenging the score affirmatively has shown that the scoring
method was arbitrary. E.g., Rox v. Department of Civil Serv.,
141 N.J. Super. 463 (App. Div. 1976) (annulling test scores based
on unsound and overly subjective testing practices). Such cases,
however, simply express the narrow proposition that obviously
arbitrary grading systems are invalid. They in no way counter
the general rule that courts will defer to an agency's grading of
a civil-service examination except in the most exceptional of
circumstances that disclose a clear abuse of discretion.
The court in Lavash v. Kountze,
604 F.2d 103 (1st Cir.
1979), aff'g
473 F. Supp. 868 (D. Mass.) rejected an invitation
to broad judicial review of agency test scoring:
What appellant wants is the right to have his
test results reviewed and to have input into
that review to foreclose the possibility that
his score might have been adversely affected
by the form of the questions and/or the
preparer's determination of the correct
answers. The private interest affected here,
failure to be promoted through an erroneous
determination, is almost chimerical. Opening
up . . . examination results to inspection
and review by all dissatisfied applicants for
promotion would impose fiscal and
administrative burdens out of all proportion
to the ends sought. Such inspection and
review would involve a challenge to the
substantive validity of the examination.
Court proceedings would require the opinion
testimony of experts and submerge the court
in the testing and grading process.
The Appellate Division, in ordering full revelation of the testing materials, was attempting to ensure agency accountability and procedural fairness in the grading process. Although this goal is an important one, we believe that the Appellate Division did not afford sufficient weight to the limitations on judicial review in the context of the DOP's administration of civil-service examinations. Consequently, the Appellate Division, in stating that, in order to perform its "review function," it required access to "the questions, the answers, and some expert explanation to tell us in what respects the questions were well or poorly answered," 289 N.J. Super. at 564 (emphasis added), imposed on itself a decisional responsibility that the court was not required to assume. Courts may not routinely review the contents of civil-service examinations and answers and, aided by expert testimony, determine whether the questions were "well or poorly answered." "Such inspection and review would involve a challenge to the substantive validity of the examination." Lavash, supra, 604 F. 2d at 105. Instead, reviewing courts may conduct only a limited review of the reasonableness of a grading system and determine simply whether the testing and grading were clearly arbitrary.
The limited role of judicial review in this context applies to the two DOP determinations that are challenged by plaintiff.
First, the DOP has determined that test security, and hence the
need for confidentiality, are essential to preserve the integrity
of the examination process and to discharge the State's
constitutional duty to hire and to promote based on merit.
N.J.A.C. 4A:4-6.4(e). Second, it has concluded that the optimal
accommodation of its competing obligations to guarantee "[t]he
security of the examination process," N.J.S.A. 11A:4-1c, and to
ensure "[t]he right to appeal adverse actions relating to the
examination and appointment process," N.J.S.A. 11A:4-1e, is to
provide aggrieved examinees with certain materials but not
others.
Regarding the first determination, the Act specifically
requires the DOP to "provide for the security of the examination
process," N.J.S.A. 11A:4-1c, and the disclosure of test questions
and scoring keys that are likely to be reused in future tests
clearly implicates exam security. Although plaintiff contends
that the DOP, in administering civil-service examinations, has
abused its discretion by reusing exam questions, that decision is
clearly within the range of responsibilities that the
Legislature has delegated to the DOP to implement the most
effective and efficient procedure to assure public hiring and
promotion based on merit. See Brotspies v. Department of Civil
Serv.,
66 N.J. Super. 492, 495-96 (App. Div. 1961). Because we
are unable to conclude that the DOP has abused its discretion in
deciding to recycle test questions, based on significant concerns
about achieving exam consistency and avoiding unnecessary cost,
we cannot take issue with its determination that confidentiality
in some form is the most effective means of fulfilling the Act's
mandate that test security be ensured. Cf. Brotspies v.
Department of Civil Serv.,
72 N.J. Super. 334, 342 (App. Div.
1962) (citing Brotspies, supra, 66 N.J. Super. at 495-96, and
approving of its statement that "[s]ince the Department may
desire to use many of the items . . . for examinations for other
positions, then once a particular set of questions and correct
answers is published, all of the value resulting from the
empirical evidence so carefully collated by the Department is
destroyed").
The second determination by the DOP, relating to the proper
balance between test security and an examinee's right to
challenge his or her score, poses a more difficult issue because
it concerns the proper procedure for ensuring fair, secure,
merit-based testing. The DOP has implemented a procedure by
which an examinee has access to all test materials except for the
actual test questions and the PCA answer key; in lieu of these
items, the examinee receives a summary of the questions and a
brief explanation of the PCAs that the examinee actually missed.
Moreover, the examinee is given only forty-five minutes to review
and to take notes on the materials and may not copy them.
Finally, at the examinee's request, a supervisor reviews the
entire exam and, in a detailed letter, describes many of the PCAs
that the examinee missed. Under these procedures, according to
the DOP, an examinee has sufficient access to the relevant
materials and a reasonable opportunity to evaluate the grading
process. The examinee, however, is not provided additional
access to the materials because that access would increase the
risk of misuse through dissemination, thus undermining the
security and integrity of the testing process. Plaintiff
complains that without full access to examination materials and a
greater opportunity to assess the examination methodology and
results, examinees and courts cannot ensure that the DOP does not
misgrade examinations.
We conclude that the DOP's accommodation of the competing
goals of examination security and examinee access is neither
arbitrary, capricious, nor unreasonable. Plaintiff essentially
seeks full access to the testing materials as a basis for
challenging and obtaining broad judicial review of the accuracy
of the agency's scoring of the examination. However, we already
have rejected the proposition that courts should engage in full
judicial review under these circumstances, given that "the
judiciary is not disposed to engage in a critical supervisory
examination of the composition of the questions to be propounded
to candidates in promotional tests . . . or to exercise an
appellate review of the scoring of the answers . . . ."
Artaserse, supra, 37 N.J. Super. at 105.
In the vast majority of cases, full disclosure thus would
confer little or no administrative or litigational benefit on the
examinee, given the general prohibition against judicial
regrading of examinations. However, full disclosure would wreak
havoc with the DOP's legitimate efforts to maintain security.
This balance clearly supports the reasonableness of the
accommodation that the DOP has chosen between test security and
examinee access. Cf. Mathews v. Eldridge,
424 U.S. 319,
96 S.
Ct. 893,
47 L. Ed.2d 18 (1976) (balancing government and private
interests and determining that due process did not require
additional process where process would not benefit individual
interest substantially and would harm government interest).
Other courts have reached similar conclusions and thus have
approved limitations on examinee access to civil-service
examination materials based on security and confidentiality
concerns. See, e.g., Lavash, supra,
604 F.2d 103; Civil Serv.
Comm'n v. Pinder,
812 P.2d 645 (Colo. 1991) (en banc); Garner v.
Department of Personnel,
835 P.2d 527 (Colo. Ct. App. 1992)
(same), cert. denied,
507 U.S. 917,
113 S. Ct. 1274,
122 L. Ed.2d 669 (1993); Patch v. Civil Serv. Comm'n,
295 N.W.2d 460 (Iowa
1980). Federal regulations also parallel the DOP's approach of
limiting access to test materials. 5 C.F.R. § 300.201; 14
C.F.R. § 1212.205 (National Air and Space Administration's
application of 5 C.F.R. § 300.201).
We note that in Martin v. Educational Testing Service,
179 N.J. Super. 317, 326 (Ch. Div. 1981), the court held that because
the plaintiff in that case had a protectible interest in accurate
grading, due process required full revelation of testing
materials in order to evaluate whether the scoring was in fact
correct. Id. at 326-27. Martin may be distinguishable in that
the test was developed and administered by Educational Testing
Service, a private entity. However, to the extent that Martin
suggests a requirement of full disclosure of civil-service
examinations without regard to security and confidentiality
concerns, it is overruled.
We thus conclude that the DOP has chosen a reasonable
balance between its interest in the confidentiality of the
examination process and examinees' interest in reviewing the
grading of examinations, and that the provision for partial or
limited access to examination materials is a valid exercise of
the agency's regulatory authority.
Our conclusion regarding nondisclosure should not be read to insulate the civil-service scoring process from judicial review entirely. If the DOP were, in the name of confidentiality and security, to deny examinees all access to testing materials, such a decision almost surely would be arbitrary, capricious, or unreasonable because it would allow the DOP to conduct the testing and grading process without any accountability and would foreclose any opportunity on the part of an examinee to demonstrate the unreasonableness or unfairness of his or her examination. The DOP, however, has not gone so far in its efforts to ensure the integrity of the process, instead allowing limited access to most of the materials and providing sufficient
opportunity for examinees to discover grading irregularities,
while not jeopardizing test security.
We note that despite the general reasonableness of the DOP's
limitation on examinee access, even this amount of access may
prove insufficient if an examinee makes a prima facie showing of
more than the mere possibility of misgrading, thus justifying
more extensive disclosure. Even the DOP admits that certain
cases call for greater access. For example, a candidate may be
able to make a prima facie showing of arbitrariness or
discrimination in grading that is so obvious and rises to such a
high level that the full materials must be produced. Although we
do not impute any precedential authority to unpublished
decisions, R. 1:36-3, we are aware that some courts have
acknowledged instances of such demonstrated arbitrariness. See,
e.g., In re Setkiewicz, No. A-2123-94T2, slip op. at 5 (N.J. App.
Div. Nov. 27, 1995) ("We think it plain that some prima facie
showing must be made to justify . . . access [by examinees to
full test materials]. We are satisfied, however, that the
rescoring here justified appellant's demand."); Dellaventura v.
Department of Personnel, No. A-3957-91T1 (N.J. App. Div. Mar. 11,
1993) (allowing access based on specific prima facie showing of
arbitrariness in grading).
We stress that such a prima facie showing requires more than
an allegation of the mere possibility of erroneous scoring.
Instead, it must be based on a specific allegation of
arbitrariness or discrimination. See Artaserse, supra, 37 N.J.
Super. at 105 ("The courts . . . will intervene to nullify such
an examination where it is affirmatively shown to have been
manifestly corrupt, arbitrary, capricious, or conspicuously
unreasonable."). Under DOP procedures, examinees appear to have
a reasonable opportunity to make such a showing. They are
provided with a summary of the questions, their complete answers,
and a summary of the PCAs. Further, they are entitled to an
internal review in which their answers are reexamined, and they
are furnished with personalized explanations of the basis for
their grade on each dimension. An examinee should be able to
determine from this process whether the DOP appears to have acted
arbitrarily in scoring his or her examination.
In this case, however, plaintiff has made no such prima
facie showing of arbitrariness despite his substantial access to
the examination materials. This failure is especially stark
given plaintiff's presumed knowledge and experience as a police
officer and his access to other police officers and union
officials who doubtlessly have a degree of expertise in the
civil-service process. With those resources, plaintiff should
have been able, after consultation, to make some sort of proffer
about how the DOP had acted arbitrarily in scoring his exam.
Instead, without offering any particular allegation of
arbitrariness, he simply asserts that his score is too low and
that he therefore should have unfettered access to the materials.
We agree with the separate opinion that, under our holding,
the possibility exists that examinees and reviewing courts will
not catch each and every instance of misgrading by the DOP. Post
at __-__ (slip op. at 1-2). However, despite the validity of
that concern, we believe that it overstates the problem posed by
limited access. Examinees receive more than "bare-bones access,"
post at __ (slip op. at 2), to their exams and to the PCAs used
to grade them. The DOP, as we have noted, initially provides
examinees with their complete answers and brief summaries of the
test questions and missed PCAs. In addition, the supervisor's
subsequent evaluation of examinees' performance contains
significantly more detailed analysis of each dimension, including
specific descriptions of many of the missed PCAs. Consequently,
although examinees do not have full access to their exam
materials, they have sufficient access to judge whether the DOP
grading process is arbitrary.
The DOP has chosen to sacrifice a small degree of scoring
accountability in order to bolster the security, and hence the
integrity, of the entire examination process. This policy may or
may not be the wisest one. That determination, however, has been
entrusted by the Legislature to the DOP, not to the judiciary.
Our role is simply to ensure that the DOP has not acted
arbitrarily, capriciously, or unreasonably in drawing on its
expertise to strike a balance between the Legislature's dual
command that the civil-service examination process be secure and
that it allow for examinees to appeal their scores. The DOP has
exercised its delegated authority and has struck a balance that
accommodates both of these mandates. Whether we agree or
disagree with that balance, we are persuaded that it has a basis
in reason and, hence, is not arbitrary.
We also believe that the separate opinion understates the
role that reviewing courts would have to assume in routinely
reviewing in camera "the examinee's answers, the correct answers,
and a sufficient explanation of the grading standards to enable
the . . . court to make a preliminary assessment of arbitrariness
and determine whether further disclosure to the examinee, an
evidentiary hearing or other relief may be appropriate." Post at
__ (slip op. at 2). Although that level of judicial review would
not be quite as expansive as that endorsed by the Appellate
Division, it nevertheless would require routine judicial second-guessing of the DOP's grading decisions, thus conflicting with
our long jurisprudential tradition of not reviewing the accuracy
of individual grades without a prima facie showing by the
examinee that the process was arbitrary and constituted a clear
abuse of discretion. The DOP's procedures allow examinees to
make such a showing, but plaintiff simply has failed to do so.
Plaintiff also contends that he was treated unfairly and denied procedural fairness in the administrative review of his examination results. The supervisor in the DOP's Selection Appeals Unit who responded to plaintiff's concerns about his score advised him that "the [Merit System] Board will only consider the proofs, arguments and issues presented at the
previous level of appeal. No new or additional proofs, arguments
or issues will be considered at the next level of appeal."
Plaintiff claims to have followed that advice by not advancing
any new arguments, most notably his contention that the
supervisor should not have relied on undisclosed evidence,
namely, the PCA scoring key. The Board, in dismissing
plaintiff's appeal, noted that he had not submitted any
"arguments, submissions or issues in support of his appeal," but
instead had "relie[d] on proofs and arguments presented below."
The Board thus denied the appeal based on the supervisor's having
adequately addressed all of plaintiff's contentions. Plaintiff
now argues that the supervisor's statement deprived him of a real
opportunity to appeal his score because, in reliance on that
statement, he did not raise before the Board the issue of the
supervisor's use of undisclosed information in rejecting his
appeal.
We are unable to discern a specific legal basis for the
supervisor's assertion that plaintiff could not raise any new
arguments in his appeal to the Board.See footnote 3 N.J.A.C. 4A:4-6.4(g)1
and 2 and N.J.A.C. 4A:4-6.6(b)1 and 2 both state that "[t]he
appeal shall contain all information which was presented to the
first level, plus a copy of the decision below" and that "[t]he
Board shall decide any appeal on the written record or such other
proceeding as the Board deems appropriate." However, nowhere do
the regulations state that an appellant may not submit any new
arguments, especially new arguments that take issue with the
supervisor's decision. Even the Board implied that plaintiff
could have advanced new "arguments, submissions or issues" on
appeal by stating in its decision that he had chosen not to do so
in favor of relying "on the proofs and arguments presented
below."
We infer that the supervisor was attempting to inform
plaintiff that the Board would reach an independent determination
of the facts and legal conclusions based on the record below, cf.
Henry v. Rahway State Prison,
81 N.J. 571, 579 (1980) (holding
that Civil Service Commission had to review disciplinary
determinations de novo), and that it would follow the established
principle that an appellate tribunal generally will not consider
issues and arguments that could have been raised below but were
not. See, e.g., Nieder v. Royal Indem. Ins. Co.,
62 N.J. 229,
234 (1973) ("It is a well-settled principle that our appellate
courts will decline to consider questions or issues not properly
presented to the trial court when an opportunity for such a
presentation is available . . . .").
In any event, we need not determine whether the supervisor's
statement was misleading and unfair because it did not prejudice
plaintiff. As we have noted, plaintiff takes issue with the
statement not because it prevented him from challenging a
specific aspect of his test score, but rather because it
allegedly deprived him of the ability to challenge the
supervisor's reliance on undisclosed information. As we already
have determined, however, the DOP acted reasonably in not
disclosing that information. Supra at __-__ (slip op. at 16-20).
Consequently, the supervisor's potentially erroneous statement
was harmless and provides no basis for a reversal of the Board's
denial of plaintiff's appeal.
The judgment of the Appellate Division is reversed.
CHIEF JUSTICE PORITZ and JUSTICES POLLOCK, O'HERN, GARIBALDI
and COLEMAN join in JUSTICE HANDLER's opinion. JUSTICE STEIN
filed a separate opinion, dissenting in part and concurring in
part.
SUPREME COURT OF NEW JERSEY
A-
103 September Term 1996
JAMES T. BRADY,
Appellant-Respondent,
v.
DEPARTMENT OF PERSONNEL,
Respondent-Appellant.
STEIN, J., concurring in part and dissenting in part.
In attempting to arrive at a proper balance between the
interest of civil-service test takers in obtaining review of
their test grades and that of the Department of Personnel (DOP)
in safeguarding the confidentiality of test questions, the Court
concludes that the Appellate Division erred in holding that as a
matter of procedural fairness all persons challenging their test
scores must be provided with copies of the questions, their
answers, and the grading standards. I do not disagree with that
aspect of the Court's opinion.
However, in my view the Court overshoots the mark and tips
the balance too far toward the interests of confidentiality when
it precludes disclosure of relevant test materials to the
reviewing court, as well as to the applicant, unless the
applicant makes a prima facie showing that the test results are
arbitrary. That holding creates a prototypical Catch-22 for a
test taker seeking to challenge his or her test results: the
bare-bones access to testing materials that the DOP customarily
makes available is an insufficient basis on which to make even a
prima facie showing of arbitrariness, and absent such a showing
neither the test taker nor the reviewing court will receive
sufficient access to the test materials to be able to evaluate
whether the test was graded arbitrarily.
The Court can and should strike a fairer balance. It could
do so by requiring the DOP to furnish the reviewing court, in
camera, with the test questions, the examinee's answers, the
correct answers, and a sufficient explanation of the grading
standards to enable the reviewing court to make a preliminary
assessment of arbitrariness and determine whether further
disclosure to the examinee, an evidentiary hearing or other
relief may be appropriate.
The Court implies that the Appellate Division conceived that
its responsibility over the test grade was plenary, ante at ___
(slip op. at 15), but a fair reading of that court's opinion
demonstrates that it intended its review to be properly
deferential to the DOP, seeking only to determine whether the
grading was arbitrary. "[O]ur review function does require us,
after we have accorded the utmost deference to the Department of
Personnel and to its technical expertise, to judge whether there
is a reasonable basis for its determinations."
289 N.J. Super. 557, 563-64 (1996). With the truncated materials provided, the
Appellate Division could not perform its review function. The
Court's disposition of this appeal should assure that reviewing
courts are not unduly impeded from performing the review that is
mandated by statute.
The pertinent facts are summarized adequately in the Court's
opinion. I highlight certain facts to emphasize the difficulty
encountered by the Appellate Division in attempting to provide
even the most cursory review of respondent Brady's written
examination grade.
As noted by the Court, the examination for police captain
consisted of an oral and written examination, each component
using a grading system of one (lowest) to five (highest). Brady
received the highest possible grade on five of the six
characteristics, or dimensions, tested on the oral examination.
His oral scores were: analytic ability (4); judgment (5);
decisiveness (5); leadership ability (5); community sensitivity
(5); oral-communication skills (5).
The written examination graded the applicants on the same
dimensions as the oral examination, with the obvious exception of
oral-communication skills, and on four additional dimensions.
Brady's scores on the written examination were not outstanding,
and his grades on the very same dimensions tested in the oral
component were significantly lower. His written examination
grades were: analysis (2); judgment (3); decisiveness (4);
leadership ability (2); community sensitivity (3); ability to
delegate (3); planning and organizational ability (3); management
capability (2); written-communication skills (3).
The written portion of the examination required the
candidate to assume the role of a Captain on the police force of
the hypothetical Township of Grandview. The testing materials
included essential factual information about the Township, the
police force, and its personnel. Each examinee was required to
respond in writing to ten hypothetical situations, and the
examinees were evaluated on the basis of the various dimensions
set forth above. In grading the examinations, graders were
provided with a list of "possible courses of action" ("PCAs")
that had been compiled by a panel of senior-command personnel
from various police departments. The list of PCAs was used by
each grader to ascertain the minimum level of acceptable
responses related to each dimension. When an examinee's response
omitted reference to a PCA on the master list, that omission was
noted as a deficiency or missed opportunity, and the aggregate
number of such deficiencies was the basis for the numerical grade
awarded on each dimension tested.
Brady, dissatisfied with his score on the written portion of
the examination, exercised his statutory right of appeal.
N.J.S.A. 11A:4-1e. Pursuant to the DOP's published review
policy, Brady was afforded only limited access to the test
materials for the purpose of preparing his appeal. He was
permitted to examine his answers to each question, and a brief
summary of each question, but not the question itself. He was
not allowed to examine the PCAs, on which his grades were based.
He was permitted to review his grades on each dimension, a brief
description of his performance on each dimension, the general
instructions for the written examination, and a summary sheet
describing how his score was computed. Brady was allotted one
hour to review those materials, allowed to take notes, but not
allowed to remove or photocopy any of the materials that he
examined.
As noted by the Appellate Division, 289 N.J. Super. at 559,
Sergeant Brady initially appealed to the DOP's Supervisor of the
Selection Appeals Unit, communicating by letter dated September
15, 1993, the reasons why he believed the grader's scoring on
each of the dimensions was too low. The response, dated May 31,
1994, noted generally that "significant opportunities to
demonstrate behaviors associated with specific dimensions were
missed," that those "behaviors" were based on the PCAs compiled
by experts, that an item-by-item review was unwarranted, and that
a review of Brady's responses in comparison with the PCAs
demonstrated that his scores were accurate.
In accordance with a statement in the Supervisor's letter
that any appeal to the Merit System Board could not be based on
new or additional proofs or argument, Brady appealed to the Merit
System Board and apparently relied only on his initial letter to
the Selection Appeals Unit. The Merit System Board affirmed.
Brady then appealed to the Appellate Division, and the
record before that court was sparse. Brady's submission
consisted of his Notice of Appeal, his letter to the Selection
Appeals Unit, the Supervisor's response, and a copy of the DOP's
one-page "Examination Review Policy for the Police Promotional
Assessment Process, Fall 1992," which essentially described the
limited review rights of a candidate seeking to appeal his or her
examination score. The Merit System Board's submission consisted
of an eleven-page orientation guide, containing background
information about the Captain/Lieutenant examination, four pages
of background facts relating to the Township of Grandview and its
police department on which the written examination was based, and
Brady's one-page score sheet for both the written and oral
examinations, together with a comment sheet containing cryptic
comments about each of the graded dimensions.
Conspicuously absent from the record furnished to the
Appellate Division were the examination questions, Sergeant
Brady's answers, the PCAs to which his answers were compared for
grading, and any explanation of the grading methodology
sufficient to assure the Appellate Division that the grading was
fair, impartial, and free from arbitrariness. As the Appellate
Division observed, "[T]he Department of Personnel must provide a
sufficient explanation for its grading decisions to enable a
reviewing court, which is obligated to give due deference to the
authority, responsibility and expertise of the agency, to
determine whether the agency's decisions have a reasonable
basis." 289 N.J. Super. at 565-66.
The standard by which courts exercise their obligation to
review challenges to civil-service test results is not a matter
in dispute. The majority opinion correctly observes that courts
generally are obliged to recognize the broad regulatory authority
of the DOP and to defer to the agency's grading of an examination
absent a showing of arbitrariness. Ante at ___ (slip op. at 13-14). As this Court stated in Zicherman v. Department of Civil-service,
40 N.J. 347, 351 (1963), "the courts cannot intervene to
nullify a civil-service examination unless it is clearly shown
that the Department has abused its discretion." In Zicherman, in
which the plaintiff challenged the appropriateness of a civil-service promotional examination for the position of Clerk of the
District Court, this Court reviewed the examination questions and
the correct answers. Although observing that the "correct"
answers to two questions appeared to be erroneous and that the
relevancy of a few questions was debatable, the Court
nevertheless upheld the validity of the examination. Id. at 352.
A similarly deferential standard of review is reflected in
other reported decisions involving challenges to results of
civil-service examinations. See, e.g., Rox v. Department of
Civil Serv.,
141 N.J. Super. 463, 467-69 (App. Div. 1976)
(acknowledging deferential standard of review, but invalidating
as too subjective oral testing format for police captains and
lieutenants, noting that one of the seven grading teams awarded
consistently lower scores to examinees than did the other six
grading teams); Brotspies v. Department of Civil-Serv.,
66 N.J. Super. 492, 494-99 (App. Div. 1961) (acknowledging that
intervention to nullify civil-service examination is warranted
only where examination is corrupt, arbitrary, capricious, or
unreasonable, and concluding that questions were fair and
appropriate, and that answers were neither unreasonable nor
implausible, following hearing at which single judge reviewed
objections to questions and to Department's version of correct
answers); Artaserse v. Department of Civil Serv.,
37 N.J. Super. 98, 102-05 (App. Div. 1955) (holding that plaintiffs had failed
to prove civil-service examination for police lieutenant was
arbitrarily administered or graded where department, after
consultation with plaintiff's counsel, eliminated six of nine
challenged questions and regraded all papers, but plaintiffs
nevertheless failed to attain minimum requisite score).
The experience of other courts confronted with the
responsibility of reviewing challenges to civil-service
examinations but hampered in discharging that duty by
insufficient information concerning the test confirms the wisdom
of mandating in camera submission to the reviewing court of all
pertinent material. Like the majority, ante at ___ (slip op. at
24), I impute no precedential authority to unpublished decisions,
see R. 1:36-3, but the factual context of the two unreported
opinions referred to by the majority is illustrative of the need
to provide adequate information to a reviewing court. In
Dellaventura v. Department of Personnel, No. A-3957-91T1 (App.
Div. Mar. 11, 1993), the plaintiff appealed the Merit System
Board's decision upholding his failing grade on the written
portion of a promotional examination for Fire Lieutenant.
Plaintiff's initial score was thirty-one and the passing grade
was thirty-five. Only examinees who passed the written test were
eligible to take an oral examination.
After reviewing his test papers, plaintiff appealed his
score, relying on his memory of the questions and answers in
preparing his appeal. The initial review process resulted in no
change in his score. A second-lev