Questions on California bar exam line up with lawyer jobs data, study finds
The content of the California bar exam lines up with national surveys about skills expected for first-year lawyers, but a state-specific assessment would be helpful, according to a recent content validation study from the State Bar of California.
The report, sent to the California Supreme Court on Dec. 1, the report (PDF) follows earlier state bar cut score studies and the court’s Oct. 18 announcement that the cut score would stay at 1440—for the time being—because it was not persuaded that changes should be made.
The State Bar of California’s Committee of Bar Examiners has advocated for leaving the cut score at 1440, while the bar’s Law School Council endorsed lowering the cut score to something between 1350 and 1390. Additionally, the Association of California Accredited Law Schools—a group of law schools with state accreditation but not ABA accreditation—sent a letter (PDF) to California bar leadership Sept. 1 asking that the cut score be lowered to 139.
The Dec. 1 content validation study relied on a panel of practicing attorneys to evaluate California bar exam questions and compare the findings to both a job analysis survey from the National Conference of Bar Examiners and occupational information from a U.S. Department of Labor database. The study refers to the process as a “crosswalk exercise.”
“After being mapped to subject matters and skills tested on [bar exam] questions, the ‘crosswalk’ matching result shows that all topics covered in the [bar exam] are represented on the job analysis survey results as knowledge domains and skills expected of entry-level attorneys,” the report states. “In other words, there were no subjects and skills tested on the [bar exam] that were considered unimportant for legal practice or used infrequently, according to survey results from the job analysis study conducted by NCBE.”
Two psychometricians (who study the science and practice of psychological measurement) found that other suggestions for modifying state bar content would require job analysis more closely reflecting the work of California lawyers, according to the study.
“Specifically, from the results of the national survey, skills and tasks were generally interpreted as more generalizable than many of the knowledge domains,” the report reads. “Given the diversity of subject areas in the law, this is not surprising. At the same time, it may also suggest that a greater emphasis on skills could be supported in the future. To answer this question, further study is warranted.”
A fourth state bar study about law school bar exam performance, is ongoing. Also, the state bar in its Dec. 1 letter to the court notes a need to study the relationship between the bar exam, access to justice issues and public protection, and what if anything a cut score change would mean for those things.
“While these complex and challenging policy analyses will likely take several years to complete, concurrent with the completion of the Law School Bar Exam Performance Study, the Bar will develop frameworks and timelines for these analyses,” the letter reads.
Out of 8,545 people who finished the July 2017 bar exam, the overall pass rate was 49.57 percent, a state bar spokeswoman told the ABA Journal in November. Comparatively, the overall pass rate in 2016 was 43.57 percent.
