Workshop for New Law Teachers

June 21–23, 2001
Alexandria, Virginia

Back to Workshop Program

Gerry Hess, Gonzaga University School of Law, Institute for Law School Teaching

Goals

1. Articulate purposes of tests and other forms of assessment 2. Describe principles of effective assessment 3. Explore types of assessment methods appropriate for legal education 4. Generate practical suggestions for designing and delivering tests and other assessments 5. Explore methods for feedback to students 6. Review assessment resources for teachers and students

Methods
Presenting	Reading
Discussing (large and small group)	Writing
Listening	Taking a test!

Purposes of tests and other methods of assessment in legal education

Assess the importance of each purpose: Primary (P), Secondary (S), Inapplicable (I). Articulate a fifth purpose. 1. Evaluate student knowledge, skills, performance _____ 2. Motivate student learning _____ 3. Provide feedback to students and teachers _____ 4. Prepare students for the bar exam and practice _____ 5.

Principles of Effective Evaluation

Valid. An evaluation tool (test, paper) is valid if it assesses what it purports to assess. The test or paper must effectively measure student competence with respect to the course objectives.

Validity problems

• Imprecise or ambiguous call of the question
• Test presumes knowledge not taught in the course
• Scoring affected by factors unrelated to course objectives

Reliable. An evaluation tool is reliable if it measure’s students’ knowledge or skills consistently. Similar performances by different students receive similar scores.

Reliability problems

• Unrepresentative sampling of course content and skills
• Scoring inconsistency

Fair. An evaluation system is fair if:

• The process is not tainted by advantage or surprise
• The grading criteria are clear to students before the evaluation
• The students have the opportunity to practice and receive feedback.

Features that improve validity, reliability, and fairness

1. Construct the assessment tool from a checklist of course content and skills; assess the content and skills in proportion to their importance during the course. 2. An effective evaluation system uses more than one kind of assessment tool. 3. An effective evaluation system assess students on more than one occasion. 4. The directions are clear and concise. 5. The questions contain no unintended ambiguity. 6. The time limit allows all or most students to finish. 7. When grading essay exams: Conceal students’ names Read a few exams to get a feel for typical responses Grade one question at a time, in one sitting if possible Reshuffle papers before grading the next question Use a prescribed reading procedure - Key (points) or Rank (piles) 8. On multiple-choice exams, allow students to explain answers. 9. In first year courses, give a mock exam.

Types of Evaluation and Assessment Tools

Exam
Format

In class, closed book
In class, open book
Take home

Essay
Short answer
Multiple choice

Feedback to Students

Why?
Learning
Motivation
Respect
Expectations

How?
Exams (model, score sheet, student samples, class discussion, conference, comments)

Quiz (class, web site)

Paper draft (teacher comments, student peer edit with evaluation criteria)

Performance

In class - teacher
Argue, negotiate etc. - students or lawyers

Practical Advice from the Trenches*

Good Things to Do 1. Test what you teach. 2. Write your own questions. 3. Have a colleague review your exam. 4. Include comprehensive instructions and grading criteria. 5. Give students feedback on their performance.

Pitfalls to Avoid 1. Do not show how smart you are by giving an impossible exam. 2. Do not include typographical errors an unintended ambiguities. 3. Do not blow the official (or unofficial) curve. 4. Do not hand in grades late. 5. Do not change grades, except for mathematical errors or extraordinary circumstances.

* Susan Becker, Steve Friedland, Gerald Hess, Kent Syvrud, Douglas Whaley

Assessment Resources

For Teachers
Books dealing with exams and other assessment tools

• Josephson, Learning and Evaluation in Law School (American Association of Law Schools 1984) - excellent, thorough discussion of test (essay, short answer, multiple choice) design, administration, and scoring.
• Jacobs & Chase, Developing and Using Tests Effectively (Jossey-Bass Publishers 1992) - clear, comprehensive discussion of test (essay and multiple choice) design and administration; alternative forms of evaluation.
• Hess & Friedland, Techniques for Teaching Law (Carolina Academic Press 1999) - Chapter 11 “Evaluation of Students” (16 ideas on the design of tests, quizzes, and alternative evaluation methods).
• Munro, Outcomes Assessment for Law Schools (Institute for Law School Teaching 2000) - addresses all levels of assessment (institution, faculty, students); sections on “How to do Assessment” and “The Assessment-Centered Course.”

Articles addressing design and critical analysis of assessment methods in legal education

• Philip C. Kissam, Law School Examinations, 42 Vand. L. Rev. 433 (1989).
• Norman Redlich and Steve Friedland, Challenging Tradition: Using Objective Questions in Law School Exams, 41 DePaul L. Rev. 143 (1991).
• Michael S. Jacobs, Law School Examinations and Churchillian Democracy: a Reply to Professors Relich and Friedland, 41 DePaul L. Rev. 159 (1991).
• Paul T. Wangerin, “Alternative Grading in Large Section Law School Classes, 6 U. Fla. J.L. & Pub. Pol’y 53 (1993).
• Douglas A. Henderson, Uncivil Procedure: Ranking Law Students Among their Peers, 27 U. Mich. J.L. Reform 399 (1994).
• Linda R. Crane, Grading Law School Examinations: Making a Case for Objective Exams to Cure What Ails “Objectified” Exams, 34 New Eng. L. Rev.785 (2000).

For Students
How to prepare for and write effective law school exams

• Fischl & Paul, Getting to Maybe: How to Excel on Law School Exams (Carolina Academic Press 1999).
• http://jurist.law.pitt.edu/exams.htm - links to advice from seventeen law teachers.
• William R. Andersen, “Writing Better Law School Exams” (CALI exercise).

For Teachers and Students: Exam banks

• http://jurist.law.pitt.edu/exams.htm - links to full texts of exams from ten law schools.
• Exam Pro series from West Publishing.