Home
Calendar
Conference on New Ideas for Experienced Teachers
June 9–13, 2001
Calgary, Alberta, Canada
Professor Douglas R. Haddock
St. Mary’s University School of Law
In an ideal world, the exhilaration of learning would be a sufficient incentive for all students to make the most of their law school classes. In such a world, perhaps the exhausting exercise of “examination and evaluation” at the end of each semester, or even more often, would be unnecessary. But examinations and grades are a significant part of the life of law students and law professors. That grades are important to law students makes examinations a significant potential source of learning for students. For many years, I have experimented with the examination process in an effort (1) to use the energy students devote to exams, especially during the last few weeks of the semester, and (2) to make examinations a significant part of my teaching.
Although approaches similar to one or both of the experiments I discuss below have been used by other law professors, I believe they deserve serious consideration and experimentation by a wider audience. In addition to their function as a means of evaluating performance, law school examinations can provide better learning experiences than they traditionally have. In this paper I refer to the two methods with which I have experimented as “focused and creative review” and “collaborative exams.”
Focused and Creative Review for Examinations
During the past fifteen years, in almost all of my exams I have used a simple process designed to motivate students not only to review the subject matter but also to “imagine” or create pertinent legal problems. About two weeks before classes end, I distribute to students a document usually titled “Information on Facts and Law” or “Factual Information.” In it, I present a story involving a fair amount of detail with the potential for numerous legal questions and disputes relevant to the subject matter of the course. Sometimes the document also contains a variety of statements about the law, usually including a number of statutory provisions or references pertinent to the course. In the instructions that accompany this document, students are informed that most, if not all, of the examination problems will be based on and governed by the facts and legal doctrine presented in it. I encourage students to study this document, alone and in groups, and to ponder what legal questions and problems might arise from the situations presented. Students take this advice seriously and often become very engaged in the process, because they have a chance to anticipate and prepare for their real examination problems. Many of them work in groups, brainstorming at some length to anticipate potential exam problems.
I invite students to ask questions about the factual information in class. I also help them understand the issues they perceive. From numerous conversations I know that many of them begin to see the variety of potential legal problems and to understand how difficult those problems can be to resolve. In studying the information in the context of reviewing the subject, and in discussing the material with other members of the class, many students develop a better sense of the course. For many, the process becomes both the most valuable learning experience of the semester and a more focused review of the course than the traditional examination process. Their imaginative reviews often take them far beyond what I can cover in a three-hour examination.
Collaborative Exams
In recent years, I have given a number of take-home examinations in which students were allowed to collaborate. In some exams I have combined the focused and creative review with collaboration during the exam itself. On other occasions, no exam information was distributed prior to the exam period.
I have experimented with two models of collaboration. On one occasion, I set no limits on the number of students who were permitted to work together. In other exams, however, I have limited the size of the groups to no more than five students. In no case have I required collaboration. Students are allowed to work alone if they prefer to do so. I am convinced that this collaborative model has significant potential as a teaching device. Many comments from students suggest that this format enhances the learning process. On several occasions, I have asked students to fill out a survey form after the exam has been completed. Some discussion of that survey is included below.
Pros and Cons
I have come to the tentative conclusion that “focused and creative review” and collaborative exams can effectively use the students’ self-interest to help them better learn legal doctrine and analysis as well as the skills involved in resolving legal problems. I summarize here what I perceive to be some of the pros and cons of each of these methods. As for what I have referred to as “focused and creative review,” I see very little downside. Collaborative exams are more controversial and problematic, but the benefits of that process may be quite significant.
Focused and creative review. Possible problems with the process of distributing facts and/or assumptions of law a number of weeks before the exam include the following:
1. Composition of the exam is a complicated matter. The professor must turn her attention to the drafting of an examination at least several weeks before the end of the semester. The facts presented can be (and in most cases probably should be) longer and more detailed than those distributed at the beginning of a typical three-hour exam. This makes the drafting process more difficult and time-consuming.
2. Some explanation of the process is required, which takes up some class time.
3. Some students who have not prepared well during the semester might receive an advantage relative to those who have worked hard all semester, because they are given part of the exam several weeks in advance of the test. During one semester, a first-year student raised this issue with me as an objection to the method. He returned after the exam to tell me that he realized he was mistaken after he saw the problems. My experience is that this is not a problem. Imaginative students might have an advantage over those more committed to putting doctrine to memory, for they may be more likely to discern and prepare for the various problems that could be on the exam. Of course those students may have some advantage whatever exam method is used.
The benefits of the focused and creative review are, in my opinion, far more substantial than its drawbacks:
1. The complexity of the exam and the process provides advantages as well as disadvantages. It forces me to focus on the exam more carefully at an earlier point in the semester and that in turn probably helps me create a better exam than I otherwise might. I also suspect that a two-part process in drafting the exam-one involving the context (the facts and the law) and the other, coming a week or two later, creating the problems themselves-results in better problems.
2. Students often get very involved, both as individuals and in groups, in the exercise of imagining legal problems that might arise from the information they’ve been given. Before the examination itself, I freely discuss the information with students and in some of those conversations I have been surprised by their creativity and the range of potential legal problems they anticipate.
3. Having a detailed set of information for several weeks in preparation for the exam helps students understand the significance of facts in legal work. Some students seem to have trouble grasping this point, and the fact that rules of law will often change with variations in factual context. I think these points come across somewhat more clearly, at least for some students, as they work through detailed factual information for the exam, over a period of weeks, often with other students.
4. The focused and creative review may also give students a somewhat more realistic idea about what being a lawyer means. The emphasis is shifted to more careful preparation for, and analysis of legal problems, rather than a vague and sometimes undisciplined review of legal doctrine followed by a hurried writing exercise to test the students’ temporary understanding.
5. On another point related to time, some of the disadvantages of strict time constraints involved in the usual exam process are ameliorated because students have several weeks to think about and digest much of the information on which exam problems are based.
6. I try to emphasize the importance of statutory interpretation, especially in first-year courses, where students sometimes seem to develop the notions that judicial decisions are their only concern and that statutes will be interpreted by someone else. I include statutes in the information distributed several weeks before the exam, and students are forced at that point, if not before, to engage in the difficult work of reading and analyzing statutory material.
Collaborative Exams. Some of the points made above also apply to collaboration on exams. There are, I think, more substantial and difficult issues to consider with this method of examination:
1. The primary concern I have had about the use of collaboration on exams is that the process may compromise the evaluative function of exams. Grades generally represent individual performance. When four individuals turn in one paper, it is difficult to know whether every individual in the group actually “deserves” the grade earned by group effort. This criticism is the main one voiced by students, and is usually stated in terms of a free-rider problem. Although some who collaborate may receive a grade different from what their individual effort would have earned, for better or worse, most get the grade they otherwise would have received. In any event, for those who place a high premium on individual evaluation through exams, this method poses some issues.
2. As with focused and creative review, collaboration introduces complexity into the examination process. I believe that a collaborative exam must be a “take home” project. My collaborative exams have allowed students either 48 hours or 72 hours to work on the problems.
3. As with focused and creative review, more effort and time is required leading up to the exam when collaboration is permitted.
4. Based on surveys I have taken and comments I have heard, some frustrations and bad feelings among students can result from collaboration on such an important project. That’s life. (I should also note that good feelings among those who collaborate seem to be more prevalent than bad.)
5. I do not require students to collaborate on exams, but I worry that some students feel forced to collaborate because of the mis-perception that an individual is disadvantaged by this testing method. I try to make the exam one that a well-prepared individual can complete within a matter of five or six hours. (Of course the students don’t necessarily believe this, because many legal problems will expand to fill whatever time is allotted to them.)
6. Perhaps due to concerns about the evaluative function of exams, there may be pressure from peers and the institution for those who attempt to use it. I have little experience with this.
Although I have used collaboration more recently and less frequently than the focused-and-creative-review method, I am more enthusiastic about it than I expected to be. There are several reasons for my enthusiasm:
1. The primary benefit of collaboration is that it seems to help some, perhaps many, students learn the subject matter and develop the skills of practicing law more effectively than ordinary course work and traditional examinations. Evidence of this proposition has consistently surfaced in the surveys I have taken and in conversations I have had with students. One representative example concerns three first-year students who rather vigorously contended with me over the prospect of being allowed to collaborate on the exam. They were very good-natured about it, but they strongly felt that each student’s performance should be evaluated and graded on its own merits, without group work or influences. That semester I was not dissuaded from giving a collaborative exam and the three students just mentioned decided to work together. One of the three visited my office some weeks after the exam to tell me that his attitude about the process had completely turned around. He and his two colleagues had spent the exam period in a cabin somewhere in the Texas Hill Country and he described those two or three days as the most beneficial learning experience he had ever had.
2. Students learn something about the benefits and the difficulties of collaboration itself. The American law school experience tends to emphasize, perhaps to a fault, individual accomplishment and adversarial exposition. I think it is desirable that students begin to learn at an early time in their study of law that collaboration is a significant part of legal work and that it is not always a simple matter. In light of the importance of the ability to work with other people in the practice of law, incorporating collaboration with some examinations seems a sensible thing to do.
3. It is difficult to get students to see the complexity of law. Many students have commented on the benefits of hearing different points of view as they work together on the examination problems. Students are often surprised at the diverse opinions they encounter, which helps them develop a more open mind and see more issues and answers.
4. Some students have reported that the benefits of this method of examination include a more lasting understanding. One student wrote, “Because of this testing method I have noticed that I have retained more information about property than any other class I took in my first semester.”
5. Allowing students a few days, rather than a few hours, to complete the exam removes some of the time pressure associated with traditional exams.
6. A pleasant byproduct of collaborative exams for the professor, although not necessarily a good reason for such exams, is a reduction in the number of exams to grade.
Student Survey on the Collaborative Exam
As noted above, on several occasions I have asked students to participate in a survey containing a number of questions about their experience with, and attitudes about the collaborative exam. The responses of students to the survey questions have been quite consistent from one year to the next. I summarize here some of the results of such a survey with responses from almost 140 first-year Property students (two sections) who took the exam at the end of the fall semester of 2000.
One set of questions in the survey asked for students’ reactions to the opportunity to collaborate with other students on the examination at different points in time, including before the exam, during the two- or three-day examination period, and after the exam was over. Opinion varies some at these different points in time, but the initial reaction is nearly identical to the reaction after the exam. Interestingly, the number of students who were opposed or very opposed to the method was at its highest after grades were received (16 students as opposed to 11 students after the exam but before grades were received). The following simple chart summarizes the students’ reaction after they had received their grades. They were asked to fit their reaction into one of five categories: very enthusiastic, enthusiastic, undecided or neutral, opposed, or very opposed.
| Very enthusiastic | ||||||||||||||||||||||||||||||||||||||||||||||| - 46 |
| Enthusiastic | ||||||||||||||||||||||||||||||||||||||||||||||||| - 49 |
| Undecided or Neutral | ||||||||||||||||||||||||| - 25 |
| Opposed | |||||||||| - 10 |
| Very opposed | |||||| - 6 |
I also asked students to rate the exam (1) “as a teaching/learning device” and (2) “as a means of evaluating students’ work” on a scale of excellent, good, neutral, poor, or very poor. The results were as follows:
| As a learning device | As a means of evaluation | |
| Excellent | |||||||||||||||||||||||||||||||||||||||||||||| - 46 | ||||||||||||||||||||||| - 23 |
| Good | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||| - 63 | ||||||||||||||||||||||||||||||||||||||||||||||||||||||| - 55 |
| Neutral | ||||||||||||||||| - 17 | ||||||||||||||||||||||||||| - 27 |
| Poor | ||||||| - 7 | ||||||||||||||||| - 17 |
| Very poor | ||| - 3 | ||||||||||||| - 13 |
I do not have space here to include the numerous comments students made on the survey but if anyone would like to see those comments, the survey form itself, more detailed data from the survey, a sample summary of grades on a collaborative exam, or copies of exams using either of these processes, I would be happy to send these items to you by e-mail. My e-mail address is haddock@law.stmarytx.edu I will also have some of this information with me at the conference for those who are interested in more details.