Hints from AP Readers

Question: How are the essay question prompts written? Who are the people that develop the questions and determine what key terms go into them? For instance, on the last exam (2008), the terms "New South" and "Market Revolution" were used in two separate questions. Many of my colleagues and I do not use these terms nor do they appear in our textbooks. How fair is it to use terms that my students have not seen?

Recently, a number of people have raised issues regarding the free-response questions on the 2008 AP US History examination. As members of the AP US History Test Development committee, we would like to respond to these concerns by clarifying the process by which the exam is developed and scored.

College faculty and high school teachers play a critical role in developing and scoring the examination at every stage with the guidance of subject matter specialists from ETS. The College Board appoints the AP US History Development Committee, which consists of college and high school faculty from institutions nationwide. This committee writes, reviews, and approves all the free-response questions on the examination. The committee’s major concern is to ensure that every question on the AP US History exam reflects material currently taught in a college level survey course and addresses topics that are mainstream and accessible to students nationwide. In order to ensure this, every AP question undergoes a rigorous review process. The committee meets several times annually to review questions for the exam. At these meetings we consider the intent and phrasing of each question carefully to ensure clarity and accuracy. In addition, our high school faculty members on the committee provide valuable feedback as to whether the topics tested are covered in their AP class rooms and whether the

On the AP U.S. history list serve and at the reading in Louisville, some teachers have raised concerns about two free-response questions on the 2008 exam—one regarding the Market Revolution, and the other regarding the New South. When the test development committee drafted these questions, members agreed that both are important concepts, and in the mainstream of historical writing about the nineteenth-century United States. A subsequent review of the most commonly-used college textbooks confirmed that both concepts are discussed at some length.

Further, students who were unfamiliar with these concepts, and yet chose to answer these questions, were not penalized for failing to demonstrate a detailed knowledge of these specific terms. As always, the committee designed these questions as prompts, providing students with opportunities to demonstrate their understanding of the broad issues of these time periods.

In addition, the AP program utilizes a stringent scoring process to ensure that the exam is scored with fairness, accuracy, and consistency. The exam is scored by college and high school faculty at the AP reading. Prior to reading, the Chief Reader, a college faculty member, works with a group of experienced readers—“Exam Leaders” and “Question Leaders” comprised of high school and college teachers of U.S. history—to develop scoring guidelines for each question. The Chief Reader, Exam Leaders, Question Leaders and Table Leaders review these scoring guidelines and test them by applying them to samplings of thousands of actual student responses. The guidelines are then revised and adjusted, if necessary, to reflect the full range of actual student responses that readers are likely to encounter. These steps ensure that the scoring rubrics fit the responses that students produce; students are not penalized for choosing a question that may seem easier or more difficult, more mainstream or more obscure, than another.

Both the Market Revolution and the New South questions performed well statistically. They produced average scores very comparable to those for other free-response questions in 2008 and in previous years, and our readers encountered the usual range of essays that we see on an AP U.S. History exam.

Question: I have a number of students who, at this stage in their writing, tend to ask questions in the middle of a paragraph hoping it will prove their point. I'm wondering: what is the stance of the College Board on this technique? Does it drive anyone else as crazy as it drives me?

Asking questions does drive me crazy as well. I have attended the reading and they are not specifically penalized but it surely does not score well with the readers. It is viewed as very limited analysis and not extremely sophisticated writing.

I agree with you. My students know I consider asking questions in essays too "chatty" and discourage it, even if the student comes up with a good "answer." Some English teacher somewhere taught them that (disrespect intended). They simply wasted a line in their 35 minute essay. It is however, an excellent technique for debate or Socratic dialogue.

Question: Is there any difference between how AP essays are scored and how my essays or papers are graded in my AP class? What are AP scorers looking for that might help my essay responses stand out?

Response: Perhaps the most visible difference between reading AP Exams and grading midterm and final exams in a college-level survey course is the difference between "scoring" and "grading." The distinction is slight but worth noting. The AP U.S. History Exam rubric of scoring essays on a nine-point scale allows for less flexibility than when grading student exams in a typical class setting. The standardization of the AP Exam is also, for good reason, very much a "public" exercise.

The early stages of a scoring session consist of a group of Readers establishing clear standards. Both a table leader and the entire group of Readers at a particular table monitor these standards throughout the week. Students can rest assured that the scoring process consists of much more than a single Reader making arbitrary decisions about a particular essay. Rather, each Reader is part of a team that constantly reviews, refines, and clarifies scoring standards.

That said, the similarities between my college grading and scoring the AP Exam are notable. Of the several criteria on which essays are scored, establishing a clear thesis and developing it throughout an essay present perhaps the greatest challenge to AP students -- and this is true for college students as well. A student who effectively develops a thesis is, from the outset, ahead of the game. We tend to teach students (in both college and high school) that a thesis statement should usually occur in an introductory paragraph, but this structure often results in a weak thesis and an essay lacking cohesion and clarity. Many students establish a thesis by restating the essay question in the opening paragraph, then launch into a narrative that may touch on the thesis at times but often strays from the major point of the piece.

Not surprisingly, students often see AP essays as a process of "dropping" as many names, events, or other historical facts as possible -- often leading to a laundry list of historical details. The art of developing a thesis throughout a piece is one that requires a great deal of practice; students would do well to work on this skill throughout their preparation for the exam -- perhaps as much as preparing content for the essays.

Details, Details...
Of course, these are historical essays, and details are critical to support a thesis. My experience has been that college students taking a midterm exam are at an advantage here, because a midterm usually covers between five and eight weeks of course material, rather than an entire year's worth of study. Students who obtain high scores on the AP U.S. History Exam provide evidence to support general points, and the best ones do so impressively.

Interestingly, evidence is usually a bit less of a problem on free-response questions than on document-based questions, at least in my experience. The reason may be that having at their disposal the fragments of evidence offered in a DBQ provides students with a sense that their essays are indeed detailed and evidentiary. The requirement of "outside information" is crucial for a DBQ. Even some Readers can be misled by an essay that effectively weaves the included documents together in a well-organized piece. However, as most AP teachers are quite aware, high scores on a DBQ depend on a student's ability to bring historical details and evidence to bear on a group of documents, in order to clarify, bolster, and refine the information provided in the documents. Again, these skills develop through practice, as the highest-scoring essays indicate an ability to blend details from the documents with details provided by the student.

The Role of Historiography
Finally, the question of historiography deserves consideration. High school exam-takers, like first-year college students, often have a great deal of difficulty blending historical details with particular approaches to historical study. Occasionally, a student opens or closes an essay with a quotation or general idea from an individual historian -- most often to illustrate a single point. This understandable desire to "name drop" usually has little impact on the overall score.

Although the AP U.S. History Exam does not intend to test various approaches to history, some of the very best essays do indicate sensitivity to historiography by speaking to the strengths and limitations of various types of historical evidence. In the DBQs, for example, an essay might not only explain the historical documents provided as part of a larger narrative, but also offer some analysis as to the advantages and disadvantages of understanding a historical issue or period using a particular type of document.

Overall, students taking the AP Exam resemble quite closely college students in my first-year history class, especially with respect to these issues of essay structure, use of evidence, and analytical sophistication. If anything, my experiences with the AP Exam have convinced me of the need for more interaction between high school teachers and college instructors -- a dialogue that proves rewarding for both groups and, most importantly, their students.

Jeffrey W. Coker is an assistant professor of history at Belmont University in Nashville, Tennessee and an AP reader since 2000.

Question: Is there an expectation that students will reference all documents in the DBQ? Should students quote parts rather than summarizing briefly in their own words / referencing each document?

Response: 1) There is no expectation that every document will be used in the DBQ response. 2) Quoting documents is a waste of time. The readers know what each document contains; all a quote does is use up space. Analysis of the documents is what sets apart the good/great essays from the mediocre.

Question: Is it OK for students to write in the margins and put arrows and asterisks to show where their line of thought really goes in an essay? I looked over the sample handwritten essays and NONE of these showed this level of re-writing going on. Sure, there were LOTS of one-word cross-outs, but NO extra sentences squeezed sideways in the margins with arrows showing where to insert this new idea. Any thoughts? Do kids ever do this on the test itself? I know that points are not taken off for pretty sketchy handwriting, but do really messy papers overall drive readers nuts—and possibly risk making a bad impression or getting a lower score?

As a reader since 2001, I have seen numerous essays with additions "arrowed" in to various sections of the essays. While it certainly doesn't make it any easier to read the essay, I've always given them a full and fair read.

Understand that readers are instructed to consider the essay they rank as "rough drafts," but that being said, anything that makes the legibility of the essay difficult will detract from the overall grade. If the reader cannot see the message, the message is lost. Squeezed-in sentences and portions of the essay in the margin along with swinging arrows definitely detract from the message.

Students have large amounts of space on the essay questionsbooklets to work out the kinds of re-writes you mention and you might want to direct them that way.

On a timed essay exam if a student spends too much time"perfecting" the phrasing of their essay, so yes, going with the first edition is probably best.

As a reader, mark-throughs are distracting, but I make every effort to give those students a good read just like the neater ones. Readers aren't supposed to hold messy writing against students. Are we human? Yes. Usually, if I'm having trouble with a messy essay, I'll let someone else at my table take a go at it and give me their opinion. I tell my messy students (usually male) that they need to figure out some way to be a little neater or neat in their mark-thrus. You don't want your students to have a reader that is having to battle a confusing essay AND handwriting.

Yes, students can cross-out words, sentences, and even whole paragraphs and "insert" new ones by writing in the margins or elsewhere and then drawing an arrow to indicate where the "new" paragraph should be inserted. The CB and ETS understand that this is a timed first draft of an essay and expect to find things like this. What I would encourage your students to do is not totally black out the word/sentence/paragraph being omitted because if the judge can still read it then he/she will read it and if by chance it helps the student out then it might gain a higher grade. On the reverse, since it has been "crossed out" if there is any incorrect information in that section it would not be held against the student's score since it was obviously "omitted". Draw a single line through the "removed" word/sentence or an X over a removed paragraph.

As a reader here is my one major suggestion. Tell them to skip lines when they write. They will have enough space in the essay booklet to complete all 3 essays (unless they have large handwriting) and then there is a place to make corrections/add info. This is much easier for the reader to follow the flow of the writing than to follow arrows and read tiny writing between lines and in the margins.

Messy handwriting is an issue but we are trained as readers to not let it effect the score. If I have to read an essay several times to understand it, I will. Our table leaders are also very experienced and I have found that if I just cannot read an essay, the table leader or someone else at the table can decode it.

Question: We wondered if there was a ballpark number (understanding so many variables play in) as to the number of MC questions a student needs to answer correctly in order to get a 5 on the AP exam? Along those lines are there distribution reports that show how many were answered correctly for students who got 4s and 3s.

Response: First of all your question implies a number of misconceptions. No one needs to get a 5 to pass. Second, there is a high correlation between the scores on the essays and the M/C scores. Third the essay readers do not know how well a student did on their m/c questions. Fourth, there is a built in penalty for guessing on m/c questions; therefore, if a student does not know an answer or cannot get it down to three choices it is best to leave the question BLANK. Yes, there are distribution reports on the relative grades students have on each of the 80 m/c questions. But neither you nor your student will find what a particular student did. All he/she gets is a score of 0 to 5.