The Pros(e) and Cons of Computers Grading Essays

There was lots of coverage today about a new computer program for grading essays.  As a retired college professor who graded thousands of student papers, I have a few thoughts on this.

1.  Humans have abilities that computers still don't have.

For an easy example, let's start with CAPTCHA which is based on the premise that human brains can make simple distinctions that computers can't make:

"CAPTCHA: Telling Humans and Computers Apart Automatically

A CAPTCHA is a program that protects websites against bots by generating and grading tests that humans can pass but current computer programs cannot. For example, humans can read distorted text as the one shown below, but current computer programs can't:
The term CAPTCHA (for Completely Automated Public Turing Test To Tell Computers and Humans Apart) was coined in 2000 by Luis von Ahn, Manuel Blum, Nicholas Hopper and John Langford of Carnegie Mellon University."

Computers do a great job of spell-checking and some grammar checking.   But when it comes to evaluating the development of an argument, supporting the argument, the flow from beginning to end, and any number of other more subtle aspects of language usage, skilled humans are still significantly better.  If you need a human to read a captcha, you need a human to read a poem or an essay. 

I took the second (italicized) sentence from the paragraph above and asked Bing's translation page to turn it into Turkish and then back into English.  Here's the resulting English sentence:

"But when it comes to evaluating the development of this argument, the argument, in the end, and many other, more subtle aspects of language use startup current supporting people still significantly better."

How is a computer that can write that sentence in English going to evaluate a student's writing?  

That doesn't mean that in ten years computers won't be much better.  And I'm not saying there aren't uses for such a program.  It would be great for students to run their own essays through such a program, just as many of them use a spell-check and grammar-check.  But ultimately they need a skilled and dedicated reader to give them really good feedback. 

Would a program like this be better than a teacher who does nothing more than put a grade at the bottom of the paper?  The question then becomes about bad teachers and mediocre machines.  Again, I'd say it should be used by the student, like spell-check, not as a substitute for the teacher.  And I suspect if it's available, students will use it.  Will teachers forbid such use as cheating?  I hope not.  There will still be issues for the teacher to address. 

2.  Computers are likely to penalize non-standard and creative approaches

Computers can evaluate essays by absorbing a lot of rules on grammar, spelling, style, and perhaps even logical arguments against which they can measure the essay.  But the rules aren't enough.  In some cases there are conflicting rules or disagreements about the rules.  And the malleability of language means the rules are constantly evolving.

But even the best writers don't necessarily follow the rules, and often it's in breaking the rule that they do their best writing.   Strunk and White wrote the basic English writing style rule book, The Elements of Style.  In the introduction they write:

"It is an old observation that the best writers sometimes disregard the rules of rhetoric. When they do so, however, the reader will usually find in the sentence some compensating merit, attained at the cost of the violation. "

But will the computer see the compensating merit or just the violation?   I'm afraid that the necessarily limited boundaries of the computer program would lead student writing to get more and more homogenous. 



3.  Grading essays is feedback for teachers about what the students are learning

Perhaps my biggest objection is in response to this statement in the NY Times article:

"The software uses artificial intelligence to grade student essays and short written answers, freeing professors for other tasks."

Other tasks, like what?  Going to departmental meetings?  I found in my teaching experience that reading student papers gave me invaluable feedback on how my students were doing which reflected directly on how well I was teaching.  Reading the papers told me most of what I needed to know to improve my teaching.   Machine grading dilutes this feedback source significantly.  The job of the teacher, as I see it,  is to know each student and find ways to get the student from where he is to where he ought to be in terms of basic skills and understanding of concepts.

The argument that faculty have too many students to be able to grade the essays says a lot about how colleges have moved away from being educational institutions  to being degree factories. 

The answer isn't more machines.   I have no problem with students using such programs the way they now use spell-check. Ultimately, though,  the teachers should still grade them in the end.