The Pros(e) and Cons of Computers Grading Essays

There was lots of coverage today about a new computer program for grading essays.  As a retired college professor who graded thousands of student papers, I have a few thoughts on this.

1.  Humans have abilities that computers still don't have.

For an easy example, let's start with CAPTCHA which is based on the premise that human brains can make simple distinctions that computers can't make:

"CAPTCHA: Telling Humans and Computers Apart Automatically

A CAPTCHA is a program that protects websites against bots by generating and grading tests that humans can pass but current computer programs cannot. For example, humans can read distorted text as the one shown below, but current computer programs can't:
The term CAPTCHA (for Completely Automated Public Turing Test To Tell Computers and Humans Apart) was coined in 2000 by Luis von Ahn, Manuel Blum, Nicholas Hopper and John Langford of Carnegie Mellon University."

Computers do a great job of spell-checking and some grammar checking.   But when it comes to evaluating the development of an argument, supporting the argument, the flow from beginning to end, and any number of other more subtle aspects of language usage, skilled humans are still significantly better.  If you need a human to read a captcha, you need a human to read a poem or an essay. 

I took the second (italicized) sentence from the paragraph above and asked Bing's translation page to turn it into Turkish and then back into English.  Here's the resulting English sentence:

"But when it comes to evaluating the development of this argument, the argument, in the end, and many other, more subtle aspects of language use startup current supporting people still significantly better."

How is a computer that can write that sentence in English going to evaluate a student's writing?  

That doesn't mean that in ten years computers won't be much better.  And I'm not saying there aren't uses for such a program.  It would be great for students to run their own essays through such a program, just as many of them use a spell-check and grammar-check.  But ultimately they need a skilled and dedicated reader to give them really good feedback. 

Would a program like this be better than a teacher who does nothing more than put a grade at the bottom of the paper?  The question then becomes about bad teachers and mediocre machines.  Again, I'd say it should be used by the student, like spell-check, not as a substitute for the teacher.  And I suspect if it's available, students will use it.  Will teachers forbid such use as cheating?  I hope not.  There will still be issues for the teacher to address. 

2.  Computers are likely to penalize non-standard and creative approaches

Computers can evaluate essays by absorbing a lot of rules on grammar, spelling, style, and perhaps even logical arguments against which they can measure the essay.  But the rules aren't enough.  In some cases there are conflicting rules or disagreements about the rules.  And the malleability of language means the rules are constantly evolving.

But even the best writers don't necessarily follow the rules, and often it's in breaking the rule that they do their best writing.   Strunk and White wrote the basic English writing style rule book, The Elements of Style.  In the introduction they write:

"It is an old observation that the best writers sometimes disregard the rules of rhetoric. When they do so, however, the reader will usually find in the sentence some compensating merit, attained at the cost of the violation. "

But will the computer see the compensating merit or just the violation?   I'm afraid that the necessarily limited boundaries of the computer program would lead student writing to get more and more homogenous. 



3.  Grading essays is feedback for teachers about what the students are learning

Perhaps my biggest objection is in response to this statement in the NY Times article:

"The software uses artificial intelligence to grade student essays and short written answers, freeing professors for other tasks."

Other tasks, like what?  Going to departmental meetings?  I found in my teaching experience that reading student papers gave me invaluable feedback on how my students were doing which reflected directly on how well I was teaching.  Reading the papers told me most of what I needed to know to improve my teaching.   Machine grading dilutes this feedback source significantly.  The job of the teacher, as I see it,  is to know each student and find ways to get the student from where he is to where he ought to be in terms of basic skills and understanding of concepts.

The argument that faculty have too many students to be able to grade the essays says a lot about how colleges have moved away from being educational institutions  to being degree factories. 

The answer isn't more machines.   I have no problem with students using such programs the way they now use spell-check. Ultimately, though,  the teachers should still grade them in the end. 

 

Permanent Fund's Future Looking Dim - "It's Our Oil" Rally: The Movie

From a PR standpoint, probably the most effective signs at yesterday's rally to prevent Republicans from transferring billions of Alaska's oil revenues to oil companies, are the ones that predicted the demise of the Permanent Fund.  That's a cause all Alaskans will fight.   But it only makes sense - if our oil revenues plummet, so will the Permanent Fund*.  


If a journalist is watching a mugging, should she just take notes or try to intervene?  When does once humanity take precedence over one's profession?  I've heard photojournalists argue that they've been in terrible situations where intervening might not have helped, but their photos calling attention to atrocities can help in the long run.   In this post I can't quite help myself.  I don't see two valid sides to this story.  I'd like to think I'm being both a human being and a reasonable journalist here.  But I'm sure others will disagree.

Despite what Republicans may have talked themselves into believing, all their rationales are just cover for transferring billions of dollars from the State of Alaska to the oil companies.  These 'incentives' by the state are matched by absolutely no commitments on the part of the oil companies.  Remember this is brought to us by the party that includes not a few members who think global climate change is a hoax, that the world is only 6000 years old, and who take their marching orders on gay rights from Deuteronomy.   There is a reason Republicans want to gut the public schools - education hurts their election chances. 


Right now, the State of Alaska is being fleeced by Republican legislators lowering the taxes on oil companies in the vague hope that this will stimulate production.  Our Governor was a lobbyist for Conoco-Phillips before becoming Sarah Palin's Lt. Governor and then, when she quit, Governor.  Two of the Senators who voted for Senate Bill 21are currently employees of oil companies when they aren't on leave to be in the legislature.  Without those two votes, the bill would not have passed.  The Senate majority voted they didn't have a conflict of interest.  If they had been members of the Sierra Club and their votes would have stopped the bill, do you think they would have been allowed to vote?  I don't think so.

So, the legislature is emptying the Bank of Alaska and taking the loot to the getaway cars driven by the  big three oil companies.  We are like a third world nation, except that this seems to have become standard operating procedure in the US now as well.  (I don't understand how Alaskan Tea Party folks who fume over the bank bailouts aren't equally upset about their legislators actions on oil (yeah the most conservative supporters of this were elected by Tea Party supporters).

The sequester will strongly weaken government's ability to not only maintain vital infrastructure - transportation, health and safety, education, crime prevention, etc. - but also makes it much harder for the government to regulate industry.  Exactly what the Koch brothers, who have funded much of this, want.  I do have to admire their ability to get ardent support from some of the people who will be hurt most.

Facts and reason have had little effect at all against the power of the oil companies.  Six years after the FBI video taped VECO executives bribing Republican legislators over oil taxes in the Baranof Hotel's Suite 602, and with most if not all of those convicted back on the streets, they've returned to business as usual. 

In any case, here's a video that shows those who mustered 9 votes to the Republicans' 11 votes in the Senate,  on 4th Avenue yesterday in front of the Legislative Information Office.

[Update April 7, 2013: Whoops, the video wasn't there. It should be now.]

The speakers include a number of former legislators, Wally Hickel's long time aide and biographer Malcolm Roberts, two people - Katie Hurley and Vic Fischer - who were participants in the Alaska Constitutional Convention, and Anchorage Assembly write-in  candidate, Nick Moe, who's challenge to Assembly Chair Ernie Hall is still too close to call. 


*We have in recent years, gotten more income from investments than oil revenue, which is how things were originally projected when people thought the oil would be almost depleted by now.  However, it appears that last year's oil revenue was greater than our investment revenue.  The Annual Report (p. 6) shows that investment gains were less than the drop in the overall portfolio value, meaning the real gain was from oil revenue.  Oil revenue last year was $915 million, about what the Governor's bill proposes to cut each year.

Drastically reducing our oil revenue like this will keep the Permanent Fund from growing as it should.  Our $45 billion Permanent Fund is dwarfed by Norway's $700 billion oil fund which started much later and is supported by smaller oil reserves.

Political Graffiti

We walked past this sign Wednesday night.  It seems someone felt Clary's connection to the Anchorage Baptist Temple deserved some recognition on his signs.

And when I checked the spelling of 'graffiti' I found this do-it-yourself graffiti creator site.  This just popped out of my finger tips.  It just seemed right to play with their "Kodiak' style.  [UPDATE March 8, 2014:  I went back to the diy site today and I can't find the page where I created the image below.  Now it looks like you have to buy it all.  Graffiti artists need to make a living too.  But you can go to fontmeme where you can generate your own graffiti in different fonts and colors.]

From the Muni Election page:

ASSEMBLY - DISTRICT 4 -SEAT F Total Number of Precincts 25 Precincts Reporting 23 92.0% Times Counted 6178/35860 17.2% Total Votes 6055 CLARY, Andy 2506 41.39% TRAINI, Dick 3497 57.75% Write-in Votes 52 0.86%

"It's Our Oil" Rally Against Giving Billions To Oil Companies

Backbone formed back in the 1999 to stop the state from signing itself over to BP.  It's come back to life recently as the Corrupt Bastards Club reappeared in Juneau and started the process to drastically lower taxes on oil companies over again,.  Ex(?) Conoco-Phillips lobbyist Governor Sean Parnell says SB 21 is needed to get the oil companies to produce more oil.  But there is nothing in the legislation that requires anything from the oil companies.  It's just a straight transfer from the State treasury to the oil companies.  (OK, not exactly.  It's money we now get through our tax system which we won't get if this passes.)  Is this a whole new Republican philosophy?  Hmmm, maybe we should pay drunks on the hope that they become sober and crooks on the hopes it will cause them to be honest.  (Sorry, do you think I have an opinion here?  Some issues are just so black and white that pretending there are two equally valid sides is the deceptive way to report it.)

So, here are some pictures from the noon rally in downtown Anchorage at the Legislative Information Office on a sunny, but definitely wind-chilled day.  Here are some photos of the demonstration.  I'll try to get some video up later.

Poll Worker Notes

Tuesday was Municipal Elections in Anchorage  By the time elections ended at 8pm, we'd already started closing down the polling place.  Some of the Vote Here signs were brought in.  We started counting the people on the register who'd voted.  Putting furniture back the way it had been. (We took pictures the day before. :) ) There were three people who'd been working since 6:15 am and two others who came about 1pm.  We did a lot of stuff reasonably well, but there were some problems too. 





I was counting the people in the register who'd voted.  They'd been marked with a

Roster smudged to hide identifiers

highlighter so it was fairly easy to count and a subtotal on each page.  But my total was one more than the votes counted on the voting machine and 9 less than the number of ballots we'd given out.  There had been 10 questioned ballots so the voting machine and the ballot stub numbers matched, but I was one off.  That got resolved when the chair remembered that a worker had marked the wrong name in the morning and then had marked the right name.  So there was one extra bright yellow highlighted name and that had been marked.  It took us a couple more runs to find where that was marked.  So that was ok.

But then they were missing something they called the memory card envelope that was supposed to hold the computer printout tapes we'd all signed..  We tore everything open and then closed everything back up looking for it.  The chair swore it had been right there.  Someone found a clear plastic envelope/bag and proposed putting the computer tapes in there instead.  We didn't know where the bag had come from and whether it belonged to the church that was hosting the polling place or what.  But we couldn't find the one we needed so it was stuck in this new clear plastic bag with green on it.

One of the workers who was still there is older and has had back and neck surgery and was clearly tired.  (Actually, most of us were older.)  But she knew what she was doing and was acting as the co-chair.  We carried stuff to the chair's car while the chair was calculating the time sheet for all the workers.  (Yes, I had a paying job!)  There was a drop from the church floor out to the parking lot that wasn't marked and J stumbled and fell to the ground and had to rest a few minutes before she had the strength to get up with my help.  She and the chair had all the stuff in the car and drove off to City Hall to drop it off.  We were only two block from City Hall and I debated if I should walk or drive.  I drove but it turned out the parking lot was jammed and so I went back and parked near the church and walked back.

You can see a video of this in yesterday's post

The Chair and J were finally near the door and unloading everything at the end of the line of people and I helped at that point.  The Chair left to park her car and I stayed with J.  It was clear to me she was really tired and I suggested I drive her back to her car and send her home and I'd stay with the Chair.  But when the Chair got back before I could say anything, she asked if I could stay and help.  I said yes, because that's what I intended to do, but then the Chair said she had a 7am flight and would leave me and J to finish up.  J agreed over my protest and the Chair said good bye and left.

Someone brought us a cart to put the boxes and bags on and we waited in line in the chilly, but not too cold night.  Soon they were ushering the line inside the building and at least had the line where it was warmer.  They also passed out water and candy and later vegies - and assured us this was not from taxpayer dollars. 


Eventually we got to two folks who took our stuff.  They had a checklist they were working through.  The voting machine, the sealed memory card in the machine, the ballots, the questioned ballots, etc.  J was concerned about the envelope that held the printed out tapes from the voting machine.  But it turned out the plastic bag/envelope we found to put them in was actually what they were supposed to be in.  Our red bag - with the questioned ballots (we didn't have any special needs) - didn't have a zip tie, but they put one on.   I know that no one tampered with them, but still . . .  it should have been sealed. 

J and I had been there since 6:15am and it was pushing 10:30pm.  We had some questions about the time sheets for the poll workers.  Our chair had put 8 hours down as the total for everyone as ST (Straight Time, I think) and all the rest was OT (overtime.)  But that didn't make sense to me.  I thought there shouldn't be overtime unless you worked 8 hours in one day.  They told me my way of calculating was correct. 

We were finally done.  I got my car and came back to pick up J and drive her to her car.  I thought about going to election central across the street at the Dena'ina Center, but I was just too tired and went home. 

Ah. . . Breakup

That time of year when sun and snow conspire to put Anchorage underwater. 

Wild End To Generally Quiet Day At The Polling Place

I'll write tomorrow after I get a good night's sleep, but the end of the day proved a lot more hectic than I expected with a huge mob of election workers jammed up at City Hall trying to turn in the ballots, voting machines, and other odds and ends.

But here's a glimpse of the election workers waiting to get their materials checked in.  This was about 9:30pm.  Polls closed at 8.  We finally got out about 10:45pm.






I watched this function last November as a poll watcher at the State and Federal election.  They had a lot more space and had runners going out to the cars as they pulled up who carried the equipment into the room.  I also know that new Municipal Clerk Barbara Jones has worked hard to make sure this election didn't suffer the problems that happened last year.  And people were out offering help - someone gave us a cart to put everything on - and someone else was offering candy and another had bottles of water.  Given the long line, in the end we got through in about an hour and were handled very efficiently and courteously by the people inside.  And as we were leaving someone was offering vegies on a tray. 

 Doing this as a poll worker gave me a whole different look at what goes on.  After getting the ballot materials turned in, I was just too tired to go over to election central and I have no idea what the results of the election were.  I'll read the paper when i get up tomorrow.  

It's 10 am and 34 People Have Voted So Far

There are 890 people listed on the register in this precinct.  Some have gotten absentee ballots and some may have voted early.  


Most of the stuff was set up last night.  We were here at 6:15am (physically here, not necessarily mentally) and had to set up of the ballot box.  We walked through the instructions starting with opening it up and checking to see it was empty in all three chambers.  One chamber is for ballots that don't work right in the main slot and in case the power goes off.  Then there's an extra one if the main chamber gets full.  You have to open the top and put the ballots into that second chamber so there is room in the main chamber. 






Then you have to slide the Accu-Vote machine into place, plug it in, then turn it on.  Next we took the cover off of the compartment that has the tape. 










  The tape automatically starts and the window is supposed to have the precinct number and have a zero.  Then it lists a bunch of zeroes - I'm assuming one for each item on the ballot. 

Then each of the poll-workers signs the tape and it gets rolled up and the cover to this compartment is locked on.  There was a little metal bar with a metal security band locked on.   The tail of the security band kept getting caught in the cover as we tried to lock that on.  Eventually we got it right. 

We all had to take an oath to uphold the laws of the US, the State of Alaska, and the Municipality of Anchorage.

We had an early voter who was here before 7am who waited until it was time to start.  Then there was a steady flow.  By 8am we had had 11 voters, one of whom was a questioned ballot. (Someone who was from another precinct.)







We had a brief scare as someone asked about the Assembly race and why that wasn’t on the ballot.  He was right.  There were just two school board races.  No Assembly race.  But we figured out that the Assembly member from this district wasn't up for reelection.  But the voter said that his wife had voted early at Loussac Library and had an Assembly race on her ballot. 

This raised a question for me - if someone votes a questioned ballot in another polling place, could they vote for a candidate that isn’t on their own ballot in their home precinct?

We checked with , reported the situation the man told us about with the election office and checked on what happens to questioned ballots with races the voter isn't eligible to vote for.  They said that questioned ballots are hand checked for precincts and if they get a ballot with races they can’t vote for, those are voided. 

We’ve got strawberries, grapes, and muffins that the other workers brought in to munch on - and for people who voted. 

This Year I'm An Election Worker - Come Vote Today

I've been a poll watcher and last year I spent a fair amount of time following up Anchorage's screwed up election.  This year I decided to see things from the inside - I'm working as a poll worker. 

I picked up a copy of the elections handbook and read it carefully. 

There's a list of all the things you're supposed to have - though there are other things listed in the Manual - like an election binder - that isn't on this list.

There are about five troubleshooting pages.

I met with the chair of the precinct and another poll worker Monday night to set things up.  We'll see how it goes.  I'm not expecting a high turnout for this non-mayoral election, though the Assembly races will make a significant long term difference.  We could increase the mayor's majority or we could get a more liberal majority. 

So, be sure to vote. 

Why I Live Here - Cross Country Skiing Campbell Airstrip

 

 What a beautiful day.  A meeting on the east side of town was an excuse to put the skis in the car and be out in the woods for an hour.  Things were great on the main trail where the tracks were good.

 But when I got onto one of the smaller trails, the tracks were sketchy, and the middle of the trail was hard and slippery.  But I did manage to stay upright. 

Fortunately, salmon season is still a ways off.

And over the bridge and back into the world of cars and roads.