What Is The Difference Between Kosher for Passover and Regular Kosher Matzah? -The Human Capacity To Disagree

Over the years philosophers, psychologists, and other observers of the human condition have identified what they thought made humans distinct from other species.  A BBC exploration of that question begins with with Aristotle:

 We are "rational animals" pursuing knowledge for its own sake. We live by art and reasoning, he wrote. 

I'm not going to claim that disagreement makes us unique, but it sure seems be be common.  I probably don't have to give you any examples.  You probably can come up with some that occurred in the last hour.

But since tonight begins Passover, here's an example that makes my point.  Not just that people disagree over things that clearly have consequences (who to vote for, what to eat for dinner), but also things that seem to be disagreement for disagreement's sake.

Since our daughter and granddaughter are visiting next week, and since it's Passover, we can't make a bread together.  So I thought we could make matzah instead.  Matzah boxes are marked
"Kosher - not for Passover"  and "Kosher for Passover."

So I wanted to know what the difference was.  It's mostly about how carefully the wheat is treated from the time it is ground to flour.

On keeping Passover Matzah kosher:

"Most authorities maintain that it is sufficient to guard the wheat from the time it is ground, in order to use it to fulfill the mitzvah of eating matzah. Some authorities dispute this, however, and maintain that the wheat must be guarded from the time that it is harvested." 

In my role as a mediator at times, I came to learn that such disagreements - that seem to be about differences that don't really make a difference - are based on unspoken assumptions or issues that are the real problem.  It's not the purity of the grain, so to speak, but some other value it represents.

Perhaps it's just about who is right or who has the power to make the decision.  Or it could be that the stricter interpretation reflects a generally greater concern for detail by that position's advocate.

I would argue that the Jews in Egypt who first baked the unleavened bread before their exodus from Egypt, did so because they didn't have time to wait for bread to rise a few times before they had to leave.  And I'm sure they didn't use special flour that had been carefully guarded.

The use of the matzah today is symbolic.  It's to remind Jews of the suffering of their ancestors and to remind them that they too were strangers in a strange land and had to flee.  And thus Jews should remember to help others today who have to flee their homelands.  So whether we use extra special wheat to make matzah probably really makes no difference.  No one eating matzah can tell the difference.  It's the symbol that matters.  And if Jews eat made from the most vigorously guarded wheat, but forget how to apply the lessons of the story to those suffering a similar fate today, they've gotten so tied up in the rules, they've missed the whole point.


For those who are unfamiliar with the Kosher labels of food packaging, here's a website that looks at the Kosher labels and the labeling authorities.

If Legislative Coalitions Require Members To Vote For The Budget In Exchange For Benefits, Is That Bribery?

Alaska state representative  David Eastman from the Mat-Su, in a commentary in today's newspaper, raises an interesting question.  He describes when legislators join the Republican controlled 'fraternity' (the majority caucus) in Juneau, they get a bunch of perks - bigger office, more staff, prime committee membership and possible committee chairs, access to better state travel money and

 "you are invited to be 'at the table' at those closed-door meetings that never take place (at least officially.)"

But all those things don't come without a string attached.

"The cost for joining the fraternity is simple: a promise to vote with the fraternity when called upon to do so, and to approve the state budget endorsed by the fraternity — no matter what's in it."

 So, a group of legislators that forms a majority caucus, sets up rules that give benefits to legislators in exchange for votes.  That's what Eastman is complaining about.

If I gave a legislator $500 for a plane trip on the condition that he vote a certain way on a particular bill, that would be clearly illegal.

Here's from the Alaska Statutes:

"Alaska Statutes.
Title 11. Criminal Law
Chapter 56. Offenses Against Public Administration
Section 100. Bribery.
previous: Chapter 56. Offenses Against Public Administration
next: Section 110. Receiving a Bribe.
AS 11.56.100. Bribery.
(a) A person commits the crime of bribery if the person confers, offers to confer, or agrees to confer a benefit upon a public servant with the intent to influence the public servant's vote, opinion, judgment, action, decision, or exercise of official discretion.
(b) In a prosecution under this section, it is not a defense that the person sought to be influenced was not qualified to act in the desired way, whether because that person had not assumed office, lacked jurisdiction, or for any other reason.
(c) Bribery is a class B felony."  [emphasis added]

Clearly, when someone joins a Republican majority coalition, as Easton has described it, that coalition "confers, offers to confer, or agrees to confer a benefit upon a public servant with the intent to influence the public servant's vote, opinion, judgment, action, decision, or exercise of official discretion."


The benefits are all those things Easton describes:

• larger office
• better committee assignments including chairs
• more staff
• better travel benefits
• and access to private meetings where key decisions are made

In exchange, the legislator must vote as the coalition dictates on key bills including the budget.  As he describes it, their exercise of official discretion is removed.


I recall when  Rep. Lora Reinbold was kicked out of the Republican coalition in 2015, she was stripped of her committee assignments and her office etc.  She was kicked out because she didn't vote for the budget the coalition had put together.  I wrote a long post then (March 30, 2015) exploring the logic and reasoning and ethics of such rules.  But I didn't talk about it being a form of bribery.  But the way Easton talks about, it certainly seems to fit.


Here's a bit of what I wrote then.  The first quote confirms Easton's allegation.

"ADN Saturday March 28, 2015:

“All I can say is, she knew what she was doing, she knew what the rules were, and chose to go the way she did. There are consequences,” [House Speaker Mike Chenault] said."

Then I called my legislator's office and was told that wasn't how the Minority (Democratic) caucus worked.  They had no rules.  He suggested I call someone from the Majority (Republican) caucus.

"I check with speaker Chenault's office. 
A male staffer answered.  I explained my query and asked where I could get a copy of the rules.   
They're unwritten rules, he told me, that the caucus has.  There is no written set of rules.  They're understood.  The main one is to vote for the budget.  If you don't, things can happen.   I asked how anyone finds out about the rules?   They're told in the caucus he said."

The 2015 post got into questions about written and unwritten rules.  I wondered whether the fact that rules were unwritten suggested they knew there was something shady about them.  But on further reflection, spurred on by Rep. Eastman, I think it's a pretty clear case of bribery.

Now, I'm also sure that the Republicans have somewhere exempted internal wrangling from being interpreted as bribery.  And it's clear that log-rolling and 'if you scratch my back, I'll scratch yours" are long accepted practices in legislatures.  

Log-rolling is part of the process of getting work done in legislative bodies made up of many individuals with different agendas.  It's how you compromise.  The legislator may get a bill he badly wants passed, but he's not getting personal benefits.  But what Eastman so clearly describes is an attempt by party leaders to force their members to bow to their will in exchange for a bunch of benefits.

I'll try to check on if and where this practice is exempted from the Statute on bribery.  If I find out, I'll post again with some options for how one could end the exemptions.  If I can't find such exemptions, I think it would be time to prosecute the leaders of the Republican coalitions for bribery.

Emerging As The Snow Retreats [Updated}

We have snow and ice and various combinations of snow packed denser from the weight of more snow.  But the sun and wind are busy melting and evaporating the snow and ice.  My daily exercise has been to chop the ice that formed on the sidewalk and street and to liberate the gutter so the melt water can make a straight path to the sewer and not melt each day and freeze again each night in the street.

Here are some pictures as the disappearing ice and snow reveal what they've been hiding during the winter.

Here's part of the rock flower bed border poking out today.

Under the trees the snow wasn't so deep and so that ground makes an early appearance.  Some of the leaves fell after the first snow falls and have melted their own shapes into the older snow.

And here's some phlox poking out, getting ready to display their tiny pink blossoms near the beginning of the parade of flowers starting in May.

I thought, wow, a moose was just by and I missed it.  But there were no holes in the snow that moose leave, so this must have been left here many snows ago, and buried until yesterday.

[UPDATE March 28, 2018 - What the snow giveth, it can take away again.  I meant to note in this post yesterday that it's still March and there tends to be a snow or two in April still, even May.  And last night it snowed ever so slightly.  But enough to cover up the moose poop:

But the sun's out again and most of last night's snow is gone already.  This time of year we have lots of light and the knowledge things will turn green again soon.]

And the magpies have also been eager to poke around in the newly revealed leaves.  This is a great time of year as the (northern) earth shrugs off its coat and all the life that's been hibernating underneath begins to awaken.

Afternoon Trip To Alaska

I picked up my car Friday from the repair place, but they hadn't quite figured out the problem, but it did need to be driven, so if I took it for the weekend and drove it, that would help them.  I'd taken it in Wednesday evening and they were supposed to fix it Thursday. The key issue was that the yellow trouble light was on.  But Thursday is when the truck hit the bridge near Eagle River and shut down the Glenn Highway.  And the mechanic assigned to my car was stuck in the giant traffic jam.  Or so they told me.  No problem, I don't use the car much anyway.

An old gag about Anchorage is that "it's alone about 20 minutes from Alaska."  So, since the car needed to be driven, I wanted to head down the Seward Highway, maybe even take a short hike if the snow and ice weren't too bad.

The trail turned out to be mostly ice at the steepest part - just past the bridge - and even though we had grippers and poles, we decided it wasn't worth it.  Our grippers work fine on flat ground, but not up a steep incline.  But here are some pictures.



Turnagain Arm from McHugh Creek.














Here's a closer picture of the ice floating on the tide.

Here's McHugh Creek from the bridge just before the icy incline.  



















And here are those icicles closer up.





















Another part of the creek.












And the moon was out too.



Oh yeah, the yellow trouble light went back on in the van.  Who knows when they're going to figure out what's setting it off.

Senator Dan Sullivan On The Checks That Exist On Trump's Possible Firing of Mueller

Just seven days ago I wrote here that people should contact their Congress members and Senators to let them know they should NOT let Trump get away with firing Mueller.  I also sent emails to my representative and senators.

Only four days later I got a reply from Senator Dan Sullivan.  While he doesn't make any commitments, it looks like his staff was prepared on this one.  So I think it's only fair to post his response.

"Thank you for contacting me regarding S.1741, the Special Counsel Integrity Act. I appreciate your thoughts on this issue and welcome the opportunity to respond."

I know this is a form letter from the first line, because I didn't mention S.1741.  I just urged him to make sure  1) Trump knows that Sen. Sullivan supports the Mueller investigation and  2) should Mueller get fired that Sullivan would work to stop Trump.  I didn't offer how, I figured he knew those details better than I did, and he did.

On August 3, 2017, Senator Thom Tillis (R-NC) introduced S.1741 to formally establish a process by which the special counsel can challenge their removal. This legislation would allow a special counsel to challenge their removal before a panel of three federal court judges whose decision could be appealed to the U.S. Supreme Court. 

Since President Donald Trump assumed office on January 20, 2017, several news outlets have published allegations that staffers for President Trump had contact with Russian officials during the 2016 presidential campaign. Following these allegations, on May 17, 2017, Deputy Attorney General Rod J. Rosenstein appointed former FBI Director Robert S. Mueller as special counsel. The special counsel is authorized to investigate any possible coordination between the Russian government and the campaign of President Donald Trump, as well as any other matters that arose from this investigation.  

The President does not have the authority to remove Robert Mueller as special counsel; his removal would require the approval of Deputy Attorney General, Rod Rosenstein. The special counsel can only be fired for misconduct, dereliction of duty, incapacity, conflict of interest, or for other good cause, including violation of DOJ policies. Additionally, the special counsel must be notified in writing as to the reason for their dismissal.  [emphasis added]

We need to continue the comprehensive investigations being conducted by both the DOJ, FBI and the Senate Intelligence Committee into Russia’s involvement in the 2016 election. As a firm advocate for maintaining proper checks and balances in our government, I will continue to work with and at the same time provide necessary oversight regarding the President’s nominees to ensure that all appointees follow the U.S. Constitution and the rule of law, and that foreign governments do not succeed in their efforts to undermine trust in the U.S. Government—a model of freedom that stands in stark contrast to Russia’s authoritarian regime. [emphasis added, and I'd note that 'both' usually refers to two things not three.]

S.1741 has been referred to the Senate Committee on the Judiciary, where it awaits action. While I am not a member of this Committee, I will be sure to keep your comments in mind as it is discussed in the Senate. 

Thank you again for contacting me on this issue. If you have any more questions or concerns, please feel free to contact me or my staff. My office can be reached at 202-224-3004, or online at www.sullivan.senate.gov.

Sincerely,

Dan Sullivan
United States Senator 

So here is the official summary of S.1741:

Special Counsel Integrity Act
This bill sets forth requirements and limitations with respect to the discipline or removal from office of a special counsel appointed under Department of Justice (DOJ) regulations.
Specifically, a special counsel may be disciplined or removed only by the personal action of an Attorney General who has been confirmed by the Senate. If the Attorney General is recused from the matter, then a special counsel may be disciplined or removed by the most senior DOJ official who has been confirmed by the Senate and is not recused from the matter.
A special counsel: (1) may only be removed for misconduct, dereliction of duty, incapacity, conflict of interest, or other good cause, including violation of DOJ policies; (2) must be informed in writing of the specific reason for the removal; and (3) may file an action for judicial review of the removal.

It was introduced, as the letter says, on August 3, 2017 by Senator Them Tillis (R-NC) and referred to the Judiciary Committee.  You have to dig around a bit to find out that Sen. Christopher Coons (D - Del) is the co-sponsor.   It sat there until September 26, where a hearing was held.

That's all that Congress.com tells us.  Presumably it's been sleeping in that committee ever since, so my Senator says it's out of his hands.

I appreciate that Sen. Sullivan got back to me quickly and gave me some substantive information, but I don't seem much enthusiasm on his part for doing anything.  Though he did vote No on the budget just about the same time. And he does want the Mueller investigation to continue.  What exactly will he do to make sure that happens?



Here's what Vox said about this bill and another similar one back in October and it seems to apply to Sullivan's response:

"Senate Republicans have been coy about whether they would move to insulate Mueller if Trump follows the conservative calls to dismiss the special counsel’s investigation.
But if they’re interested in taking action, they have options already on the table."

If you poke around long enough you start to get stuff.  Here's part of what CT Mirror said on March 19, 2018:

"After Mueller was appointed nearly a year ago to head the investigation into possible ties between the Trump administration and Russia, senators, including Richard Blumenthal, introduced two bipartisan bills last year aimed at protecting a special counsel from political pressure from the White House
The “Special Counsel Integrity Act,” introduced by Republican Sen. Thom Tillis of North Carolina and Democratic Sen. Chris Coons of Delaware, would allow a special counsel to be fired only for “misconduct, dereliction of duty, incapacity, conflict of interest, or other good cause, including violation of Justice Department policies.
The “Special Counsel Independence Protection Act,” introduced by Sen. Lindsey Graham, R-S.C.; Cory Booker, D-N.J.; and Blumenthal would require a federal judge to first sign off on any action to discipline or fire a special counsel.
The Senate Judiciary Committee held a hearing in September on the two bills, but the legislation has not moved since then. Similar legislation introduced in the U.S. House is also stalled.
When the Senate returns on Tuesday, a push to move the bills out of committee onto the Senate floor is expected."

Let's see if  either of these bills get any new co-sponsors, especially now that Bolton is officially in the administration.

I Have A Potential Shooter/Terrorist/Bomber To Report . . .

After every shooting and bombing, well, any event where one 'mentally ill' person manages to kill three or more people, we hear the refrain "everyone should have seen the signs before it came to this.  It's a failure of the system.  Guns aren't the problem, it's mental illness."

Well I'd like to report a guy who thinks he's president of the United States, which he believes, allows him to do whatever he wants.  He hangs out with some truly strange people who urge him to do really bizarre stuff,  but they tend not to stay for long.  If they disagree with him on anything, he kicks them out.

He tweets threatening messages to people who disagree with him, whether they are members of Congress, judges, journalists, FBI, football players, former sex partners,  ordinary citizens, or leaders of other countries.

And for him the truth is whatever he says it is at any given moment.

He has a suitcase, I'm told with which he can blow up the world.  It has instructions on how to set off nuclear weapons. (One positive is that he doesn't like getting instructions from anyone.)  He's got this huge cache of weapons and he's recently gotten approval for an additional nearly $700 billion in weapons and other war supplies.  I think there should be a two year waiting period before he gets any of that.

I doubt there is any other single individual with more resources to do harm to the world, whether though actual arms and weapons or through destruction of the environment, civilization and community, and the good will that humans have been able to develop over the years.  And his behavior is erratic and unpredictable.  (I don't even think that's completely true - his specific actions may be, but the underlying motivation seems to be satisfying his own need for the moment.).  He's unpredictable and decidedly misogynist.

So rather than be sorry, I think it's my duty as an American citizen and as a human being to report him.  He needs serious psychological review.  I

f Republicans in Congress are serious about stopping violence through catching the mentally ill who show signs they might commit acts of violence, it's time to review this man's stability, particularly given the stash of weapons he's collecting.

I predict a serious act of violence if he is allowed continued free reign (pun intended.)  And then everyone left will be wringing their hands either saying, "who could have known" or "I always thought . . ."

Tried Out My Bike Today - Conditions Were Better Than Expected And It Felt Great

It was brilliantly sunny, though the temps are back closer to normal.  Warm enough in the afternoon that there was water from melting snow and ice.  I wasn't sure how far the conditions would be ice-free and deep-puddle free, but I thought I go exploring.

Soon I found myself out on Dowling looking at the Chugach.

Then coming back on Elmore, I had a view from the first bridge of the dog mushing trail.

A little further and I was over the southern fork (I think) of Campbell Creek.

The next bridge gave me a view of my second moose since we got back to Anchorage last week.

As I said, for the most part the trail was ice free.  There'd been one spot where a thick chunk was floating over a puddle, but there was a bit of room to go around it.  But then, almost home, I got to this hard packed ice near Providence.

I know, there are people who ride all winter and deal with this sort of thing all the time.  But I was away a lot of the winter and didn't get studs for my tires.  And a couple of years ago around this time I found myself flat on my face, hard, after hitting a small patch of ice I didn't even see.  So I navigate this stuff carefully.

It wasn't a long ride - maybe three miles - and I didn't go fast - I have a back fender but not a front one and didn't want to get too wet.  But it felt great.  And I'm guessing those hilly rides on Bainbridge made a difference.  The much more modest hills today seemed like nothing.

Then it was back to filling out tax information.  Really, I'd vote for a candidate who pushed for a much simplified tax code, one that didn't require people to hire someone to do their taxes for them.  A progressive tax with no deductions - except maybe for folks on the poor end who'd had a catastrophic event.  The tax rate could be modest then because people would actually pay that rate, especially those with much higher incomes.

I have to admit that these pictures make it look like I live in the wilderness.  We have lots of urban wilderness of sorts, but I didn't take pictures of the more urban parts I passed.  Maybe next time.  I even passed a Walgreens and a YMCA.

One Reason Some Alaska Airlines Flights Are Priced Well Is Jet Blue - Whose Flight Attendants Began Voting On a Union This Week

Alaskans have benefited from Jet Blue flights to Anchorage, because Alaska Airlines responds with lower (than their normal) fares.  One reason, people say, Jet Blue can fly for less is that their employees don't get paid as much because they aren't unionized.

But the flight attendants vote to unionize began Monday this week and goes through April 17.

Forbes seems to have the most coverage.  Here's their latest.

And the pilots unionized four years ago.  And Alaska seems to be able to match Jet Blue fares when they have to.  So maybe unions aren't the problem.

 My personal experience is that some union officials can be as stubborn and power hungry as some management officials.  But just as there are decent managers, there are plenty of decent union folks and their job is to work for the workers.  It's not just about wages, it's also about benefits, and having people on your side when management treats you badly.

But it doesn't always work - Jeff Graham's union didn't help him.

Settling Back In At Home

Chopping ice, the violin concert, taxes, sorting old mail, getting the car tuned, arranging some home repairs, have all kept us busy.  We only made it to the Thai Kitchen tonight for the first time, and only now saw our first moose since we got back last week.  But the light has been spectacular.

Here's a view from in front of our house last night as the sun was setting behind me.  I never get tired of this.  

And walking through the Helen Louise McDowell sanctuary on the way back from dropping off my car.  

Anchorage's first mail only election ends April 3.  Waiting for my granddaughter to arrive so she can help me fill in the ballot and take it to one of the drop off centers (an option if you don't want to mail it.)  At five she's a sponge, absorbing everything she touches, so I'm trying to give her as much to experience as I can.  Need to find some little cross-country skis cause that's what she was most enthusiastic about when I asked what she wants to do here.  

Graham v MOA #10: The Exams #3: The Oral Technical Exam

[All of the posts in this series are indexed at the "Graham v MOA' tab under the blog header above.  Or you can just click here.]

The details are complicated so here's an overview of some key problems; details below.

• Overall, oral exams (including interviews) are highly invalid and unreliable predictors of success on the job, despite the fact that they are used a lot.
• Imagine  a question about how to do something fairly complex like, "tell me what you know about brakes."  Where do you start?  You have four minutes.  Go.  
• The graders may or may not understand the answer, but they have four or five pages taken from a textbook,  they can scan through, with nothing to indicate what's important and what isn't.  
• The grading scale has uneven intervals - 5, 4.5, 3.5, 2.5, 1.5, 0 - with 3.5 being the lowest possible pass.  This odd scale is not what people are used to and skews the grades downward.
• You have to answer each question, they have to assess your answer, all in about four minutes.
• Lack of Accountability.  There's no recording, no way to check to see what a candidate actually said.  

That should give you an idea of the kinds of problems that exist with the oral technical exam that Jeff Graham took.

The Details

As I mentioned in an earlier post, firefighters first have to pass a written exam on technical issues.  Questions come from a national bank of questions that have been shown to be relevant to being an engineer (the next step above fire fighter).

Then comes the the practical exam.  Here the firefighters have to get onto trucks and use equipment based on 'evolutions'  (AFD's term) which are more or less scenarios that the firefighters have to role-play.  This is the most hands-on part of the exam process.

If you pass these two tests, then you can go on to the oral exams - made up of ten questions of the technical oral exam and then of five questions of the 'peer review' oral exam.  That's fifteen questions in an hour, or four minutes per question.

Jeff Graham passed the written and the practical exams.  His dispute was over the oral exams which he complained about as being subjective and easy to manipulate.  So let's, in this post, look at the problems with the technical part of the oral exams.

All Oral Exams, Including Interviews, Are Suspect

Before we even start on the oral technical exam,  I need to reiterate this point I made in an earlier post.

Despite the popularity of job interviews, experts agree that they are among the most biased and result in the least accurate predictions of candidate job performane.  

 You can read the whole article or search the internet on this topic and find I'm not exaggerating or cherry picking.  It's pretty much only people who conduct interviews who think they know what they're doing.

" interviewers typically form strong but unwarranted impressions about interviewees, often revealing more about themselves than the candidates."
"Research that my colleagues and I have conducted shows that the problem with interviews is worse than irrelevance: They can be harmful, undercutting the impact of other, more valuable information about interviewees."

Or see this link.  I offer a few quotes.

"Consider the job interview: it’s not only a tiny sample, it’s not even a sample of job behaviour but of something else entirely. Extroverts in general do better in interviews than introverts, but for many if not most jobs, extroversion is not what we’re looking for. Psychological theory and data show that we are incapable of treating the interview data as little more than unreliable gossip. It’s just too compelling that we’ve learned a lot from those 30 minutes."

Some of these comments are made about unstructured interviews.  The AFD engineer exam was a structured interview which researchers find to be a little better.  But still not that good.  From one of the most well known academics writing on human resources management, Edward E. Lawler III:

"Years of research on job interviews has shown that they are poor predictors of who will be a good employee. There are many reasons for this, but perhaps the key explanation is that individuals simply don’t gather the right information and don’t understand what the best predictors of job success are. A careful analysis of the background of individuals and their work history and work samples are more accurate predictors of success on the job than are the judgments which result from interviews."

The graders in the AFD oral exam did not have background information on the candidates' performance, they didn't review the performance reviews done by the candidates' supervisors.

Excuse me for reiterating this point.  It's one that is counter to people's perception and to practice.  But it's important to make this point loud and clear from the beginning.

My Problems With the Exams

On the positive side, the AFD oral technical and peer review exams were structured.  But there were numerous problems with the structure.

The Questions

There were ten questions on technical topics.  My understanding of the conditions of my reviewing the  exam questions themselves, is that I can't talk about the specific questions publicly.  Only things that were discussed publicly in court.

There were ten technical questions.  The MOA provided no evidence - though Graham's lawyer asked the MOA to provide documentation of how the exams were validated or even that the exam was validated.  That is, we have no evidence to show that the ten questions were predictors of a candidate's likely success in the position of engineer.  There is content related to the job, but AFD has produced no evidence showing that, for example, a candidate with a score of 90 will be a better engineer than a candidate with a score of 65. It may very well be that AFD may only be testing who has the best oral test taking skills.

Two of the ten were about how to prepare for the exam itself.  The test creator said these were intended to be softball questions to relax candidates.  Graham was scored low on them.  The other questions were about things like how equipment worked and about AFD procedures for different things.

The Answers

For tests to be reliable, the graders need to be able to compare the candidate's answer to the ideal answer.  The key points should be listed with the value of each point.  The graders were given a package of answers.  Some questions had no real answers attached.  Most looked like they were cut and pasted together from text books or AFD policy and procedure manuals.  For one question, I timed myself reading the four pages of answer that were provided.  It took me 11 minutes and 30 seconds just to read the answer.  But there are only about four minutes available per question. How can a grader reliably a) listen to the candidate's answer, b) take notes, and c) read through the answer sheet to match what the candidate said to the expected correct answer?  He can't.  Graham appears to have done better on the few questions that had bullet points on the provided answers rather than several pages raw material.

Assuming that the question itself is valid, the answer sheets graders had should have had a list of key points that the candidate should mention for each question.  The best such answer sheets would identify the most important points that should be in the response with a score for each.  There was nothing like that.  Instead the graders had to balance the answer the candidate gave, the pages of 'answer' copied out of the text or manual, and give the candidate a score on a completely different score sheet.

The Grading Scale

Rubric For Oral Technical Exam 5  - Outstanding, exemplary performance far exceeds expectations 4.5 - Above average performance exceeds level of basic skills/abilities 3.5  - Adequately demonstrated the basic abilities/skills 2.5 - Needs improvement, falls short of what is expected 1.5 - Unsatisfactory, performance is substandard 0 - Unacceptable, does not demonstrate comprehensive and/or application of required skills sets

Note:  There's a half point difference between "Outstanding" and "Above average".  Then it drops, not .5 like from 5 to 4.5,  but  1 point to 3.5 "Adequate".  So a grader could think, ok, this is good enough, it's adequate.   But 3.5 isn't 'adequate' it's really 'just barely adequate."  It's the lowest possible passing score.  The top two scores are very high marks.  The next one is barely passing.  1.5 is "unsatisfactory", but 3.5 (the same interval from 5 as 1.5 is from zero

But on the bottom of the scale, it goes from 0 to 1.5 - both of which are almost equivalent failing grades.  But the 1.5 is the same interval as from 5 to 3.5.



1.  This scale has uneven intervals. That is, the distance between the points on the scale are not the same.  Look closely.  The top two scores are both strong passing scores and the bottom two scores are both very poor failing scores.  But the top two are separated by .5 points, while the bottom two are separated by 1.5 points.   That's the same interval as from 5 to 3.5.

If 1.5 is 'unsatisfactory' the 3.5 should be "satisfactory' but it's only 'adequate' and more accurately 'barely adequate' because 3.5 is the lowest you can get and still pass. 3.4 is a failing grade.  (It's less than the 70% (20*3.5) needed to pass.)

The scale skews the scores down.  The points graders can mark go from 100% (5) to 90% (4.5) to 70% (3.5) which is the lowest possible passing score.  Why isn't there an 80% (4)?  That's what normal scales with equal intervals would have next.

2.  Passing on this test is 3.5. It took me a bit to figure that out, but 70% is the passing score, so each of these numbers need to be multiplied by 20 to get the actual score.  70% (3.5*20) is the minimal passing score.  Let's compare that to other scoring forms you know.  Say the standard Likert scale on most surveys:

"How do you feel about the Principal's performance this year?
5 – strongly approve
4 – somewhat approve
3 – neutral/no opinion
2 – somewhat disapprove
1 – strongly disapprove" (from Brighthub Education though it had the scale reversed)

Note that here a score of 3 is in the middle and is neutral, whereas in the AFD rubric, 3.5 is the lowest passing score.  The 3 or neutral would have a range from 3.5 to 2.5.  So a 3.5 would be on the high end, not the low end  as it is on the AFD scale.

I chose this 5 point scale, because the AFD calls its scale a five point scale.  But if you look at it, it's really a six point scale, since it has six possible scores from 0, 1.5, 2.5, 3.5, 4.5, and 5.  They didn't even realize they had a six point scale!   In most six-point Likert scales, there are three positive and three negative options. On such a scale (5, 4, 3, 2, 1, 0)  a 3.5 would be a strong passing score, not barely passing.

Let's look at it again from a different angle.  In the common academic scoring system, 90-100 is an A, 80-89 is a B, 70-79 is a C, 60-69 is a D, and below 60 is an F.

But in the AFD system, below 70 is an F.   On the AFD scale the first two numbers (a ten percent interval) are both very good scores.  The next score (20 percent less) is barely passing.  All the rest are failing scores.

Why does this matter?  First, it matters because the intervals are not equal.  It's not common to have uneven intervals  so that the distance between one score and another can vary from 10% (5 to 4.5) to 20% (4.5 to 3.5) to 30% (1.5 to 0).

It matters because the scale isn't like any scale we normally see and so the graders don't have a good sense of how skewed it is toward failing candidates.  The 3.5 isn't 'good', it's barely passing.  Yet most people are used to the academic scoring system and would assume that 70% would be something like a low C.  There is no 'good' on this scale.  There's 'walks on water,' 'really outstanding,' then 'barely passing'.  It took me a while to understand why the scale seemed so off.  I don't imagine an AFD grader would figure this out in the time they have to grade.  There isn't enough time to even figure out what the right score should be, let alone analyze the scale to see how skewed it is.

Reliability

Tests have to be both valid and reliable.  Valid means a high score predicts success (in this case, 'will perform the job well') and a low score predicts (in this case) 'will perform the job poorly'.  Even if the test itself is valid (and the AFD never did have anyone test for validity), they also have to be reliable if they will be accurate predictors.  Without reliability, the scores aren't accurate, and so the test is no longer a good predictor (no longer valid.)
Reliable means that the same candidate taking the test under different conditions (different time, different place, different graders) would essentially get the same score.

Given the lack of clear answers for the graders to use to compare the candidate's answer to, and given the lack of a clear rating system, it's not likely that this is a reliable test.

And in fact, when I looked at the scores individuals got on the same question by different graders, there were some big differences in score.  Some were a point off, some were 2 points off and even 2.5 points off.  There were even a few that were 3 points off.

That may not sound like much, but on a six point scale, it's a lot.  Even though they actually used a six point scale, they said that 3.5 was equal to 70% - the lowest passing grade.  So each point was worth 20%.  Therefore, two points equals a 40% difference.  2.5 equals a 50% difference on the score.  3 equals 60%.  If this test were reliable, the scores by different graders shouldn't vary by more than 10% or so.  That would be half a point on any question.

On this exam,  graders gave scores that were up to 60% (3 points) different for the same candidate on the same question!  That's the difference - in an academic setting - between a 95% (A) and 35% (low F).  But even a one point difference is a 20% difference - that could be a 90% and a 70% or the difference between an A- and a C-.   That's a huge spread, and it's there because the answer sheets don't tell the graders what a good answer looks like, and the grading scale easily tricks graders into giving lower ratings than they realize they are giving.

The two people in charge of setting up the tests (and the overall training director at the time) were actually not qualified to prepare the tests.  They all had Alaska Fire Safety Training Instructor Certificate Level 1.  They were eligible to administer training and tests that had been prepared by someone else, someone with a Level II certificate.  Here's the earlier post on the certifications.

Accountability

The exam is an hour long.  Some of the graders wrote brief notes on their score sheets, but there isn't much room.  Those notes are the only record of what actually happened in the oral exam.  There's no way for the candidate or the graders to check what was actually said.  Even as they review the pages that contain the answer, they can't replay the tape to see if the candidate covered something or not.  And years later?  At trial?  Well I know that five minutes later my wife and I have totally different recollections of what each of us said.  And in trial, there were a number of occasions when Jeff Jarvi, Jeff Graham's attorney, asked the graders about something they had said at their depositions or at the State Human Rights Commission hearing.  They'd answer.  Then Jarviv would have them read the transcripts, and the transcripts were not at all what they remembered.

Video or audio recordings of the oral exam probably could have averted this whole trial.  The tapes would have demonstrated whether Jeff Graham had covered the points as he said he did, or that he didn't as some of the graders said.

Not having recordings means no one can check how well the graders did their jobs.  Or whether people with low scores really did answer less accurately than people with high scores.  And it means that candidates can't get feedback on how they did so they can better prepare for the next exam.  (I'd note, that as a professor, the most convincing way to demonstrate to students that his paper wasn't very good, was to have them read the best papers.  It worked every time.)

Unequal Conditions

One other twist here.  This takes us ahead to the oral peer review, but it affects how people do on the technical as well.  Prior to the oral peer review, candidates are 'pre-scored' by the graders based on their personal knowledge of the candidates.  Imagine random co-workers some who might work closely with you and others who don't, evaluating you based on, well, whatever they want.  No one asks the basis.  It's not based on written materials they have about your work history, because there is the option of N/O - not observed.  If there were some written documentation, they wouldn't need the N/O option.

When the candidate walks into the room, he (it's almost always a he) is given his pre-scores.  If you pass (get over 70%) you can choose to spend more time on the technical questions and skip the peer-review questions altogether.  You can gain 20 minutes.

Imagine the emotional impact of being told, before you even start the oral technical exam, that the graders had failed you on the pre-scoring of the peer-review.  Or that they passed you with a high score.  In Jeff's case, he was stunned to learn he'd been failed by the graders in the pre-scoring of the peer-review.  It set him back as the technical oral began.


You've Made It To The End

If you made it this far, congratulations.  Going through details like this is necessary to truly get a sense of how badly these exams were designed and implemented.  But it does take work.  Thanks for hanging in there.  These are the kinds of details that jury had to sit through over the three week plus trial.

This post covered the technical oral exam problems and the next post will cover the oral 'peer review' exam that was part of the one hour along with the technical oral exam.