Graham v MOA #9: Exams 2 - Can You Explain These Terms: Merit Principles, Validity, And Reliability?

The Municipality of Anchorage (MOA) Charter [the city's constitution] at Section 5.06(c) mandates the Anchorage Assembly to adopt

“Personnel policy and rules preserving the merit principle of employment.”   AMC 3.30.041 and 3.30.044 explain examination types, content, and procedures consistent with these merit principles.  

Âs defined in the Anchorage Municipal Code Personnel Policies and Rules,

“Examination means objective evaluation of skills, experience, education and other characteristics demonstrating the ability of a person to perform the duties required of a class or position.” (AMC 3.30.005)

[OK, before I lose most of my readers, let me just say, this is important stuff to know to understand why the next posts will look so closely at the engineer test that Jeff Graham did not pass.  But it's also important to understand one of the fundamental principles underlying government in the United States (and other nations.)  And I'd add that the concepts behind merit principles are applied in most large private organizations to some extent, though they may have different names.

Jeff Graham's attorney made me boil this down to the most basic points to improve the likelihood I wouldn't put the jury to sleep.  So bear with me and keep reading.

And, you can see an annotated index of all the posts at the Graham v MOA tab above or just link here.]  

Basic Parts of Government In The United States

Governments can be broken down into several parts.

• The elected politicians who pass the laws and set the broad policy directions (legislature)
• The elected executive who carries out the laws.
• The administration is led by the elected executive - the president, the governor at the state level, and the mayor at the city level.
• Civil Service refers to the career government workers who actually carry out the policies.  There are also appointed officials at the highest levels who are exempt from some or all of the civil service rules.

Merit principles are the guidelines for how the career civil servants are governed.  

So What Are Merit Principles?

Probably the most basic, as related to this case, are:

• Employees are chosen solely based on their skills, knowledge, and abilities (SKAs) that are directly related to their performance of the job. 
• The purpose of this is to make government as as effective and efficient as possible by hiring people based on their job related qualities and nothing else.  
• That also means other factors - political affiliation, race, color, nationality, marital status, age, and disability should not be considered in hiring or promotion.  It also means that arbitrary actions and personal favoritism should not be involved. 
• Selection and promotion criteria should be as objective as possible.   

So Steve, what you're saying, this sounds obvious.  What else could there be?

Before the merit system was the Spoils System.  Before merit principles were imposed on government organizations, jobs (the spoils) were given to the victors (winning politicians and their supporters)   The intent of the Merit System is to hire the most qualified candidates.

In 1881, President Garfield was assassinated by a disgruntled job seeker, which spurred Congress to set up the first version of the federal civil service system - The Pendleton Act.

Only a small number of federal positions were covered by this new civil service act, but over the years more and more positions were covered and the procedures improved with improvements in the technology of testing.  The merit system, like any system can be abused, but it's far better than the spoils system.  Objective testing is a big part of applying merit principles.

What does 'objective criteria' mean? 

Objectivity has a couple common and overlapping meanings:

• Grounded on facts.  Grounding your understanding or belief on something concrete, tangible.  Something measurable that different people could 'see' and agree on.
• Unbiased.  A second, implied meaning from the first, is that you make decisions neutrally, as free as you can be from bias, preconceived ideas.  That’s not easy for most people to do, but there are ways to do it better. 

What Ways Can Make  Tests More Objective And Free Of Bias?

I think of objectivity as being on one end of a continuum and subjectivity being on the other end.  No decision is completely objective or subjective, nor should it be.  But generally, the more towards the objective side, the harder it is to introduce personal biases.* 

objective ...............................................................................................subjective

First Let's Define "Test"

In selection and promotion, we have tests. Test is defined as any thing used to weed out candidates, or rank candidates from poor to good.  So even an application form can be a test if it would lead to someone being cut out of the candidate pool.  Say candidates are required to have a college degree and someone doesn’t list one on an application.  They would be eliminated already.  

Again,  how do you make tests more objective?

There are two key terms we need to know:  validity and reliability.

What’s Validity?

Validity means that if a person scores higher on a test, we can expect that person to perform better on the specific job.  

Or saying it another way, the test has to truly test for what is necessary for the job.  So, if candidates without a college degree can do the job as well as candidates with a degree, then using college degree to screen out candidates is NOT valid.  

And what is reliability?

Reliability means that if  a person takes the same test at different times or different places, or with different graders, the person should get a very similar result.  Each test situation needs to have the same conditions, whether you take the test on Monday or on Wednesday, in LA or Anchorage, with Mr. X or Miss Y administering and/or grading the test.  

How Validity and Reliability Relate To Each Other

To be valid, the selection or promotion test must be a good predictor of success on the job. People who score high on the exam, should perform the job better than those who score low.  And people who score low should perform worse on the job than people who score high.

BUT, even if the test is intrinsically valid, the way it is administered could invalidate it.  If the test is not also reliable (testing and grading is consistent enough that different test takers will get a very similar score regardless of when or where they take the test and regardless of who scores the test) the test will no longer be valid.  This is because the scores will no longer be good predictors of who will do well on the job.

How do you go about testing for validity and reliability?

This can get complicated, especially for  factors that are not easy to measure.  I didn't go into this during the trial.  I wanted to point out some pages in a national Fire Safety Instructor Training Manual used by the Municipality of Anchorage, but I was not allowed to mention it.  It talks about different levels of validity and how to test for them.  It also says that for 'high stakes' tests, like promotion tests, experts should be hired to validate the test.  The jury didn't get to hear about this. But it's relevant because as I wrote in an earlier post, the people in charge of testing, and specifically in charge of the engineer exam, only had Level I certification, which allows them to administer training and testing designed by someone with Level II certification.  It's at Level II that validity and reliability are covered.  

There really wasn't need to get detailed in the trial, because the oral exam was so egregiously invalid and unreliable that you you could just look at it and see the problems.  And we'll do that in the next posts.

That should be enough but for people who want to know more about this, I'll give a bit more below.

-----------------------------------------------------------------------

Extra Credit

*"the harder it is to introduce bias"  There are always was that bias can be introduced, from unconscious bias to intentionally thwarting the system.   When civil service was introduced in the United States, there was 'common understanding' that women were not qualified for most jobs.  That was a form of bias.  Blacks were also assumed to be unqualified for most jobs.  Over the years these many of these sorts of cultural barriers have taken down.  But people have found other ways to surreptitiously obstruct barriers.  

Merit Principles

If you want to know more about merit principles I'd refer you to the Merit System Protection Board that was set up as part of the Merit System Reform Act of 1978.  

A little more about reliability problems (because these are important to understand about the engineer promotion exam)

In the main part of this post I wrote that all the important (could affect the score) conditions of the test need to be the same no matter where or when or with whom a candidate takes the test.  Here are some more details

• Location - If one location is less comfortable - temperature, noise, furniture, lighting, whatever - it could skew the scores of test takers there.
• Time -  could be a problem in different ways.  
• All candidates must have the same amount of time to take the test.  
• Instructions - all instructions have to be identical
• Security of the test questions - if some applicants know the questions in advance and others do not, the test is not reliable.

The scoring, too, has to be consistent from grader to grader for each applicant.

And there are numerous ways that scoring a test can go wrong.

• Grader bias  - conscious and unconscious.   Raters who know the candidates may rate them differently than people who don’t know them at all. 
• The Halo effect means if you have a positive view of the candidate, you’re likely to give him or her more slack.  You think, I know they know this?  
• The Horn or Devil Effect is the opposite - If you already have a negative opinion about a candidate, you consciously or unconsciously give that a candidate less credit.  These are well documented biases.
• Testing order bias affects graders and candidates.  
• After three poor candidates, a mediocre candidate may look good to graders.  

• Grading Standards - Is the grading scale clear and of a kind that the graders are familiar with?
• Are the expected answers and how to score them clear to the graders?
• Do the graders have enough time to calculate the scores consistently?
• Grader Training -
•  If they aren't well trained, it could take a while to figure out how to use their scoring techniques, so they score different at the end from the beginning. 

How Do You Overcome the Biases In More Subjective Tests Like Essays, Interviews, and Oral Exams?

Despite the popularity of job interviews, experts agree that they are among the most biased and result in the least accurate predictions of candidate job performane.  Or see this link.

You have to construct standardized, objective rubrics and grading scales - this is critical, particularly for essay and oral exams.

On November 9, 2016 when the electoral college vote totals were tallied, everyone saw the same facts, the same results.  But half the country thought the numbers were good and half though they were bad. When evaluating the facts of a job or promotion candidate, the organization has to agree, before hand, what ‘good’ facts look like and what ‘bad’ facts look like. Good ones are valid ones - they are accurate predictors of who is more likely to be successful in the position.   Good and bad are determined by the test maker, not by the graders.  The graders merely test whether the performance matches the pre-determined standard of a good performance.

What’s a rubric?

It’s where you describe in as much detail as possible what a good answer looks like.  If you’re looking at content, you identify the key ideas in the answer, and possibly how many points a candidate should get if they mention each of those ideas.  It has to be as objective as possible. The Fire Safety Instructor Training Manual has some examples, but even those aren't as strong as they could be.

Good rubrics take a lot of thought - but it's thought that helps you clarify and communicate what a good answer means so that different graders give the same answer the same score.

Here are some examples:
UC Berkeley Graduate Student Instructors Training
Society For Human Resource Management - This example doesn't explicitly tell graders what the scores (1,2, 3, 4, 5) look like, as the previous one does.
BARS - Behaviorally Anchored Rating Scales - This is an article on using BARS to grade Structured Interviews.  Look particularly at Appendices A & B.
How Olympic Ice Skating is Scored - I couldn't find an actual scoring sheet, but this gives an overall explanation of the process.

My experience is that good rubrics force graders to ground their scores on something concrete, but they can also miss interesting and unexpected things.  It's useful for graders to score each candidate independently, and then discuss why they gave the scores they did - particularly those whose scores vary from most of the scores.  Individual graders may know more about the topic which gives their scores more value.  Or may not have paid close attention.   Ultimately, it comes down to an individual making a judgment.  Otherwise we could just let machines grade.  But the more precise the scoring rubric, the easier it is to detect bias in the graders.


Accountability

Q:  What if a candidate thinks she got the answer right on a question, but it was scored wrong?

Everything in the test has to be documented.  Candidates should be able to see what questions they missed and how they were scored.  If the test key had an error, they should be able to challenge it.

Q:  Are you saying everything needs to be documented?

If there is going to be any accountability each candidate’s test and each grader’s score sheets must be maintained so that if there are questions about whether a test was graded correctly and consistently from candidate to candidate, it can be checked.

In the case of an oral exam or interview, at least an audio (if not video) record should be kept so that reviewers can see what was actually said at the time by the candidate and the graders.

Q:  Have you strayed a bit from the Merit Principles?

Not at all. This all goes back to the key Merit Principle - selecting and promoting the most qualified candidates for the job.  There won’t be 100% accuracy. But in general, if the test is valid,  a high score will correlate with a high job performance.  But unless the test is also reliable, it won’t be valid. The more reliable the test, the more consistent the scores will be under different conditions and graders.  The best way to make tests more reliable is to make them as objective as possible.

Send This Powerful Video By Marjory Stoneman Douglas HS Student To Your Congress Members

Please watch this video of a Marjory Stoneman Douglas high school student talking about stopping mass shootings at schools.  It's powerful.

I predict the survivors of this shooting will be cited in years to come as the turning point, where the NRA's stranglehold on Congress began to fade.  If #Metoo is galvanizing women, this shooting - if this video is an indication - is going to galvanize students and their parents.  Any time the pendulum goes too far in one direction, it can start swinging back very quickly.




I urge you to send the link to this video -

https://youtu.be/ZxD3o-9H1lY

-  to your US Senators and Congress Member. Just google the name + contact info. If you don't know the name, google [Your State} US Senators Contact Info and [Your Zip Code] US Congress Member Contact Info.  Or go to

• this US Senate link that lists all senators and contact info.   
• this link gets you your US Representative by zip code.

The forms are quick and easy. If each Senator and Rep get 100 people sending in this video, it will make a difference. 1000 people would be even better.

I'd note my previous post was about Marjory Stoneman Douglas, the woman after whom the high school was named. She is credited with saving the Everglades among other things and died at 108.

 I'd like to think that the young woman in this video got some of her passion and courage to act from having a school named after such a powerful and determined role model.  (I would just note that I'm sure that during much, if not most, of her life, Marjory Stoneman Douglas didn't think of herself as powerful.)

Marjory Stoneman Douglas - A Remarkable Woman Who Lived To 108

The first thing I noticed when I googled "Marjory Stoneman Douglas High School" was a bio of a woman who lived to 108, was a key, if not 'the' key, person behind saving the Everglades, and who has a lot of stuff named after her besides the high school.

Highlighting her seemed a good idea, but I got caught up in examining Trump's speech on the shooting.  Now as I am ready to post about her, I see that others had the same idea since the shooting.

But for those of you who haven't heard already, this is a good time to get to know a remarkable woman.

From Wikipedia intro on her:

"Marjory Stoneman Douglas (April 7, 1890 – May 14, 1998) was an American journalist, writer, feminist, and environmentalist known for her staunch defense of the Everglades against efforts to drain it and reclaim land for development. Moving to Miami as a young woman to work for The Miami Herald, she became a freelance writer, producing over a hundred short stories that were published in popular magazines. Her most influential work was the book The Everglades: River of Grass (1947), which redefined the popular conception of the Everglades as a treasured river instead of a worthless swamp. Its impact has been compared to that of Rachel Carson's influential book Silent Spring (1962). Her books, stories, and journalism career brought her influence in Miami, enabling her to advance her causes.
As a young woman Douglas was outspoken and politically conscious of the women's suffrage and civil rights movements. She was called upon to take a central role in the protection of the Everglades when she was 79 years old. For the remaining 29 years of her life she was "a relentless reporter and fearless crusader" for the natural preservation and restoration of the nature of South Florida.[1] Her tireless efforts earned her several variations of the nickname "Grande Dame of the Everglades"[2] as well as the hostility of agricultural and business interests looking to benefit from land development in Florida. Numerous awards were given to her, including the Presidential Medal of Freedom, and she was inducted into several halls of fame.
Douglas lived to 108, working until nearly the end of her life for Everglades restoration. Upon her death, an obituary in The Independent in London stated, "In the history of the American environmental movement, there have been few more remarkable figures than Marjory Stoneman Douglas."[3]"

The President's Speech On Florida Shooting - Between The Lines

Here's my take on the President's speech this morning on the Parkland school shooting.  I've done a reasonably close transcript (since I couldn't find any yet)[I did find one before I was done here- it's officially a 'statement'' not a speech] and I'll comment between the lines of his speech.  Let me say, that much of the rhetoric and tone were appropriate for this sort of situation.  But given the President's own words and behaviors, much of it seems ironic.  And if we look carefully at the text, we'll see it's full of clichés but lacking any real substance.  This would be an appropriate speech, perhaps after the first school shooting this year, but ABC News reports there have already been 18 school shootings in the first 45 days of 2018. (Not all were intentional, some were suicides.)

Our president and legislators can't keep using the same condolence platitudes.

But before I look at the speech in detail, I want to point out this was very much "a speech."  He's using his speech voice as he reads.  He is not speaking 'from the heart' or in his natural voice.  That's fairly easy to test.  Just listen to the last line of the speech:

"Thank you and God bless you all." 

If he were actually feeling the words he was speaking, his 'thank you' would have been real.  But it wasn't.  He didn't even know that he'd just said 'thank you'.  How do I know that?  Because immediately after finishing the speech, he says in a much more natural tone of voice:

"Thank you very much."  

You can hear the switch from speech voice to real voice clearly. Here's the end tape of the tape.  [I have it set to begin at 6:22.  It seems to do that the first time you play it, then it goes back to the beginning.  Which allows you to hear the whole speech if you have nothing better to do.]






The Speech  [Trump's words are in italics and indented.  Mine are not.]

My fellow Americans, today I speak to a nation in grief. Yesterday, a school filled with innocent children and caring teachers became the scene of terrible violence. hatred, and evil.

Cliché alert.  What does 'innocent children' actually mean? These are high school students, not elementary school kids.  They may be relatively young, but I'm guessing innocent is not the best word to describe most of them.  The shooter, after all, was one of the students last year.

Cliché alert:  'Evil.'  While evil is used to mean unspeakably bad, Collins also defines it this way:

"Evil is a powerful force that some people believe to exist, and that causes wicked and bad things to happen."

Evil, in this religious sense, is an underlying natural force in the world.    It's often used to deflect blame or suggest there are no solutions.    Later in the speech Trump talks about tackling the issue of mental health.  Is he demonizing mental illness?  Or is evil just one of the words speechwriters always choose for this topic?

"Around 2:30 yesterday afternoon, police responded to reports of gunfire at Marjory Stoneman Douglas High School in Parkland, Florida, a great and safe community."

I guess it isn't all that safe.   Or if people thought it was, it means there are no safe communities.

"There, a shooter who is now in custody opened fire on defenseless students and teachers.  He murdered 17 people and badly wounded at least 14 others.  Our entire nation with a one heavy heart is praying for the victims and their families.  To every parent, teacher, and child who is hurting so badly,  we are here for you whatever we can do, whatever you need, to ease your pain."

Does that include this student from the high school who tweeted in response to Trump's condolence tweet ?

I can't confirm this really is a student at that school, though Newsweek quoted the tweet.  But I'm sure there are some students at the school who feel this way.  Will gun control be off the agenda when Trump visits the school, like it was in this speech?

"We are all joined together as one American family, and your suffering is our burden also.  No child, no teacher should ever be in danger in an American school.  No parent should ever have to fear for their sons and daughters when they kiss them goodbye in the morning." 

True, though I wonder how many parents actually kiss their high school students before they head off in the morning?    And we've been having school and other mass shootings regularly going back before Sandy Hook.  Things haven't gotten safer.  What concrete action do you plan to take?

"Each person who was stolen from us yesterday, had a full life ahead of them, a life filled with wondrous beauty and unlimited potential and promise.  Each one had dreams to pursue and love to give, and talents to share with the world.    And each one had a family to whom they meant everything in the world."

This is really quite beautiful and insightful.  And he read it well, but I have some questions.  Does he feel this about the kids in ghetto Baltimore?  The women who are accusing him of sexual abuse?   What about the Dreamers he wants to send back to dangerous, if not life threatening, situations?  These are, I'm afraid, just words that the President read, that someone else wrote for him.

"Today we mourn for all of those who lost their lives.  We comfort the grieving and wounded. And we hurt for the entire community of Parkland, Florida  that is  now in shock and pain and searching for answers.  To law enforcement, first responders and teachers who responded so bravely in the face of danger:  We thank you for your courage."  

Appropriate sentiments, not much substantive content.

"Soon after the shooting I spoke to Gov Scott to convey our deepest sympathies to the people of Florida and our    determination to assist in any way we can.  I also spoke with Florida attorney general Pam Bondi and  Broward County Sheriff Scott Israel.   I’m making plans to visit Parkland to meet with families and local officials and continue coordinating the Federal response." 

 Pam Bondi, if you recall, was deciding on whether to prosecute Trump University when her campaign got a $25,000 check from aTrump foundation.  She made her decision shortly after to drop the case.

"In these moments of heartache and darkness, we hold on to God’s word in scripture, “I have heard your prayer, and seen your tears, I will heal you.”  We trust in that promise and we hold fast to our fellow Americans in their time of sorrow."

I'm not a biblical scholar.  But I'd note this quote comes from a story in  II Kings 20:5 and also in  Isaiah, Chapter 38.   Hezekiah is ill.  Isaiah hears from a prophet that Hezekiah will die. Isaiah tells Hezekiah, who then prays to the Lord to remember he's lived a good life.  It is then, that Isaiah hears the words Trump quotes.

"Before Isaiah had left the middle court, the word of the Lord came to him: 5 “Go back and tell Hezekiah, the ruler of my people, ‘This is what the Lord, the God of your father David, says: I have heard your prayer and seen your tears; I will heal you. On the third day from now you will go up to the temple of the Lord. 6 I will add fifteen years to your life. And I will deliver you and this city from the hand of the king of Assyria. I will defend this city for my sake and for the sake of my servant David.’"

This is not a prayer for grieving parents, but for a dying King who is given reprieve.

But Trump is talking to the grieving relatives of dead children and teachers, and unlike Hezekiah, they won't be healed.

"I want to speak now directly to America’s children."

This is perhaps the scariest part of the speech for me.  How many people would want Trump to give advice to their children?  Especially their high school age daughters?  Gives me the creeps.

"Especially to those who feel lost, alone, confused, or even scared.  I want you to know that you are never alone and never will be.  You have people who care about you and love you and will do anything to protect you.at all.  If you need help turn to a teacher, a family member, a local police officer, or a faith leader.  

In general, this is something a loving and caring president might tell children.  Yet, for the most vulnerable of children, this isn't necessarily true.  Think of the children of religious families who were kicked out when they said they were gay.  The kids of drug addicts may not have anyone.  And I know that a lot of African-American parents would not tell their kids to report things to local police officers.   Kids with mental health problems feel isolated.  Like the shooter.  Who was helping him?  Apparently there were lots of signs he needed help desperately.  Are people like him included here? Or are they the evil Trump warns about?

Answer hate with love and cruelty with kindness.  We must also work together to create a culture in our country that embraces the dignity of life  that creates deep and meaningful human connections and that turns classmates and colleagues into friends and neighbors."

Really?  Trump is telling people to answer hate with love, cruelty with kindness?  Create deep and meaningful human connections?  Were the speechwriters giggling at the irony as they wrote this?

"Our administration is working closely with local authorities to investigate the shooting and learn everything we can. " 

He also tweeted that 

"We have done a great job with the almost impossible situation in Puerto Rico."
 ...people are now starting to recognize the amazing work that has been done by FEMA and our great Military. All buildings now inspected....."

Can we believe him?

"We are committed to working with state and local leaders to help secure our schools tackle the the difficult issue of mental health."

As much as Republicans want to talk about mental health, rather than guns, as the way to prevent shootings, their track record on supporting mental health regularly comes into question.

"Later this month I will be meeting with the nation’s  governors and attorney generals [sic] where making our schools and children safer will be our top priority."

I can't find anything about a meeting between the president and the governors and AG's.  The only official looking schedule for the president I can find is at Fact Based, but it only goes backward in time from today.  Not into the future.  

There is a National Governor's Association Winter meeting scheduled for February 24-  in DC.  Maybe that's what he's referring to.  Its agenda doesn't include school shootings or mental health.  How exactly does the president meet with the 50 governors and attorneys general? Almost half the state AG's have already sued the President over various issues in his first year in office.   So they don't seem to have "created deep and meaningful human connections" with the president yet.

They're only meeting for two days and they already have a full agenda.  I just don't see this as more than a way to look like he's going to do something.  The kind of thing the next sentence of his speech warns against.

"It is not enough to simply take actions that make us feel like we are making a difference.  We must actually make that difference.  In times of tragedy, the bonds that sustain us are those of family, faith, community, and country.  These bonds are stronger than the forces of hate and evil.  And these bonds grow even stronger in the hours of our greatest need."

So we're supposed to rely on our bonds of family, faith, community, and country to end school shootings?  More on the forces of hate and evil.

"And so always, but especially today, let us hold our loved ones close, let us pray for healing and for peace, and let us come together as one nation and wipe away the tears and strive for a much better tomorrow.  Thank you and God bless you all."

That ends the official speech, but then, finally, Trump talks in his real voice.  Which, as I pointed out above, shows that he wasn't even comprehending what he was reading.  Since the end of the official speech already ended with a thank you.

Thank you very much.

I'd also point out that simply talking about mentally ill people being the problem is also problem.  TheNational Alliance on Mental Illness claims that18% of US adults experience mental health problems in any year.  That's almost 20% of our population.  The vast majority are NOT high risk to be shooters.  Linking mental illness so broadly to mass shootings is likely to continue the stigma of mental health, and  mean fewer people seek treatment for their mental health problems.            

Seattle Basking In Cold Sunshine and Soccer

The clouds and rain took a break for the last few days and we've had sun, since Friday I think.  It's a very different look.

The weekend consisted of three soccer games for me.  I drove the youngest of the three step-grandsons to Olympia for his game on Saturday.  Sunday there were two championship games at Starfire soccer complex near SEATAC.  It was mostly sunny, but being outside in the low 40s˚F weather with a constant breeze was bracing.  But the oldest's team won the US Youth Soccer Championship for Washington.  The game was tied 2-2 at the end, so it went into a scoreless 30 minute overtime. So it was decided on penalty kicks.  Here's a bit of the ballet of the last game.

A very exciting, if chilly day.

The sun continued Monday.  After dropping Z off, I did my walk through the park then drove off the island to get a headlight bulb, picked Z up in the afternoon and took her to theater camp, then home to check out a kindergarten school for next fall.  It's an alternative school in the regular school district, but it's small and students are chosen by lottery.

Today was sunny again, but colder.  Right around freezing.  There was frost on the lawn.

And ice in part of the pond.

And the puddles were more seriously iced up.

My granddaughter's life is pretty full.  There was an astronomy activity at the planetarium Saturday when we got back from the soccer game.  But all the activities give her opportunities to learn through playing and she enjoys them.  And she helped me change the headlight bulb yesterday.  And keeping up with her is cutting into my blogging.  But I'm sure it's much more worthwhile.

Why Do I Immediately Translate This To Men's Lives?

I can't help but wonder if he really means 'men's lives.'

Peoples lives are being shattered and destroyed by a mere allegation. Some are true and some are false. Some are old and some are new. There is no recovery for someone falsely accused - life and career are gone. Is there no such thing any longer as Due Process?

— Donald J. Trump (@realDonaldTrump) February 10, 2018

And I'm delighted that he knows about Due Process.   Though the man who wants to send immigrants back to countries where they face a high likelihood of violence and death without hearings and process seems the least likely person to ask this question.  But if you believe that all criticism about you is false and merely part of conspiracy to destroy you, then you don't have to take any blame for making bad decisions.

Like all legal terms there's no simple definition.  The link goes into lots of the complications, but here's the part that I think is critical.

"The clause also promises that before depriving a citizen of life, liberty or property, government must follow fair procedures. Thus, it is not always enough for the government just to act in accordance with whatever law there may happen to be. Citizens may also be entitled to have the government observe or offer fair procedures, whether or not those procedures have been provided for in the law on the basis of which it is acting. Action denying the process that is “due” would be unconstitutional."

Ostranenie

Ostranenie      defamiliarization

"A central concept in Russian Formalism's attempt to describe and define what constitutes literaturnost (literariness). A neologism, it implies two kinds of actions: making strange, and pushing aside. Consistent with this double meaning, the concept refers to the techniques writers use to transform ordinary language into poetic language, which for the Russian Formalists is language which induces a heightened state of perception. Habit, according to the Russian Formalists, is the enemy of art, therefore to produce art the writer has to force the reader outside of the usual patterns of perception by making the familiar appear strange or different. The principal theorist of this concept, Victor Shklovsky, uses a famous passage in Tolstoy's War and Peace (1869), where an opera is described as ‘painted cardboard and oddly dressed men and women who moved, spoke and sang strangely in a patch of blazing light’ to exemplify this concept. Basically what Tolstoy does, according to Shklovsky in Theory of Prose (1990), is view things out of context, or to put it another way he fails to see the thing that makes the actions he describes either meaningful or coherent and in this way he defamiliarizes them. In The Prison-House of Language (1972), Fredric Jameson enumerates three advantages of the concept of ostranenie: firstly, it enables literary theory itself to come into being by providing a way of distinguishing its object—namely, poetic language; secondly, it enables a hierarchy to be established within works and between works (i.e. more or less defamiliarizing); thirdly, it generates a new way of thinking literary history in terms of ruptures and breaks rather than continuities and influences. The problem with this concept, however, is that it is psychological rather than purely textual, inasmuch as it is premised on the deadened senses of the reader being awakened by clever writing rather than something specific to the writing itself. Obviously, too, this process suffers from the logic of diminishing returns—what was shocking yesterday is all too familiar today, thus demanding an ever greater level of shock to achieve a decreasingly small level of shock value (this, as many commentators have observed, is the problem contemporary non-representational art also faces). See also cognitive estrangement; estrangement-effect."

I found this word in a list of words that people should have in their vocabularies.  Most are words few people have in their vocabularies.

And, of course, this is a word that captures well, the idea I have of this blog at its best - making something common seem new and different.

Morning Routine This Week - Lots of Water Birds

Being a nomad screws up any sort of regular routine.  Since mid-November we've been in Anchorage   a little over three weeks.  But the trade off is time with family, particularly grandkids, mostly the one in Seattle.

This week my routine was to drive her to her school by 8:30 and then spend about 30 minutes walking the loop in the park across the street.  And so I've been noticing the most obvious birds:

• mallards
• widgeons
• geese
• robins
• gulls

I can't tell if there is one flock each of the water birds that are spread over the park, or more smaller ones.  The geese and widgeons spend time in the water and on the lawns.  

As we left Anchorage last week I was looking at the cases with the stuffed ducks and geese and noticed that the greater and lesser Canada geese looked very similar, but one was larger.  Another man pointed out which ones he'd hunted and said he'd gotten a greater Outside, that the ones in Anchorage were lesser. I'd always thought they were pretty big, but now I'm not sure.  

This pair announced itself from a distance as they flew to this pond.  There was already a flock in the pond and one goose started honking in response.  After they landed one of the new geese and the responder goose swam towards each other necks out low along the water.  They seemed to have a little challenge match and then everyone seemed to get along.  

The American widgeons move from pond to pond and also wander out on the lawn.  They also seem to mix with the mallards.  Just eyeballing it, I'd say there are more geese and widgeons than mallards.  

The mallards seem to stay mostly in the water.  I find myself dismissing mallards because they are so common.  But when I look through the camera, I'm always reminded how beautiful they are.  If every human were really beautiful, would we stop noticing?  

The robins run in crowds (are they a flock?) on the lawns, poking through the leaves for worms and other tasty bugs.  

There are other birds.  Most notably the other day there were buffleheads and ring-necked ducks.  If you look above at the mallard, they are true ring-necked ducks while the ring-necked ducks have rings on their beaks, not necks.  

The buffleheads are in the back and the ring-necked duck is up front.  I thought it was a scaup until I looked more carefully on line.  I'm not that familiar with the ring-necked ducks which have a similar color scheme to the scaup.  

Here's a better picture of the ring-necked duck.  

There are also lots of smaller song birds that flit around or sing from the bushes or trees.  But they are much harder to get on camera.  This one isn't a bad picture, but it's not much good for me to identify.  It was singing away up there.  

I also got a couple of bike rides in to a small beach on a tiny road where you can sit and listen to tiny waves and find nice shells and rocks.  Yet another similar duck - black with a white belly, was swimming out in the salt water - a golden eyes.  

And from that beach I could see downtown Seattle about 9 miles across the water.  With an Evergreen  container ship in-between.  

Enough for now.  Tomorrow the three boys each have soccer games in different locations.  I'm driving the youngest to Olympia for his game and luckily, some old friends from Anchorage live nearby and will come out to join us.


And I had to check up on this rule about stopping for school buses.  I did stop, but I wasn't sure if I had to when I was on the other side of the street.  This is for Washington State.

 "If you're on a two-lane road, both directions need to stop. If you're on a road with three or more lanes, you need to stop if you're traveling in the same direction as the bus. If you're traveling in the opposite direction, you can keep going."

There were a couple of days we came upon the school bus with flashing lights on the other side of the street.  On a two lane road.  

Did God Tell ' Michele Bachmann Not To Run For Senate. Really!?

I read a blog post that said God told Michele Bachman not to run for the US Senate.  I started thinking - how often does God talk to individuals?  So I googled to see how often he spoke to people in the Bible.

Someone at BibleStudyForum made a rough count and listed them.  Maybe about 30 folks between the old and the new testaments.  He wasn't sure if he got them all.

At the Titus Institute, there's a post (by Ron Jones?) titled, "How has God spoken to his people throughout the OT and NT?"  He concludes that it changed over time.  He spoke to Adam and Eve directly, before the Fall.  Thereafter he spoke through intermediaries.

"Even when God spoke to these intermediaries, we see infrequent communication rather than frequent regular communication.
He spoke to Noah 5 times over 950 yrs, Abraham 8 times over 175 yrs, Isaac 2 times and 1 time to Rebekah over 180 yrs, Jacob 7 times and 1 time to Laban over Jacob's lifetime. These are just some examples.
We also see that God does not address personal issues, only issues that involve his redemptive plan.
In the OT, God did not speak to his intermediaries regarding personal matters unless it involved his redemptive plan."

He goes on to say that today God communicates with humans through the Scripture.  (Yes, this is just one guy's interpretation.  He does give citations.)

Given the infrequency of God's communications with humans in the old and new testaments, it's hard to take seriously the many claims people make about talking to God today.

So then I went looking for Bachman's actual words so I could quote them.  From Salon:

"I took it to the Lord in a very quiet way, I took it before the Lord, I prayed, I tried to have my ears open and hear what God was saying to me," Bachmann explained.

The former Republican U.S. representative went on to detail that she "considered" running for Franken's old seat "quite a long time" but ultimately, it was God who persuaded her otherwise. 

"From the very first day when Al Franken had announced his resignation from the U.S. Senate, I went before the Lord and it became very clear to me that I wasn't hearing any call from God to do this," she explained. "I've always prayed and tried to seek out what God’s will for me would be, and each time before, I've had this inner sense that I'm supposed to do this, I'm supposed to run."  [emphasis added] 

Well, she's not exactly saying God told her not to run.  She says, she didn't hear from him.  And those times that she did run, she didn't say he actually spoke to her.  Rather "I had this inner sense. . . "

When people have difficult decisions to make - should I get married?  quit my job?  run for office?  buy this house? - they often go somewhere quiet to reflect.  I know that running or walking alone has always been a time when my brain could work things out until I 'saw' the answer.

I'm not much of a praying man, but I can imagine that what people call praying works pretty much the same way.  You block off everything else and mentally get into the question you're trying to answer.  And then it comes to you.  Which is what I hear when Bachman says, "I've had this inner sense that I'm supposed to run."

By this time I'm starting to understand the frustration of religious people who pray to get answers to hard questions.  It's their form of running or walking, just clearing their heads and sorting through the conflicting forces for or against an action.  I can understand their resentment when people mock them for it.  It's not different from when people make fun of people who exercise or who don't eat meat.  Maybe 'talk to God' isn't something they mean literally.  Maybe it's something their friends understand to mean this sort of self-reflection.

But the rest of us take it literally and it sounds hokey.  It sounds like a way to justify whatever someone wants to justify.  "God told me to . . . "  And I'm sure there are those who do say this, intending it to add weight to their conclusion on something.

The Titus Institute post does say that God doesn't deal with personal issues except as they are related to the redemptive plan.  So football teams that ask God to help them win. . . No, that would trivialize God's time and purpose.

I know that some readers will think I'm being way too lenient here.  But if we're going to take this country back from Russian bots and Fox News and all the others who are stoking hatred and destroying the communal trust  necessary for a democracy, we can't let others' nastiness excuse our own.  We have to stop being self-righteous, and treat others with respect.  Just as we say hurtful racial slurs should be avoided, I think hateful religious slurs should as well.   It's possible to disagree respectfully.

And for me, finding out that Bachman didn't literally say that God told her not to run was a reminder of how both sides reword things to fit their prejudices.  Let's stop being baited by tweets.  Let's turn the other cheek and show love for people hurting enough to cheer Trump's fear mongering.  Or, if you prefer, imagine Gandhi as your conscience.

Despite this picture, God did not talk to Michele Bachman, and she didn't exactly say that he did.

Graham v MOA #8: The Exams 1: How The Process Works

[The Graham v MOA tab above lists all the posts in this series and gives some overview of the case and why I think it's important.]

The Exams - How The Process Worked In 2012

The exams firefighter Jeff Graham sued the Municipality of Anchorage over were to determine who would be promoted from the entry level Anchorage Fire Department (AFD) position - firefighter - to the next level - engineer.  A firefighter has to promote to engineer to move up in the AFD.  If you passed the exams, you would go on a list based on your score.  Then, as there were openings for engineers, names would be taken from the top of the list.  If the list was used up before the next scheduled exam - in two years - an interim exam could be held.  If you were on the list, but not called in two years, then you had to take the exams all over again.  

In 2012, the exams had three parts:  1) a written exam, 2)  a practical exam, and 3)  an oral exam.  It was the first year an oral exam was part of the engineer promotion exams.   It was the oral exam that Graham objected to.  You had to pass ALL THREE parts to get on the engineer list.  The first two exams were relatively objective.  The oral exam was problematic in many ways, giving graders lots of leeway to pass or fail candidates without much accountability.   Let’s look at them all before going into the details of the oral exam in the next posts.  

The Written Exam 

This was based on a standard set of questions from a national bank of engineer test questions about technical knowledge.  It’s multiple choice.  Test makers can choose from many questions. This allows them to modify the test to be appropriate to local conditions.  Overall, the bank of questions has been validated nationally- the questions are related to what a fire engineer should know and this national association determines the  correct answers.  

Graham passed this exam with a score of 85.  He needed 70 to pass.  

The Practical Exam

The practical exam is made up of a series of 'evolutions,' as they call them, that test the candidates' abilities to handle the trucks and equipment as needed on duty.   The evolutions (think of them as scenarios) involve actually driving vehicles, hooking up hoses, responding to different types fires, etc.  This exam was designed by local test makers.  Casey Johnson was in charge of this in 2012 and he followed the basic model that had been used in previous exams, but creating his own specific scenarios. 

The different evolutions on the exam are supposed to be unknown to the test takers until they take the test.   But the exams are given outside on consecutive days and people taking it on the second or third day can learn from others what events they will be asked to respond to.

A related issue that came up has to do with training outside the Academy.  Senior AFD officers often assist firefighters in their stations when there are no emergency calls.  So different candidates will get different coaching on different possible evolutions at their stations.  In one case, it was argued that one of the people who helped prepare the practical exam gave his subordinates, at the station, training on a new process that hadn’t been used at AFD yet, but was on the exam.  Questions were raised whether they had gotten advance information to prepare for that event.  The suggestions were denied.  

This exam was not validated professionally.  Jeff Graham has not challenged the events on this exam - they are related to what people have to do as engineers, but whether successfully completing the events on this test is the best, or even a good, predictor of success as engineer is not known.  

Graham did have some questions about the reliability of the exam.  Scores on the first day of the exam were low and the fail rate was very high.  By the third day, the fail rate dropped significantly.  Why might this be?

The exams are done out in the open where they can be seen by anyone.  The first people to take the exam do not know what they will be asked to do.  By the third day, people have been able to see what events were used, plus people who took the test can talk to friends who haven’t taken the test, so the later test takers can be better prepared.  

There was also some unconfirmed discussion at trial about whether the grading standards were loosened by the third day because the success rate was so low.  Which would raise questions about how the grading criteria were established.  

A reliable test is one where a test taker’s score should not vary regardless of the conditions of the test - which includes what day they took the test.  All test takers must face the same test conditions for the test to be reliable.

Jeff Graham passed the practical test comfortably.

The Oral Exam

The oral exam was created especially for the 2012 exam, by Casey Johnson.  Oral interviews had been held for higher level positions, like captain, but not for the technical job of engineer.  The exam consisted of two parts:  1)  a technical part and 2)  a “peer review.”  This is the part that Jeff Graham failed, by one point.  This was the part that Graham complained about before the exams even began, after he was told he failed the exam, and later to the Alaska Human Rights Commission, and finally in court.  

The technical part consists of ten questions, supposedly about technical issues, though the 2012 exam had two questions about how to prepare for the test and some that were more AFD policy and administration questions.  

The ‘peer review’ consists of five questions that seem to be designed to determine whether someone’s character is good enough to become an engineer.  

There are five testers for the oral exam.  Engineers are asked to volunteer to assist with various parts of the Academy (the training program designed to prepare people for the exam).  The Academy administrators, in this case Chad Richardson and his assistant Casey Johnson, decide who will perform what duties at the Academy and in the exams.  They can also encourage people to apply, which at least a couple of the testers said happened to them.  

Before the exam takes place, the testers pre-grade the peer review part of the exam.  That means, they give each test taker a score based on their knowledge of that person.  There was mention of reviewing the application for promotion, but graders had different responses about whether these were considered.  If they have no knowledge of that person, they can leave that part blank. So, even though they, theoretically, had access to someone’s application, they could skip the pre-score, which suggests that either the application wasn’t important, they didn’t look at the applications, or prior personal knowledge of the candidate was the key factor.  There was conflicting testimony about when this pre-grading was done. Graders were asked to come in early and do things like score candidates on the pretest.  But testimony showed  that didn’t necessarily happen.  Pre-grade scoring could be done in the morning before the testers came in, before anyone was tested, or before each individual came in to be tested.  

The Peer Review test process

The candidate walks into the room.  He’s given his pre-scores on the peer review.  He then has an hour to answer 15 questions - the ten technical questions and the five character questions.  That gives someone about four minutes per question.  The questions are projected on a screen and the candidate begins answering them.  

If the candidate got a passing score on the peer review pre-score, he can elect to skip any or all of the peer review questions and spend more time on the technical questions.  This, obviously, gives an advantage to people who were pre-scored well.  

Jeff Graham’s pre-score grades were below the needed 70. He got 69.  He was surprised by this.  

Overall Test Scoring

To pass the engineer exam, candidates have to pass all three parts of the exam.  That means that if they fail any of the three parts of the exam (get less than 70%), they fail the whole exam - even if their overall average on all the exams was 71% or 80% or 89%.  

Since you have to get 70% or better on ALL three exams, if someone gets a 69 on the first exam (the written exam), they do not go on to take the practical exam.  Those who pass the written and then  pass the practical exam can go on to take the oral exam.  

I don’t have the exact numbers available, but a large number took the written exams and fewer took each succeeding exam.  At trial, Deputy Chief Hetrick said people who made it to the orals had good scores - around 85 or more - on the written exams.   

From Exam To Promotion

Passing the exam doesn’t mean a firefighter will be promoted to engineer.  Those who pass go on a list based on their scores.  The higher the score, the higher they are on the list.  When there is an opening for an engineer, the top person on the list is promoted.  The list is good for two years.  If all the people on the list are promoted before the two years are up and they have new openings, they can have an interim Academy and test.  

Anyone left on the list after two years is no longer eligible and has to take the whole exam again. 

The cost of Academies is quite high in money and in time. It is the full time job for at least two people (in 2012 Chad Richardson and Casey Johnson) for a period of time, plus the time of all the volunteers and all the candidates.  Then there is the equipment and other things used.  The practicing on various rigs and gas that takes.  One figure I heard was about $60,000 but I don’t have confirmation of that.  

The Meaning of the High Fail Rate

A lot of people go to the Academy and a relatively small number make it onto the promotion list each time.  We can’t be sure why so many fail, but there are several possible explanations that come to mind.  

1. People take the Academy to learn more about the promotion process and they might take the written test just to see how they do.  Sort of a  trial to gauge how difficult it is and how much they’ll have to study when they take it seriously.  People mentioned this was the case for some.
2. The quality of the firefighters is low.  Only a high school degree or a GED is required.  They may not be particularly good at studying and/or test taking.
3. The training at the academy is inadequate to prepare most people to pass the exams.  
4. The tests are necessarily rigorous to make sure only the most technically competent are promoted from firefighter to engineer.
5. The tests are unnecessarily difficult or harshly graded. 
6. Fewer engineers means more overtime for those who are engineers 

I suspect there is an aspect of all six reasons (and perhaps some I haven’t thought of.)  

Let me explain number six a little more.  Because of the 24 hour shifts several days a week, AFD line employees have a lot of time away from work.  Many use this time to run other businesses.  But for many others this is an opportunity to work overtime.  Not only does the overtime give them time-and-a-half pay, it also raises the annual pay that their retirement benefits will be based on.  Some have argued that by making the testing for engineer difficult, the pool of engineers is kept small, and those who become engineers can work more overtime.  At trial, the MOA’s expert witness hired to calculate possible compensation for Graham in the chance he won, testified that Graham had very little overtime compared to many who had 1000-2000 hours of overtime.  Firefighters work three days of 24 hour shifts per week.  1500 hours of overtime would be the equivalent of 6 weeks.  That’s a lot of overtime and a lot of pay at time and a half.  One has to ask whether hiring more employees would make overtime less necessary and save the MOA money.    

I realize this bit on overtime goes beyond just an overview of the exams, but I’m not sure where else it will fit in, and it gives context to questions that come up about the exams overall.  I’m raising this issue because it came up. I don't know how significant it is.  I haven’t studied it, but it seems like something that should be followed up on.  

The Devil is in the Details

The next posts on Graham v MOA will focus on the details of the oral exams.  

Crimes of violence tend to be very tangible, very graphic.  We can imagine someone with a gun threatening a mugging victim or a bank teller.  We can imagine a stabbing.  We can see it vividly.  These are crimes that involve people on people violations.  

But administrative crimes are much harder to imagine.  They are structural crimes that are less visible and easier to hide.  They are tied up in details, rows of numbers, pages of text. Easier to conceal.  

How a scoring sheet for a test is designed, can make the difference between whether someone passes the exam or not.  This is why we hear stories of people who have embezzled money for years and years before they were caught.  A petty thief can face much stiffer legal obstacles than a white collar criminal who has bilked people of millions.  The latter crime seems less problematic because it's so abstract and harder to visualize.  

The details can be tedious.  One reason I’m slow in getting these posts out is that I’m trying to make them as easy to read and understand as possible.  'Interesting' is a goal, but that's more elusive.  I keep revising and revising and eventually I say, ok, enough, just post it already.  Even though I’m sure it’s still hard for the average person with a busy schedule to read, let alone digest.