Graham v Municipality of Anchorage #6 - Test Makers Lack Certification

We're starting to get into the details.  Some might think "This seems minor" for some of these posts, but I hope to explain why they aren't minor.  But if you don't agree with me,  I'd also point out that a pattern of minor issues can collectively become a much more serious issue.

In this post I'm reporting the point that Jeff Graham's attorney made in court:  that the people who designed the training, the tests, and the grading procedures didn't have the training or the certification to do it right.

In court, Jeff Graham's attorney, Jeff Jarvi, gave Chad Richardson, the person in charge of the engineer academy (engineer is the step above a basic fire fighter and the academy is the training and testing program to promote to an engineer) a copy of his Fire Service Instructor Certificate and asked him to read off what level the certification was.  Chad Richardson read it:  Level I.  He was then asked what the difference between Level I and Level II was.  He didn't know.  He was then asked to read the above version of how the state certifying board distinguishes them.  Here's what Richardson read in court: 

From the State of Alaska Department of Public Safety (p. 3 of 16)

 AFSC Fire Service Instructor Certification Levels: 
Instructor I: (NFPA 1041 2012 ed., 3.3.2.1): An individual who has demonstrated the knowledge and ability to deliver instruction effectively from a prepared lesson plan, including instructional aids and evaluation instruments; adapt lesson plans to the unique requirements of the students and authority having jurisdiction; organize the learning environment so that learning is maximized; and meet the record-keeping requirements of authority having jurisdiction.  

Instructor II: (NFPA 1041 2012 ed., 3.3.2.2): An individual who has met the requirements for FSI Level I qualifications, and has demonstrated the knowledge and ability to develop individual lesson plans for a specific topic including learning objectives, instructional aids, and evaluation instruments; schedule training sessions based on overall training plan of authority having jurisdiction; and supervise and coordinate the activities of other instructors.  

If you missed the bold above, Level I is qualified to give training and tests someone else has designed.  You need Level II to create the training and the testing (including how the tests are graded.)

Jarvi did the same with Casey Johnson who prepared the exam.    And current Deputy Chief Jodie Hettrick who was then the new head of all training at the AFD.  They all only had Level I certification.  While I can understand the difficulty of coming into a new situation when the academy had already been planned, one thing Jodie Hettrick could have done in her initiation period was to check the certifications of the two in charge of the academy.  She had, after all, been in  charge of the state certification program  just before taking the job in Anchorage.  

Why does this matter?

First and foremost, the people in charge of testing didn't have the knowledge needed to create a valid and reliable exam and exam process necessary for a system based on merit principles, as required by  the MOA Charter at Section 5.06(c). 

Second, it undermines the credibility of the AFD's claims in court about how professional the fire department is.  The people preparing the exam didn't have the training or certification to do their jobs right.  There are also certifications for what different levels of paramedics can and cannot do.  Several people testified to not being able to perform certain functions because you needed a higher certification to perform it.  Are people ignoring those certifications too?  I suspect not, simply because they see that as their primary function, while they might see testing as 'merely' administrative.  I don't know for sure.  

Third, it raises questions about integrity, the area that Jeff Graham was marked down on, just enough in his oral exam, for him to fail to qualify for promotion.  I'll get into that matter later, but I want people to remember this as one of number of questions about integrity that make the judgment of Graham's integrity seem hypocritical and which I'm sure the jury didn't miss.  

On August 4, at the end of the MOA's defense, Deputy Chief Hettrick was called back to the stand by the MOA attorney and asked questions about the lack of proper certification.  
It was unfortunate, she told the court.  That State of Alaska didn't offer training in Anchorage for a number of years.  But it turns out there were people at AFD who had been certified at Level II when there still was training available in Anchorage.  

Jeff Jarvi, Graham's attorney, asked why people didn't go to Juneau or Outside for training, and Hettrick said it depended on available travel funds.  He then pointed out that Casey Johnson (who  created the exams)  had been allowed to testify at the trial out of sequence so he could attend a non-critical conference in North Carolina. That's four years since the exam and they still haven't gotten training.  

Below are some of my typed notes in the courtroom, so they aren't verbatim but close. 

MOA Attorney Monica Elkinton  "Are you saying that between 1996 and 2012 courses weren't offered?"
Hettrick:  Two small groups and no one from Anchorage took it.  There are no state statutes that require certification.  Local government makes that decision.  For police they have statutes.  There are recommendations, but without the instructor program, state couldn't do that. 
Elkinton:  Sounds like the authority having jurisdiction can prescribe what to do.  
Hettrick:  Yes

So, she's saying without a Municipal statue requiring training people to proper certification, it's ok if unqualified people make up the exams and grading procedures.

When it was Jeff Jarvi's turn to cross examine.

Jarvi:  Chief Dennis was a certified Level 2 right?  
Hettrick:  Yes
Jarvi:  Were there others?
Hettrick:  We have, I believe, less than 10.  
Jarvi:  There are others [with Level II Certification]?
Hettrick:  Yes
Jarvi:  Did they travel Outside to take the exam?
Hettrick:  No, they got it before 1996, but they can have them renewed.  The other half of that group attended the course we did in 2016.  . . . 
Jarvi:   We heard that Dennis did his in Juneau and Casey Johnson [Who created the test but was only certified at Level I] was accommodated in this trial to go to a conference in North Carolina?
Hettrick:  Yes.  


I would also note here - it's difficult to find the perfect place to put everything because some facts relate to more than one point, such as Casey Johnson's descriptions of how he himself had prepared for his own promotion exam.  MOA attorney Elkinton made one of her key arguments in the case that Graham didn't pass the test because he didn't study hard enough.*  To prove that point she asked various other people how hard they studied. 

Casey Johnson, in answering Elkinton's questions, said he studied many hours every day for months and months.  It was important for him to know everything.  I'd note that there was no real way to verify how much time he actually spent other than his word.  There were logs of people attending academy sessions or working with others in their stations, but not independent reading and studying.  

I raise this because Casey Johnson also said that training had become his passion.  Yet, if he was so dedicated to studying and memorizing as much as possible so he could pass the exams and do his job well, how is it that he didn't make sure he got the Level II Fire Safety Instructor training?  How is it that he didn't know about validity which was discussed in some detail in the national Fire Safety Instructor Training manual?  

*I do have to mention that Jeff Graham passed the written test and the practical test comfortably. So apparently he studied enough for those exams.  It was the highly subjective oral exam that he didn't pass. By one point.  I'll got into much more detail about what was wrong with that exam in future posts.