After every shooting and bombing, well, any event where one 'mentally ill' person manages to kill three or more people, we hear the refrain "everyone should have seen the signs before it came to this. It's a failure of the system. Guns aren't the problem, it's mental illness."
Well I'd like to report a guy who thinks he's president of the United States, which he believes, allows him to do whatever he wants. He hangs out with some truly strange people who urge him to do really bizarre stuff, but they tend not to stay for long. If they disagree with him on anything, he kicks them out.
He tweets threatening messages to people who disagree with him, whether they are members of Congress, judges, journalists, FBI, football players, former sex partners, ordinary citizens, or leaders of other countries.
And for him the truth is whatever he says it is at any given moment.
He has a suitcase, I'm told with which he can blow up the world. It has instructions on how to set off nuclear weapons. (One positive is that he doesn't like getting instructions from anyone.) He's got this huge cache of weapons and he's recently gotten approval for an additional nearly $700 billion in weapons and other war supplies. I think there should be a two year waiting period before he gets any of that.
I doubt there is any other single individual with more resources to do harm to the world, whether though actual arms and weapons or through destruction of the environment, civilization and community, and the good will that humans have been able to develop over the years. And his behavior is erratic and unpredictable. (I don't even think that's completely true - his specific actions may be, but the underlying motivation seems to be satisfying his own need for the moment.). He's unpredictable and decidedly misogynist.
So rather than be sorry, I think it's my duty as an American citizen and as a human being to report him. He needs serious psychological review. I
f Republicans in Congress are serious about stopping violence through catching the mentally ill who show signs they might commit acts of violence, it's time to review this man's stability, particularly given the stash of weapons he's collecting.
I predict a serious act of violence if he is allowed continued free reign (pun intended.) And then everyone left will be wringing their hands either saying, "who could have known" or "I always thought . . ."
Pages
- About this Blog
- AIFF 2024
- AK Redistricting 2020-2023
- Respiratory Virus Cases October 2023 - ?
- Why Making Sense Of Israel-Gaza Is So Hard
- Alaska Daily COVID-19 Count 3 - May 2021 - October 2023
- Alaska Daily COVID-19 Count - 2 (Oct. 2020-April 2021)
- Alaska Daily COVID-19 Count 1 (6/1-9/20)
- AIFF 2020
- AIFF 2019
- Graham v Municipality of Anchorage
- Favorite Posts
- Henry v MOA
- Anchorage Assembly Election April 2017
- Alaska Redistricting Board 2010-2013
- UA President Bonus Posts
- University of Alaska President Search 2015
Friday, March 23, 2018
Thursday, March 22, 2018
Tried Out My Bike Today - Conditions Were Better Than Expected And It Felt Great
It was brilliantly sunny, though the temps are back closer to normal. Warm enough in the afternoon that there was water from melting snow and ice. I wasn't sure how far the conditions would be ice-free and deep-puddle free, but I thought I go exploring.
Soon I found myself out on Dowling looking at the Chugach.
Then coming back on Elmore, I had a view from the first bridge of the dog mushing trail.
A little further and I was over the southern fork (I think) of Campbell Creek.
The next bridge gave me a view of my second moose since we got back to Anchorage last week.
As I said, for the most part the trail was ice free. There'd been one spot where a thick chunk was floating over a puddle, but there was a bit of room to go around it. But then, almost home, I got to this hard packed ice near Providence.
I know, there are people who ride all winter and deal with this sort of thing all the time. But I was away a lot of the winter and didn't get studs for my tires. And a couple of years ago around this time I found myself flat on my face, hard, after hitting a small patch of ice I didn't even see. So I navigate this stuff carefully.
It wasn't a long ride - maybe three miles - and I didn't go fast - I have a back fender but not a front one and didn't want to get too wet. But it felt great. And I'm guessing those hilly rides on Bainbridge made a difference. The much more modest hills today seemed like nothing.
Then it was back to filling out tax information. Really, I'd vote for a candidate who pushed for a much simplified tax code, one that didn't require people to hire someone to do their taxes for them. A progressive tax with no deductions - except maybe for folks on the poor end who'd had a catastrophic event. The tax rate could be modest then because people would actually pay that rate, especially those with much higher incomes.
I have to admit that these pictures make it look like I live in the wilderness. We have lots of urban wilderness of sorts, but I didn't take pictures of the more urban parts I passed. Maybe next time. I even passed a Walgreens and a YMCA.
Soon I found myself out on Dowling looking at the Chugach.
Then coming back on Elmore, I had a view from the first bridge of the dog mushing trail.
A little further and I was over the southern fork (I think) of Campbell Creek.
The next bridge gave me a view of my second moose since we got back to Anchorage last week.
As I said, for the most part the trail was ice free. There'd been one spot where a thick chunk was floating over a puddle, but there was a bit of room to go around it. But then, almost home, I got to this hard packed ice near Providence.
I know, there are people who ride all winter and deal with this sort of thing all the time. But I was away a lot of the winter and didn't get studs for my tires. And a couple of years ago around this time I found myself flat on my face, hard, after hitting a small patch of ice I didn't even see. So I navigate this stuff carefully.
It wasn't a long ride - maybe three miles - and I didn't go fast - I have a back fender but not a front one and didn't want to get too wet. But it felt great. And I'm guessing those hilly rides on Bainbridge made a difference. The much more modest hills today seemed like nothing.
Then it was back to filling out tax information. Really, I'd vote for a candidate who pushed for a much simplified tax code, one that didn't require people to hire someone to do their taxes for them. A progressive tax with no deductions - except maybe for folks on the poor end who'd had a catastrophic event. The tax rate could be modest then because people would actually pay that rate, especially those with much higher incomes.
I have to admit that these pictures make it look like I live in the wilderness. We have lots of urban wilderness of sorts, but I didn't take pictures of the more urban parts I passed. Maybe next time. I even passed a Walgreens and a YMCA.
Wednesday, March 21, 2018
One Reason Some Alaska Airlines Flights Are Priced Well Is Jet Blue - Whose Flight Attendants Began Voting On a Union This Week
Alaskans have benefited from Jet Blue flights to Anchorage, because Alaska Airlines responds with lower (than their normal) fares. One reason, people say, Jet Blue can fly for less is that their employees don't get paid as much because they aren't unionized.
But the flight attendants vote to unionize began Monday this week and goes through April 17.
Forbes seems to have the most coverage. Here's their latest.
And the pilots unionized four years ago. And Alaska seems to be able to match Jet Blue fares when they have to. So maybe unions aren't the problem.
My personal experience is that some union officials can be as stubborn and power hungry as some management officials. But just as there are decent managers, there are plenty of decent union folks and their job is to work for the workers. It's not just about wages, it's also about benefits, and having people on your side when management treats you badly.
But it doesn't always work - Jeff Graham's union didn't help him.
But the flight attendants vote to unionize began Monday this week and goes through April 17.
Forbes seems to have the most coverage. Here's their latest.
And the pilots unionized four years ago. And Alaska seems to be able to match Jet Blue fares when they have to. So maybe unions aren't the problem.
My personal experience is that some union officials can be as stubborn and power hungry as some management officials. But just as there are decent managers, there are plenty of decent union folks and their job is to work for the workers. It's not just about wages, it's also about benefits, and having people on your side when management treats you badly.
But it doesn't always work - Jeff Graham's union didn't help him.
Settling Back In At Home
Chopping ice, the violin concert, taxes, sorting old mail, getting the car tuned, arranging some home repairs, have all kept us busy. We only made it to the Thai Kitchen tonight for the first time, and only now saw our first moose since we got back last week. But the light has been spectacular.
Here's a view from in front of our house last night as the sun was setting behind me. I never get tired of this.
And walking through the Helen Louise McDowell sanctuary on the way back from dropping off my car.
Anchorage's first mail only election ends April 3. Waiting for my granddaughter to arrive so she can help me fill in the ballot and take it to one of the drop off centers (an option if you don't want to mail it.) At five she's a sponge, absorbing everything she touches, so I'm trying to give her as much to experience as I can. Need to find some little cross-country skis cause that's what she was most enthusiastic about when I asked what she wants to do here.
Tuesday, March 20, 2018
Graham v MOA #10: The Exams #3: The Oral Technical Exam
[All of the posts in this series are indexed at the "Graham v MOA' tab under the blog header above. Or you can just click here.]
The details are complicated so here's an overview of some key problems; details below.
The Details
As I mentioned in an earlier post, firefighters first have to pass a written exam on technical issues. Questions come from a national bank of questions that have been shown to be relevant to being an engineer (the next step above fire fighter).
Then comes the the practical exam. Here the firefighters have to get onto trucks and use equipment based on 'evolutions' (AFD's term) which are more or less scenarios that the firefighters have to role-play. This is the most hands-on part of the exam process.
If you pass these two tests, then you can go on to the oral exams - made up of ten questions of the technical oral exam and then of five questions of the 'peer review' oral exam. That's fifteen questions in an hour, or four minutes per question.
Jeff Graham passed the written and the practical exams. His dispute was over the oral exams which he complained about as being subjective and easy to manipulate. So let's, in this post, look at the problems with the technical part of the oral exams.
All Oral Exams, Including Interviews, Are Suspect
Before we even start on the oral technical exam, I need to reiterate this point I made in an earlier post.
Despite the popularity of job interviews, experts agree that they are among the most biased and result in the least accurate predictions of candidate job performane.
You can read the whole article or search the internet on this topic and find I'm not exaggerating or cherry picking. It's pretty much only people who conduct interviews who think they know what they're doing.
Or see this link. I offer a few quotes.
Excuse me for reiterating this point. It's one that is counter to people's perception and to practice. But it's important to make this point loud and clear from the beginning.
My Problems With the Exams
On the positive side, the AFD oral technical and peer review exams were structured. But there were numerous problems with the structure.
The Questions
There were ten questions on technical topics. My understanding of the conditions of my reviewing the exam questions themselves, is that I can't talk about the specific questions publicly. Only things that were discussed publicly in court.
There were ten technical questions. The MOA provided no evidence - though Graham's lawyer asked the MOA to provide documentation of how the exams were validated or even that the exam was validated. That is, we have no evidence to show that the ten questions were predictors of a candidate's likely success in the position of engineer. There is content related to the job, but AFD has produced no evidence showing that, for example, a candidate with a score of 90 will be a better engineer than a candidate with a score of 65. It may very well be that AFD may only be testing who has the best oral test taking skills.
Two of the ten were about how to prepare for the exam itself. The test creator said these were intended to be softball questions to relax candidates. Graham was scored low on them. The other questions were about things like how equipment worked and about AFD procedures for different things.
The Answers
For tests to be reliable, the graders need to be able to compare the candidate's answer to the ideal answer. The key points should be listed with the value of each point. The graders were given a package of answers. Some questions had no real answers attached. Most looked like they were cut and pasted together from text books or AFD policy and procedure manuals. For one question, I timed myself reading the four pages of answer that were provided. It took me 11 minutes and 30 seconds just to read the answer. But there are only about four minutes available per question. How can a grader reliably a) listen to the candidate's answer, b) take notes, and c) read through the answer sheet to match what the candidate said to the expected correct answer? He can't. Graham appears to have done better on the few questions that had bullet points on the provided answers rather than several pages raw material.
Assuming that the question itself is valid, the answer sheets graders had should have had a list of key points that the candidate should mention for each question. The best such answer sheets would identify the most important points that should be in the response with a score for each. There was nothing like that. Instead the graders had to balance the answer the candidate gave, the pages of 'answer' copied out of the text or manual, and give the candidate a score on a completely different score sheet.
The Grading Scale
The details are complicated so here's an overview of some key problems; details below.
- Overall, oral exams (including interviews) are highly invalid and unreliable predictors of success on the job, despite the fact that they are used a lot.
- Imagine a question about how to do something fairly complex like, "tell me what you know about brakes." Where do you start? You have four minutes. Go.
- The graders may or may not understand the answer, but they have four or five pages taken from a textbook, they can scan through, with nothing to indicate what's important and what isn't.
- The grading scale has uneven intervals - 5, 4.5, 3.5, 2.5, 1.5, 0 - with 3.5 being the lowest possible pass. This odd scale is not what people are used to and skews the grades downward.
- You have to answer each question, they have to assess your answer, all in about four minutes.
- Lack of Accountability. There's no recording, no way to check to see what a candidate actually said.
The Details
As I mentioned in an earlier post, firefighters first have to pass a written exam on technical issues. Questions come from a national bank of questions that have been shown to be relevant to being an engineer (the next step above fire fighter).
Then comes the the practical exam. Here the firefighters have to get onto trucks and use equipment based on 'evolutions' (AFD's term) which are more or less scenarios that the firefighters have to role-play. This is the most hands-on part of the exam process.
If you pass these two tests, then you can go on to the oral exams - made up of ten questions of the technical oral exam and then of five questions of the 'peer review' oral exam. That's fifteen questions in an hour, or four minutes per question.
Jeff Graham passed the written and the practical exams. His dispute was over the oral exams which he complained about as being subjective and easy to manipulate. So let's, in this post, look at the problems with the technical part of the oral exams.
All Oral Exams, Including Interviews, Are Suspect
Before we even start on the oral technical exam, I need to reiterate this point I made in an earlier post.
Despite the popularity of job interviews, experts agree that they are among the most biased and result in the least accurate predictions of candidate job performane.
You can read the whole article or search the internet on this topic and find I'm not exaggerating or cherry picking. It's pretty much only people who conduct interviews who think they know what they're doing.
" interviewers typically form strong but unwarranted impressions about interviewees, often revealing more about themselves than the candidates."
"Research that my colleagues and I have conducted shows that the problem with interviews is worse than irrelevance: They can be harmful, undercutting the impact of other, more valuable information about interviewees."
Or see this link. I offer a few quotes.
"Consider the job interview: it’s not only a tiny sample, it’s not even a sample of job behaviour but of something else entirely. Extroverts in general do better in interviews than introverts, but for many if not most jobs, extroversion is not what we’re looking for. Psychological theory and data show that we are incapable of treating the interview data as little more than unreliable gossip. It’s just too compelling that we’ve learned a lot from those 30 minutes."Some of these comments are made about unstructured interviews. The AFD engineer exam was a structured interview which researchers find to be a little better. But still not that good. From one of the most well known academics writing on human resources management, Edward E. Lawler III:
"Years of research on job interviews has shown that they are poor predictors of who will be a good employee. There are many reasons for this, but perhaps the key explanation is that individuals simply don’t gather the right information and don’t understand what the best predictors of job success are. A careful analysis of the background of individuals and their work history and work samples are more accurate predictors of success on the job than are the judgments which result from interviews."The graders in the AFD oral exam did not have background information on the candidates' performance, they didn't review the performance reviews done by the candidates' supervisors.
Excuse me for reiterating this point. It's one that is counter to people's perception and to practice. But it's important to make this point loud and clear from the beginning.
My Problems With the Exams
On the positive side, the AFD oral technical and peer review exams were structured. But there were numerous problems with the structure.
The Questions
There were ten questions on technical topics. My understanding of the conditions of my reviewing the exam questions themselves, is that I can't talk about the specific questions publicly. Only things that were discussed publicly in court.
There were ten technical questions. The MOA provided no evidence - though Graham's lawyer asked the MOA to provide documentation of how the exams were validated or even that the exam was validated. That is, we have no evidence to show that the ten questions were predictors of a candidate's likely success in the position of engineer. There is content related to the job, but AFD has produced no evidence showing that, for example, a candidate with a score of 90 will be a better engineer than a candidate with a score of 65. It may very well be that AFD may only be testing who has the best oral test taking skills.
Two of the ten were about how to prepare for the exam itself. The test creator said these were intended to be softball questions to relax candidates. Graham was scored low on them. The other questions were about things like how equipment worked and about AFD procedures for different things.
The Answers
For tests to be reliable, the graders need to be able to compare the candidate's answer to the ideal answer. The key points should be listed with the value of each point. The graders were given a package of answers. Some questions had no real answers attached. Most looked like they were cut and pasted together from text books or AFD policy and procedure manuals. For one question, I timed myself reading the four pages of answer that were provided. It took me 11 minutes and 30 seconds just to read the answer. But there are only about four minutes available per question. How can a grader reliably a) listen to the candidate's answer, b) take notes, and c) read through the answer sheet to match what the candidate said to the expected correct answer? He can't. Graham appears to have done better on the few questions that had bullet points on the provided answers rather than several pages raw material.
Assuming that the question itself is valid, the answer sheets graders had should have had a list of key points that the candidate should mention for each question. The best such answer sheets would identify the most important points that should be in the response with a score for each. There was nothing like that. Instead the graders had to balance the answer the candidate gave, the pages of 'answer' copied out of the text or manual, and give the candidate a score on a completely different score sheet.
The Grading Scale
Rubric For Oral Technical Exam 5 - Outstanding, exemplary performance far exceeds expectations 4.5 - Above average performance exceeds level of basic skills/abilities 3.5 - Adequately demonstrated the basic abilities/skills 2.5 - Needs improvement, falls short of what is expected 1.5 - Unsatisfactory, performance is substandard 0 - Unacceptable, does not demonstrate comprehensive and/or application of required skills sets |
Note: There's a half point difference between "Outstanding" and "Above average". Then it drops, not .5 like from 5 to 4.5, but 1 point to 3.5 "Adequate". So a grader could think, ok, this is good enough, it's adequate. But 3.5 isn't 'adequate' it's really 'just barely adequate." It's the lowest possible passing score. The top two scores are very high marks. The next one is barely passing. 1.5 is "unsatisfactory", but 3.5 (the same interval from 5 as 1.5 is from zero
But on the bottom of the scale, it goes from 0 to 1.5 - both of which are almost equivalent failing grades. But the 1.5 is the same interval as from 5 to 3.5.
1. This scale has uneven intervals. That is, the distance between the points on the scale are not the same. Look closely. The top two scores are both strong passing scores and the bottom two scores are both very poor failing scores. But the top two are separated by .5 points, while the bottom two are separated by 1.5 points. That's the same interval as from 5 to 3.5.
If 1.5 is 'unsatisfactory' the 3.5 should be "satisfactory' but it's only 'adequate' and more accurately 'barely adequate' because 3.5 is the lowest you can get and still pass. 3.4 is a failing grade. (It's less than the 70% (20*3.5) needed to pass.)
The scale skews the scores down. The points graders can mark go from 100% (5) to 90% (4.5) to 70% (3.5) which is the lowest possible passing score. Why isn't there an 80% (4)? That's what normal scales with equal intervals would have next.
2. Passing on this test is 3.5. It took me a bit to figure that out, but 70% is the passing score, so each of these numbers need to be multiplied by 20 to get the actual score. 70% (3.5*20) is the minimal passing score. Let's compare that to other scoring forms you know. Say the standard Likert scale on most surveys:
I chose this 5 point scale, because the AFD calls its scale a five point scale. But if you look at it, it's really a six point scale, since it has six possible scores from 0, 1.5, 2.5, 3.5, 4.5, and 5. They didn't even realize they had a six point scale! In most six-point Likert scales, there are three positive and three negative options. On such a scale (5, 4, 3, 2, 1, 0) a 3.5 would be a strong passing score, not barely passing.
Let's look at it again from a different angle. In the common academic scoring system, 90-100 is an A, 80-89 is a B, 70-79 is a C, 60-69 is a D, and below 60 is an F.
But in the AFD system, below 70 is an F. On the AFD scale the first two numbers (a ten percent interval) are both very good scores. The next score (20 percent less) is barely passing. All the rest are failing scores.
Why does this matter? First, it matters because the intervals are not equal. It's not common to have uneven intervals so that the distance between one score and another can vary from 10% (5 to 4.5) to 20% (4.5 to 3.5) to 30% (1.5 to 0).
It matters because the scale isn't like any scale we normally see and so the graders don't have a good sense of how skewed it is toward failing candidates. The 3.5 isn't 'good', it's barely passing. Yet most people are used to the academic scoring system and would assume that 70% would be something like a low C. There is no 'good' on this scale. There's 'walks on water,' 'really outstanding,' then 'barely passing'. It took me a while to understand why the scale seemed so off. I don't imagine an AFD grader would figure this out in the time they have to grade. There isn't enough time to even figure out what the right score should be, let alone analyze the scale to see how skewed it is.
Reliability
Tests have to be both valid and reliable. Valid means a high score predicts success (in this case, 'will perform the job well') and a low score predicts (in this case) 'will perform the job poorly'. Even if the test itself is valid (and the AFD never did have anyone test for validity), they also have to be reliable if they will be accurate predictors. Without reliability, the scores aren't accurate, and so the test is no longer a good predictor (no longer valid.)
Reliable means that the same candidate taking the test under different conditions (different time, different place, different graders) would essentially get the same score.
Given the lack of clear answers for the graders to use to compare the candidate's answer to, and given the lack of a clear rating system, it's not likely that this is a reliable test.
And in fact, when I looked at the scores individuals got on the same question by different graders, there were some big differences in score. Some were a point off, some were 2 points off and even 2.5 points off. There were even a few that were 3 points off.
That may not sound like much, but on a six point scale, it's a lot. Even though they actually used a six point scale, they said that 3.5 was equal to 70% - the lowest passing grade. So each point was worth 20%. Therefore, two points equals a 40% difference. 2.5 equals a 50% difference on the score. 3 equals 60%. If this test were reliable, the scores by different graders shouldn't vary by more than 10% or so. That would be half a point on any question.
On this exam, graders gave scores that were up to 60% (3 points) different for the same candidate on the same question! That's the difference - in an academic setting - between a 95% (A) and 35% (low F). But even a one point difference is a 20% difference - that could be a 90% and a 70% or the difference between an A- and a C-. That's a huge spread, and it's there because the answer sheets don't tell the graders what a good answer looks like, and the grading scale easily tricks graders into giving lower ratings than they realize they are giving.
The two people in charge of setting up the tests (and the overall training director at the time) were actually not qualified to prepare the tests. They all had Alaska Fire Safety Training Instructor Certificate Level 1. They were eligible to administer training and tests that had been prepared by someone else, someone with a Level II certificate. Here's the earlier post on the certifications.
Accountability
The exam is an hour long. Some of the graders wrote brief notes on their score sheets, but there isn't much room. Those notes are the only record of what actually happened in the oral exam. There's no way for the candidate or the graders to check what was actually said. Even as they review the pages that contain the answer, they can't replay the tape to see if the candidate covered something or not. And years later? At trial? Well I know that five minutes later my wife and I have totally different recollections of what each of us said. And in trial, there were a number of occasions when Jeff Jarvi, Jeff Graham's attorney, asked the graders about something they had said at their depositions or at the State Human Rights Commission hearing. They'd answer. Then Jarviv would have them read the transcripts, and the transcripts were not at all what they remembered.
Video or audio recordings of the oral exam probably could have averted this whole trial. The tapes would have demonstrated whether Jeff Graham had covered the points as he said he did, or that he didn't as some of the graders said.
Not having recordings means no one can check how well the graders did their jobs. Or whether people with low scores really did answer less accurately than people with high scores. And it means that candidates can't get feedback on how they did so they can better prepare for the next exam. (I'd note, that as a professor, the most convincing way to demonstrate to students that his paper wasn't very good, was to have them read the best papers. It worked every time.)
Unequal Conditions
One other twist here. This takes us ahead to the oral peer review, but it affects how people do on the technical as well. Prior to the oral peer review, candidates are 'pre-scored' by the graders based on their personal knowledge of the candidates. Imagine random co-workers some who might work closely with you and others who don't, evaluating you based on, well, whatever they want. No one asks the basis. It's not based on written materials they have about your work history, because there is the option of N/O - not observed. If there were some written documentation, they wouldn't need the N/O option.
When the candidate walks into the room, he (it's almost always a he) is given his pre-scores. If you pass (get over 70%) you can choose to spend more time on the technical questions and skip the peer-review questions altogether. You can gain 20 minutes.
Imagine the emotional impact of being told, before you even start the oral technical exam, that the graders had failed you on the pre-scoring of the peer-review. Or that they passed you with a high score. In Jeff's case, he was stunned to learn he'd been failed by the graders in the pre-scoring of the peer-review. It set him back as the technical oral began.
You've Made It To The End
If you made it this far, congratulations. Going through details like this is necessary to truly get a sense of how badly these exams were designed and implemented. But it does take work. Thanks for hanging in there. These are the kinds of details that jury had to sit through over the three week plus trial.
This post covered the technical oral exam problems and the next post will cover the oral 'peer review' exam that was part of the one hour along with the technical oral exam.
1. This scale has uneven intervals. That is, the distance between the points on the scale are not the same. Look closely. The top two scores are both strong passing scores and the bottom two scores are both very poor failing scores. But the top two are separated by .5 points, while the bottom two are separated by 1.5 points. That's the same interval as from 5 to 3.5.
If 1.5 is 'unsatisfactory' the 3.5 should be "satisfactory' but it's only 'adequate' and more accurately 'barely adequate' because 3.5 is the lowest you can get and still pass. 3.4 is a failing grade. (It's less than the 70% (20*3.5) needed to pass.)
The scale skews the scores down. The points graders can mark go from 100% (5) to 90% (4.5) to 70% (3.5) which is the lowest possible passing score. Why isn't there an 80% (4)? That's what normal scales with equal intervals would have next.
2. Passing on this test is 3.5. It took me a bit to figure that out, but 70% is the passing score, so each of these numbers need to be multiplied by 20 to get the actual score. 70% (3.5*20) is the minimal passing score. Let's compare that to other scoring forms you know. Say the standard Likert scale on most surveys:
"How do you feel about the Principal's performance this year?Note that here a score of 3 is in the middle and is neutral, whereas in the AFD rubric, 3.5 is the lowest passing score. The 3 or neutral would have a range from 3.5 to 2.5. So a 3.5 would be on the high end, not the low end as it is on the AFD scale.
5 – strongly approve
4 – somewhat approve
3 – neutral/no opinion
2 – somewhat disapprove
1 – strongly disapprove" (from Brighthub Education though it had the scale reversed)
I chose this 5 point scale, because the AFD calls its scale a five point scale. But if you look at it, it's really a six point scale, since it has six possible scores from 0, 1.5, 2.5, 3.5, 4.5, and 5. They didn't even realize they had a six point scale! In most six-point Likert scales, there are three positive and three negative options. On such a scale (5, 4, 3, 2, 1, 0) a 3.5 would be a strong passing score, not barely passing.
Let's look at it again from a different angle. In the common academic scoring system, 90-100 is an A, 80-89 is a B, 70-79 is a C, 60-69 is a D, and below 60 is an F.
But in the AFD system, below 70 is an F. On the AFD scale the first two numbers (a ten percent interval) are both very good scores. The next score (20 percent less) is barely passing. All the rest are failing scores.
Why does this matter? First, it matters because the intervals are not equal. It's not common to have uneven intervals so that the distance between one score and another can vary from 10% (5 to 4.5) to 20% (4.5 to 3.5) to 30% (1.5 to 0).
It matters because the scale isn't like any scale we normally see and so the graders don't have a good sense of how skewed it is toward failing candidates. The 3.5 isn't 'good', it's barely passing. Yet most people are used to the academic scoring system and would assume that 70% would be something like a low C. There is no 'good' on this scale. There's 'walks on water,' 'really outstanding,' then 'barely passing'. It took me a while to understand why the scale seemed so off. I don't imagine an AFD grader would figure this out in the time they have to grade. There isn't enough time to even figure out what the right score should be, let alone analyze the scale to see how skewed it is.
Reliability
Tests have to be both valid and reliable. Valid means a high score predicts success (in this case, 'will perform the job well') and a low score predicts (in this case) 'will perform the job poorly'. Even if the test itself is valid (and the AFD never did have anyone test for validity), they also have to be reliable if they will be accurate predictors. Without reliability, the scores aren't accurate, and so the test is no longer a good predictor (no longer valid.)
Reliable means that the same candidate taking the test under different conditions (different time, different place, different graders) would essentially get the same score.
Given the lack of clear answers for the graders to use to compare the candidate's answer to, and given the lack of a clear rating system, it's not likely that this is a reliable test.
And in fact, when I looked at the scores individuals got on the same question by different graders, there were some big differences in score. Some were a point off, some were 2 points off and even 2.5 points off. There were even a few that were 3 points off.
That may not sound like much, but on a six point scale, it's a lot. Even though they actually used a six point scale, they said that 3.5 was equal to 70% - the lowest passing grade. So each point was worth 20%. Therefore, two points equals a 40% difference. 2.5 equals a 50% difference on the score. 3 equals 60%. If this test were reliable, the scores by different graders shouldn't vary by more than 10% or so. That would be half a point on any question.
On this exam, graders gave scores that were up to 60% (3 points) different for the same candidate on the same question! That's the difference - in an academic setting - between a 95% (A) and 35% (low F). But even a one point difference is a 20% difference - that could be a 90% and a 70% or the difference between an A- and a C-. That's a huge spread, and it's there because the answer sheets don't tell the graders what a good answer looks like, and the grading scale easily tricks graders into giving lower ratings than they realize they are giving.
The two people in charge of setting up the tests (and the overall training director at the time) were actually not qualified to prepare the tests. They all had Alaska Fire Safety Training Instructor Certificate Level 1. They were eligible to administer training and tests that had been prepared by someone else, someone with a Level II certificate. Here's the earlier post on the certifications.
Accountability
The exam is an hour long. Some of the graders wrote brief notes on their score sheets, but there isn't much room. Those notes are the only record of what actually happened in the oral exam. There's no way for the candidate or the graders to check what was actually said. Even as they review the pages that contain the answer, they can't replay the tape to see if the candidate covered something or not. And years later? At trial? Well I know that five minutes later my wife and I have totally different recollections of what each of us said. And in trial, there were a number of occasions when Jeff Jarvi, Jeff Graham's attorney, asked the graders about something they had said at their depositions or at the State Human Rights Commission hearing. They'd answer. Then Jarviv would have them read the transcripts, and the transcripts were not at all what they remembered.
Video or audio recordings of the oral exam probably could have averted this whole trial. The tapes would have demonstrated whether Jeff Graham had covered the points as he said he did, or that he didn't as some of the graders said.
Not having recordings means no one can check how well the graders did their jobs. Or whether people with low scores really did answer less accurately than people with high scores. And it means that candidates can't get feedback on how they did so they can better prepare for the next exam. (I'd note, that as a professor, the most convincing way to demonstrate to students that his paper wasn't very good, was to have them read the best papers. It worked every time.)
Unequal Conditions
One other twist here. This takes us ahead to the oral peer review, but it affects how people do on the technical as well. Prior to the oral peer review, candidates are 'pre-scored' by the graders based on their personal knowledge of the candidates. Imagine random co-workers some who might work closely with you and others who don't, evaluating you based on, well, whatever they want. No one asks the basis. It's not based on written materials they have about your work history, because there is the option of N/O - not observed. If there were some written documentation, they wouldn't need the N/O option.
When the candidate walks into the room, he (it's almost always a he) is given his pre-scores. If you pass (get over 70%) you can choose to spend more time on the technical questions and skip the peer-review questions altogether. You can gain 20 minutes.
Imagine the emotional impact of being told, before you even start the oral technical exam, that the graders had failed you on the pre-scoring of the peer-review. Or that they passed you with a high score. In Jeff's case, he was stunned to learn he'd been failed by the graders in the pre-scoring of the peer-review. It set him back as the technical oral began.
You've Made It To The End
If you made it this far, congratulations. Going through details like this is necessary to truly get a sense of how badly these exams were designed and implemented. But it does take work. Thanks for hanging in there. These are the kinds of details that jury had to sit through over the three week plus trial.
This post covered the technical oral exam problems and the next post will cover the oral 'peer review' exam that was part of the one hour along with the technical oral exam.
Labels:
AFD,
Graham v. MOA,
Justice,
Knowing,
test
Monday, March 19, 2018
Rights Of Children Of Anonymous Sperm Donors; Stopping Gerrymandering; Montana History; And Cambridge Analytica FB Harvesting Explained
While I'm busy on other things, here are some interesting articles to keep you busy.
Anonymous sperm and egg donation has consequences for donor offspring like me.
By Courtney McKinney - In adoptions, the child's welfare is the major factor in placement decisions. But McKinney points out that in anonymous sperm donor babies, the child's interest is not a factor. The article looks at the implications and possible remedies.
The Geeks Who Put A Stop To Pennsylvania's Gerrymandering - This article explains in pretty clear language how the experts demonstrated to the Court that the Republicans had intentionally packed and cracked districts so that Republicans got 72% of the seats with only 49% of the vote. It's a good roadmap of how to quantitatively and qualitatively demonstrate gerrymandering and intent.
I'd note that as of March 19, 2018, the US Supreme Court rejected the Republicans second attempt to block the Pennsylvania Supreme Court's ruling.
Montana Historical Landscape - this blogger posts pictures of historical buildings and sites around Montana grouped into categories - by location, function (restaurants in historic buildings, public gymnasiums). A great, specialized site. I got there because google said I had a lot of links from the site, but I couldn't find anything that linked back to my blog. My post that most fits with what is done there is one on Clinton, BC.
Anonymous sperm and egg donation has consequences for donor offspring like me.
By Courtney McKinney - In adoptions, the child's welfare is the major factor in placement decisions. But McKinney points out that in anonymous sperm donor babies, the child's interest is not a factor. The article looks at the implications and possible remedies.
The Geeks Who Put A Stop To Pennsylvania's Gerrymandering - This article explains in pretty clear language how the experts demonstrated to the Court that the Republicans had intentionally packed and cracked districts so that Republicans got 72% of the seats with only 49% of the vote. It's a good roadmap of how to quantitatively and qualitatively demonstrate gerrymandering and intent.
I'd note that as of March 19, 2018, the US Supreme Court rejected the Republicans second attempt to block the Pennsylvania Supreme Court's ruling.
Montana Historical Landscape - this blogger posts pictures of historical buildings and sites around Montana grouped into categories - by location, function (restaurants in historic buildings, public gymnasiums). A great, specialized site. I got there because google said I had a lot of links from the site, but I couldn't find anything that linked back to my blog. My post that most fits with what is done there is one on Clinton, BC.
Cambridge Analytica whistleblower: 'We spent $1m harvesting millions of Facebook profiles' – video - watch Christopher Wylie explain what he and Cambridge Analytica did (and also explain how and why they got that name.) Take it with a grain of salt, but it's interesting and pretty clear.
Sunday, March 18, 2018
We've Been Here Before - Time To Contact Your Republican Congress Members To Protect Mueller
For those who weren't around, and those who didn't get the meaning of all this when they read it in their history. From the Washington Post: Watergate 25:
I can believe that there aren't 50,000 people who are seriously concerned about the possibility of Robert Mueller being fired by Trump. (That's what Trump does best, right?) And I bet those 50,000 could each write a quick email, letter, or leave a phone message with their Republican Congress members. There are currently 238 Republican members of the House and 51 Republican Senators.
50,000 letters to Republican members of Congress (house and senate) would - if they were evenly distributed - come to about 103 messages per member. I have three Republicans representing me, so I have to send three messages so we'd probably get more than that per Senator and House member.
Californians and other deeply blue state residents - send your folks letters to share with their Republican colleagues.
Doesn't have to be complicated. "Don't allow Trump to get away with firing Mueller." That's the basic message. My emails to my two Senators and my Representative called on them to work with their Republican colleagues to let the president know they would not stand for his firing Mueller, and if he does, to start impeachment hearings.
I'm sure many of the high school students who walked out on Wednesday for an end to school shootings can see the relationship between this issue and theirs. Let them join the cause.
You can find your House member contact info here.
You can find your Senator contact member here. (top left)
I'd note that despite the Saturday massacre, impeachment hearings proceeded and Nixon eventually resigned rather than get removed by Congress.
But if Republicans don't stand up to Trump if and when he fires Mueller, they never will. And Trump has until November to mess with the elections.
[UPDATE March 30, 2018 - Sen. Dan Sullivan's office responded within a few days - you can see his response at this post.]
Nixon Forces Firing of Cox; Richardson, Ruckelshaus QuitYou can read the rest here.
President Abolishes Prosecutor's Office; FBI Seals Records
By Carroll Kilpatrick
Washington Post Staff Writer
Sunday, October 21, 1973; Page A01
In the most traumatic government upheaval of the Watergate crisis, President Nixon yesterday discharged Special Prosecutor Archibald Cox and accepted the resignations of Attorney General Elliot L. Richardson and Deputy Attorney General William D. Ruckelshaus.
The President also abolished the office of the special prosecutor and turned over to the Justice Department the entire responsibility for further investigation and prosecution of suspects and defendants in Watergate and related cases.
Shortly after the White House announcement, FBI agents sealed off the offices of Richardson and Ruckelshaus in the Justice Department and at Cox's headquarters in an office building on K Street NW.
An FBI spokesman said the agents moved in "at the request of the White House."
Agents told staff members in Cox's office they would be allowed to take out only personal papers. A Justice Department official said the FBI agents and building guards at Richardson's and Ruckelshaus' offices were there "to be sure that nothing was taken out."
Richardson resigned when Mr. Nixon instructed him to fire Cox and Richardson refused. When the President then asked Ruckelshaus to dismiss Cox, he refused, White House spokesman Ronald L. Ziegler said, and he was fired. Ruckelshaus said he resigned.
Finally, the President turned to Solicitor General Robert H. Bork, who by law becomes acting Attorney General when the Attorney General and deputy attorney general are absent, and he carried out the President's order to fire Cox. The letter from the President to Bork also said Ruckelshaus resigned.
These dramatic developments were announced at the White House at 8:25 p.m. after Cox had refused to accept or comply with the terms of an agreement worked out by the President and the Senate Watergate committee under which summarized material from the White House Watergate tapes would be turned over to Cox and the Senate committee.
In announcing the plan Friday night, the President ordered Cox to make no further effort to obtain tapes or other presidential documents. , ,
I can believe that there aren't 50,000 people who are seriously concerned about the possibility of Robert Mueller being fired by Trump. (That's what Trump does best, right?) And I bet those 50,000 could each write a quick email, letter, or leave a phone message with their Republican Congress members. There are currently 238 Republican members of the House and 51 Republican Senators.
50,000 letters to Republican members of Congress (house and senate) would - if they were evenly distributed - come to about 103 messages per member. I have three Republicans representing me, so I have to send three messages so we'd probably get more than that per Senator and House member.
Californians and other deeply blue state residents - send your folks letters to share with their Republican colleagues.
Doesn't have to be complicated. "Don't allow Trump to get away with firing Mueller." That's the basic message. My emails to my two Senators and my Representative called on them to work with their Republican colleagues to let the president know they would not stand for his firing Mueller, and if he does, to start impeachment hearings.
I'm sure many of the high school students who walked out on Wednesday for an end to school shootings can see the relationship between this issue and theirs. Let them join the cause.
You can find your House member contact info here.
You can find your Senator contact member here. (top left)
I'd note that despite the Saturday massacre, impeachment hearings proceeded and Nixon eventually resigned rather than get removed by Congress.
But if Republicans don't stand up to Trump if and when he fires Mueller, they never will. And Trump has until November to mess with the elections.
[UPDATE March 30, 2018 - Sen. Dan Sullivan's office responded within a few days - you can see his response at this post.]
Saturday, March 17, 2018
Why I Live Here - Intimate Evenings With Incredible People: Dainis Medjaniks [Corrected]
Last night we heard a violin come alive - coaxed by a very talented violinist, music of great composers, and a room in someone's house that gave the notes space to fly and dance in the air and in our heads.
Although i was close enough to see the violinist's expressions as he played, I spent much of the concert with my eyes closed entranced by the intricate ballet of sounds that surrounded me. The violin truly showed its soul, tempted by the perfect set of conditions.
The Back Story
Anchorage is a small town, really. About 300,000 people. And Alaska is a state most everyone wants to visit, even if it's only to say "I've been to all 50 states." So lots of brilliant and talented people come here and often they perform or give talks or meet with the public. So over the 40 years I've lived here I've had the chance to meet with people I never would have met, say, if I had stayed in Los Angeles.
Last night was one of those occasions. A friend invited us to a concert by young Latvian violinist. The one page program we got last night included the pieces he was going to play on one side and a biography of accomplishments of Medjaniks in his short career.
When the emailed invitation came a couple of weeks ago saying that if we wanted to come, we should reserve soon, seats were very limited. It sounded like the perfect Anchorage concert - an international musician in a small intimate session. The contribution was modest and all would go to the artist. AND we would be back in town. Couldn't pass it up.
This picture is posed, after the concert. Even if I had my small camera with me - I left it in LA and it's supposed to be coming home courtesy of the US postal service - this was not the night to intrude. I did borrow my wife's phone to get these pictures before and after he played.
This was a night to bathe in the magic vibrations that flowed from Dainis' violin, not to be distracted by a camera. An audio night, not a visual night (though the setting was beautiful.) The evening's program includes this note about the violin:
[CORRECTION - March 20, 2018: This is a great story, but I learned a couple of days later, it was old information. Dainis had been lent that old violin for several years, but the time was up and the violin he had with him here was a new one - made by the mother of the cellist in a group he plays with. That one is on loan too.]
And I knew that nothing I could do would come close to capturing the experience, but I did find this video online of Dainis in 2015.
Dainis Medjaniks - Preliminary Round 1 from JosephJoachimTV on Vimeo.
As is the custom in Anchorage, guests left their shoes at the door, including Dainis. I asked him if this was the first time he performed before an audience in his socks, and he smiled and said, "Yes."
Thank you to the host whose home was perfect for this concert and whose hospitality is greatly appreciated.
Although i was close enough to see the violinist's expressions as he played, I spent much of the concert with my eyes closed entranced by the intricate ballet of sounds that surrounded me. The violin truly showed its soul, tempted by the perfect set of conditions.
The Back Story
Anchorage is a small town, really. About 300,000 people. And Alaska is a state most everyone wants to visit, even if it's only to say "I've been to all 50 states." So lots of brilliant and talented people come here and often they perform or give talks or meet with the public. So over the 40 years I've lived here I've had the chance to meet with people I never would have met, say, if I had stayed in Los Angeles.
Last night was one of those occasions. A friend invited us to a concert by young Latvian violinist. The one page program we got last night included the pieces he was going to play on one side and a biography of accomplishments of Medjaniks in his short career.
When the emailed invitation came a couple of weeks ago saying that if we wanted to come, we should reserve soon, seats were very limited. It sounded like the perfect Anchorage concert - an international musician in a small intimate session. The contribution was modest and all would go to the artist. AND we would be back in town. Couldn't pass it up.
This picture is posed, after the concert. Even if I had my small camera with me - I left it in LA and it's supposed to be coming home courtesy of the US postal service - this was not the night to intrude. I did borrow my wife's phone to get these pictures before and after he played.
This was a night to bathe in the magic vibrations that flowed from Dainis' violin, not to be distracted by a camera. An audio night, not a visual night (though the setting was beautiful.) The evening's program includes this note about the violin:
"Sebastian Klozt of Mittenwald, Germany made the violin Dainis plays in 1782. The violin from the Foundation of Lower Saxony is on loan to him."If you aren't the calculating type, it was born six years after the Declaration of Independence was signed, when George Washington, Thomas Jefferson, Benjamin Franklin were fighting for the United States' independence. Before Napoleon ruled France. It's aged very well.
[CORRECTION - March 20, 2018: This is a great story, but I learned a couple of days later, it was old information. Dainis had been lent that old violin for several years, but the time was up and the violin he had with him here was a new one - made by the mother of the cellist in a group he plays with. That one is on loan too.]
And I knew that nothing I could do would come close to capturing the experience, but I did find this video online of Dainis in 2015.
Dainis Medjaniks - Preliminary Round 1 from JosephJoachimTV on Vimeo.
As is the custom in Anchorage, guests left their shoes at the door, including Dainis. I asked him if this was the first time he performed before an audience in his socks, and he smiled and said, "Yes."
Thank you to the host whose home was perfect for this concert and whose hospitality is greatly appreciated.
Labels:
Anchorage,
music,
Why I Live Here
Friday, March 16, 2018
Back In Anchorage
We left a sunny Bainbridge by ferry
Walked up to the train station to catch the light rail to the airport. Flew out of Seattle and over clouds most of the way home and arrived about 8 pm over mostly cloudy Cook Inlet, but with hints of the sun.
There was sun yesterday morning, but wet snow by evening. It is breakup - lots of puddles in the streets. Temperature now is 35˚F on our backyard thermometer and the snow on the railing in the picture below is almost all gone.
Now we have to take stock of our neglected house and bring it into the modern world. It's easy to get used to a water stain on the ceiling, old worn rugs, cracks in the cement, and other minor problems. I've even tried to line up people to help out, but I was either uncomfortable with them, or they didn't have time. But after our experience with fixing up my mom's house in LA, it's time to get serious and get our house to the point that visitors don't wonder how we live in such a well worn space.
But first I have to shovel snow a bit.
Walked up to the train station to catch the light rail to the airport. Flew out of Seattle and over clouds most of the way home and arrived about 8 pm over mostly cloudy Cook Inlet, but with hints of the sun.
There was sun yesterday morning, but wet snow by evening. It is breakup - lots of puddles in the streets. Temperature now is 35˚F on our backyard thermometer and the snow on the railing in the picture below is almost all gone.
Now we have to take stock of our neglected house and bring it into the modern world. It's easy to get used to a water stain on the ceiling, old worn rugs, cracks in the cement, and other minor problems. I've even tried to line up people to help out, but I was either uncomfortable with them, or they didn't have time. But after our experience with fixing up my mom's house in LA, it's time to get serious and get our house to the point that visitors don't wonder how we live in such a well worn space.
But first I have to shovel snow a bit.
Wednesday, March 14, 2018
Bainbridge High School Student Walkout
I went over to Bainbridge High School to see the student walkout this morning. Here are some pictures and a bit of video.
Students lying down in protest for 17 minutes of silence.
Community members were gathered across the street from the school.
And I got the video at the end as the students were getting up and going back to class.
Students lying down in protest for 17 minutes of silence.
Community members were gathered across the street from the school.
Subscribe to:
Posts (Atom)