How Not To Do An Online Survey

I had to call a company yesterday, and during the call I spoke to two different representatives.  The first one was very good, listened, understood my issue and said she would fix it and then check to be sure it was really fixed.  She could not, however, do anything about my other issue.  That had to be handled by a different department.

That person said he could change the setting on the computer.  I'd had the same problem last year with the company.  He said I should fill out an online form. I looked at it.  It wasn't short and required that I look a number of things up.   (I had earlier, before this call,  filled out paper work online for the rental car accident payment. Our rental car in October had been hit in a parking lot while we were eating lunch.  I was fed up with all the paper work and not receptive to doing more work like that just then.)  I asked him to do what the other rep had done - fix it and then check in a few days.  His response was, I told you a way to fix it, but you don't want to take it. (He was n't wrong, but his tone of voice was.)  I said, look, if your company can't fix this over the phone without me filling out the whole form over again, then maybe I'll just have to take my account elsewhere.  "But I gave you a way to do it ('you dolt' wasn't said, but was in his tone.)

So today I got an email asking me to take a survey about my phone call with the company.  It wanted me to rate 'the representative' I'd talked to.  I couldn't find a way to ask, "Which one?"  I couldn't find a way to leave a comment.  I couldn't go on (to see if there was a place for a comment somewhere else, without rating the representative.  I didn't want to rate the good rep poorly nor did I want to rate the poor rep positively.

So, they didn't get my feedback.  I just closed that window and went on to other things.

Of course, I don't know if they really wanted to know, or whether this was a gimmick to make me think they cared how I felt.  If it was a gimmick, it didn't work.  If they really wanted good feedback, it didn't work.

Surveys should always have comment options because:

• People may not understand the question, and comments let the surveyors know that.
• The options offered may not include what the responder actually experienced.
• The question may not be appropriate.   (my case here - which rep did they mean?)

So, if any of these things happens, and there is no comment option and no 'skip the question' option, then the respondent has two choices:

• Answer the best she can, knowing the response isn't really right
• Quit the survey

I understand the companies that do surveys for other companies want to do things as quickly and easily as possible.  Just run the survey responses through the computer and provide the client with whatever statistical analysis of the numbers they need.

Comments, for that sort of surveyor, just get in the way.
They can't be so easily quantified.  And if they actually point out flaws in the survey instrument, then everything would have to be thrown out and started over again.  Data you've already collected are now suspect (how many other respondents had the same problems, but just answered the question anyway?), so you have to ask those people again (not what anyone wants to do) or keep the bad data or start over.  And it would be hard to do that without telling the client that you - however you sugar coat it - screwed up.

So who is this post for?  I guess

• First it's for me, to just get this off my chest.
• Second, for others facing frustratingly bad survey questions - just quit if they don't  give you a way to point out why the question doesn't work for you.
• Third, for companies that hire people to do surveys for them.  Don't trust the survey consultants..  Take the survey yourself.  Have a number of other people test out the survey - perhaps even actual customers.  Require the survey company  to allow people to comment throughout the survey.
• Fourth, for ethical surveyors, who perhaps didn't learn all they needed to learn, but are willing to listen because they want to do it right.