First day of Yoursphere remote usability testing

The first day of usability testing for Yoursphere went really well. I interviewed two kids, one 10 and the other 15. The difference between the two interviews was pretty staggering. The most interesting thing that came out of the process for me personally was a feeling of empathy for our users. As a result of programming and designing in a virtual closet for several months, I've started to make some strange assumptions about our users:

  1. They must get really frustrated, because the project isn't all it could be
  2. However, certain parts of the site are so snazzy that they make up for it

After spending a couple hours talking to our users, I both identified these assumptions and was able to let them go. The kids on our site might actually be pretty patient, and have found a way to interface with the site in a way that makes sense to them. Also, the parts *I* really like about the site don't particularly stand out to the kids.

Specifically, I was testing a new implementation of Organic Groups / Panels which allows the members to have some more controlled control over their groups. Overall, the feedback was good, and some hurdles were identified and can be easily corrected. In essence, the experience was really quite cool.

A few tips for myself for future remote tests like this:

  • Give regular encouragement, i.e. "You're doing great, this is exactly what we're looking for" as the subject bashes their head against your product.
  • Practice polite ways to turn questions around. Q: "So, do I click here" A: "What would you do if I wasn't here?". Silence seems weird, and so does "I don't know, what do you think?". There's some finesse that needs to be exercised here.
  • Make sure that the remote computer is plugged in and has a mouse (otherwise the subject will run out of batteries and have to use a trackpad. Ick!)
  • Print a copy of the script for each test subject so you can make notes on them
  • Give at least a 1/2 hour after a test to jot down thoughts and ideas
  • Start recording 5-10 minutes before the interview, so you don't forget among the initial hubbub
  • Test to make sure you're computer's recording audio before recording an entire interview without it.
  • If you're working with kids, be prepared to improvise a bit. You may want to test something they really aren't ready to use or understand, or you may use terminology that's beyond them.
  • Try not to say 'um' more than the kids you're interviewing. Bad form, Chris, bad form!

Comments

Chris,

Having worked as a SW test engineer for the part 15 years had given me a view on users from the usability perspective which is something that we as engineers don't think about a whole lot. We just think about the greatest features that the final product will have and on how our end product will solve all the problems of the world; including global warming and world hunger and make us rich in the process.
One thing that I have seen work and that makes a lot of engineers and designers humble themselves is to have usability sessions on a usability lab observation booth recording the users' actions while the engineers and designers are observing. Engineers and designers tend to make a lot of assumptions and most of them come crashing down while observing users "use" their creations.
I have participated on these sessions watching users from the moment they open the box, attempt to install the product (HW first? or SW first?) to the moment they start the use the product. A lot is learned from these sessions.
I think interviews are a good course of action, however during interviews users might tend to answer your questions they way they think you want the questions to be answered. During recorded and observed usability sessions users are careful because they know they being watched, however old habits are hard to give up and users will still give a lot of information they don't know are giving you and that you would not otherwise get during interviews.
The target audience for your product is an interesting one and I think one that is more forgiving about usability flaws, but at the same time will impose and teach you new usability models by finding workarounds to those issues, that is what and when you need to observe and take notes.
I have a teen daughter that the first thing she does after coming home from school is to log into Facebook, I have observed her and it is amazing how she can wade through Facebook and adapt to the constant changes that Facebook is constantly making to the UI, she never complains, she just adapts and find workarounds for herself. One of the things I have learned while observing her use Facebook is that she does not really care how it looks, she just wants to get in touch with her friends, get updated on their happenings (post/view pictures) and move on.

Some great thoughts, thank you.

You're right, the desire to resolve global warming (or other similarly lofty goals) seems to overshadow practical thinking when I really get down to the process of building something. After having experienced some usability testing of my own work, I'm really humbled and see that it's a little beyond me to accurately predict what users are going to want. The trick, perhaps, is reigning yourself in before spending too much time and effort on hunches, before gathering some genuine data.

Your observations about your daughter are pretty much spot on with what I observed. The younger subjects in particular were really oblivious of the interface, and were very focused on particular goals. I didn't hear a lot of criticism, even when one subject ran into a bug that prevented them from moving forward. He just patiently waited a couple of minutes and then asked if he could refresh the page.

Since being introduced to the word "satisficing" in Don't Make Me Think (Krugg), it's become part of my internal language (a blend of "satisfy" and "suffice"). We really do expect to stumble through things, and as soon as we find a way, we're on to the next problem. When I look at my own patterns, I see the same behavior I observed in our test subjects. I'm just using unfamiliar tools to accomplish something I want, and I'm used to failing for a while before I get it.

I envy your experiences with usability testing. I'd definitely like to get a little more experience in the field to gather some more ideas and see how the professionals get it done.

Makes me wonder how the site is doing now? Do you still work with this group?

Post new comment

The content of this field is kept private and will not be shown publicly.