First day of Yoursphere remote usability testing
The first day of usability testing for Yoursphere went really well. I interviewed two kids, one 10 and the other 15. The difference between the two interviews was pretty staggering. The most interesting thing that came out of the process for me personally was a feeling of empathy for our users. As a result of programming and designing in a virtual closet for several months, I've started to make some strange assumptions about our users:
- They must get really frustrated, because the project isn't all it could be
- However, certain parts of the site are so snazzy that they make up for it
After spending a couple hours talking to our users, I both identified these assumptions and was able to let them go. The kids on our site might actually be pretty patient, and have found a way to interface with the site in a way that makes sense to them. Also, the parts *I* really like about the site don't particularly stand out to the kids.
Specifically, I was testing a new implementation of Organic Groups / Panels which allows the members to have some more controlled control over their groups. Overall, the feedback was good, and some hurdles were identified and can be easily corrected. In essence, the experience was really quite cool.
A few tips for myself for future remote tests like this:
- Give regular encouragement, i.e. "You're doing great, this is exactly what we're looking for" as the subject bashes their head against your product.
- Practice polite ways to turn questions around. Q: "So, do I click here" A: "What would you do if I wasn't here?". Silence seems weird, and so does "I don't know, what do you think?". There's some finesse that needs to be exercised here.
- Make sure that the remote computer is plugged in and has a mouse (otherwise the subject will run out of batteries and have to use a trackpad. Ick!)
- Print a copy of the script for each test subject so you can make notes on them
- Give at least a 1/2 hour after a test to jot down thoughts and ideas
- Start recording 5-10 minutes before the interview, so you don't forget among the initial hubbub
- Test to make sure you're computer's recording audio before recording an entire interview without it.
- If you're working with kids, be prepared to improvise a bit. You may want to test something they really aren't ready to use or understand, or you may use terminology that's beyond them.
- Try not to say 'um' more than the kids you're interviewing. Bad form, Chris, bad form!
Recent blog entries
Twitter feed:
- Any tips on where to get the word on on a press release like this? http://is.gd/dR9hC #fb3 hours ago
- Press release for the "Save Joseph" campaign http://is.gd/dR9hC #fb3 hours ago
- Added bounty for the person with the lead that lands Joseph a job. Only 1 week left in the Save Joseph campaign! http://is.gd/dKybt #fb3 hours ago
- @aplusk A friend suggested you might be able to help me. Any tips on getting the word out about this: http://savejoseph.org? Thanks!15 hours ago
- @rvtctweat The scrolling module was Views Infinite Scroll http://drupal.org/project/views_infinite_scroll16 hours ago
- 8 more days to help me find a job for an amazing artist before CERTAIN DEATH! Please send ideas! http://savejoseph.org #fb1 day ago
- How to use images for menu items in Drupal with a simple preprocessing function http://is.gd/dOXc71 day ago
- The campaign to find a kick-ass artist a job is on Twitter. Follow if you believe in the cause! http://is.gd/dMwHq #fb2 days ago
- @pcambra Thanks, I'm glad you're liking them! Cheers!2 days ago
- How to create an image based dynamic countdown timer with Imagecache http://is.gd/dKHnR2 days ago
Topics
- activism
- apple software
- book reviews
- design
- Drupal
- drupal administration
- drupal community
- drupal events
- drupal modules
- drupal resources
- freaky stuff
- geeky entertainment
- iphone ipod touch
- jquery
- learning
- life hacks
- looping
- maps
- minimalism
- music
- philosophology
- presentations
- productivity
- rambling
- restraint
- software
- theming
- treadmill desk
- usability
- usability testing
- video demonstrations
- web hosting
- original songs
- This American Life
- evernote
Blog archive
- March 2009 (2)
- April 2009 (15)
- May 2009 (10)
- June 2009 (3)
- July 2009 (3)
- August 2009 (6)
- September 2009 (2)
- October 2009 (4)
- November 2009 (3)
- December 2009 (1)
- January 2010 (1)
- March 2010 (1)
- June 2010 (1)
- July 2010 (43)










Comments
Chris,
Having worked as a SW test engineer for the part 15 years had given me a view on users from the usability perspective which is something that we as engineers don't think about a whole lot. We just think about the greatest features that the final product will have and on how our end product will solve all the problems of the world; including global warming and world hunger and make us rich in the process.
One thing that I have seen work and that makes a lot of engineers and designers humble themselves is to have usability sessions on a usability lab observation booth recording the users' actions while the engineers and designers are observing. Engineers and designers tend to make a lot of assumptions and most of them come crashing down while observing users "use" their creations.
I have participated on these sessions watching users from the moment they open the box, attempt to install the product (HW first? or SW first?) to the moment they start the use the product. A lot is learned from these sessions.
I think interviews are a good course of action, however during interviews users might tend to answer your questions they way they think you want the questions to be answered. During recorded and observed usability sessions users are careful because they know they being watched, however old habits are hard to give up and users will still give a lot of information they don't know are giving you and that you would not otherwise get during interviews.
The target audience for your product is an interesting one and I think one that is more forgiving about usability flaws, but at the same time will impose and teach you new usability models by finding workarounds to those issues, that is what and when you need to observe and take notes.
I have a teen daughter that the first thing she does after coming home from school is to log into Facebook, I have observed her and it is amazing how she can wade through Facebook and adapt to the constant changes that Facebook is constantly making to the UI, she never complains, she just adapts and find workarounds for herself. One of the things I have learned while observing her use Facebook is that she does not really care how it looks, she just wants to get in touch with her friends, get updated on their happenings (post/view pictures) and move on.
Some great thoughts, thank you.
You're right, the desire to resolve global warming (or other similarly lofty goals) seems to overshadow practical thinking when I really get down to the process of building something. After having experienced some usability testing of my own work, I'm really humbled and see that it's a little beyond me to accurately predict what users are going to want. The trick, perhaps, is reigning yourself in before spending too much time and effort on hunches, before gathering some genuine data.
Your observations about your daughter are pretty much spot on with what I observed. The younger subjects in particular were really oblivious of the interface, and were very focused on particular goals. I didn't hear a lot of criticism, even when one subject ran into a bug that prevented them from moving forward. He just patiently waited a couple of minutes and then asked if he could refresh the page.
Since being introduced to the word "satisficing" in Don't Make Me Think (Krugg), it's become part of my internal language (a blend of "satisfy" and "suffice"). We really do expect to stumble through things, and as soon as we find a way, we're on to the next problem. When I look at my own patterns, I see the same behavior I observed in our test subjects. I'm just using unfamiliar tools to accomplish something I want, and I'm used to failing for a while before I get it.
I envy your experiences with usability testing. I'd definitely like to get a little more experience in the field to gather some more ideas and see how the professionals get it done.
Makes me wonder how the site is doing now? Do you still work with this group?
Post new comment