usability testing

Recently, I conducted a number of in-depth interviews with members of our company to get a fix on the current status of our project, and to gather any research had been done in our market so that I could share it with other team members. The process was really interesting, and something I'd advocate for anyone who feels like their team has maybe become a little disconnected from one another. Here's a quick list of the technology I used for folks who might also be conducting remote in-depth interviews they will need to review later.

• The interviews were conducted over Skype, partly because I use Skype as my business line, but also so I could run the sound streams through recording software directly.
• I used WireTap Studio (recently purchased as part of the MacHeist bundle) to record two streams during the inverview, one from my USB headset mic and other from Skype itself
• After the interviews, I listened to each one and transcribed pertinent parts to text. I tested several options for variable rate playback, so I could listen to the interviews in double-time (there was a lot of material to review), and ended up using Quicktime Player. After you open an audio file, you can select Window > Show A/V controls, and use the Playback Speed option to chose a rate. For the more pronounced speakers, I could use a 2.5x speed playback. For faster talkers, I had to take it down to 2x. The Jog Shuttle also helped a lot to rewind a sentence or two back.
• I posted the resulting document in Google Pages, which we use for our in-house wiki, and which will give everyone an option to review and add comments.

The interviews went pretty smoothly, technically speaking. It helped to conduct an interview with a fellow coworker initially, before interviewing the major stakeholders, so I could get any technical difficulties out of the way, and basically do a trial run.

The first day of usability testing for Yoursphere went really well. I interviewed two kids, one 10 and the other 15. The difference between the two interviews was pretty staggering. The most interesting thing that came out of the process for me personally was a feeling of empathy for our users. As a result of programming and designing in a virtual closet for several months, I've started to make some strange assumptions about our users:

1. They must get really frustrated, because the project isn't all it could be
2. However, certain parts of the site are so snazzy that they make up for it

After spending a couple hours talking to our users, I both identified these assumptions and was able to let them go. The kids on our site might actually be pretty patient, and have found a way to interface with the site in a way that makes sense to them. Also, the parts *I* really like about the site don't particularly stand out to the kids.

Specifically, I was testing a new implementation of Organic Groups / Panels which allows the members to have some more controlled control over their groups. Overall, the feedback was good, and some hurdles were identified and can be easily corrected. In essence, the experience was really quite cool.

A few tips for myself for future remote tests like this:

• Give regular encouragement, i.e. "You're doing great, this is exactly what we're looking for" as the subject bashes their head against your product.
• Practice polite ways to turn questions around. Q: "So, do I click here" A: "What would you do if I wasn't here?". Silence seems weird, and so does "I don't know, what do you think?". There's some finesse that needs to be exercised here.
• Make sure that the remote computer is plugged in and has a mouse (otherwise the subject will run out of batteries and have to use a trackpad. Ick!)
• Print a copy of the script for each test subject so you can make notes on them
• Give at least a 1/2 hour after a test to jot down thoughts and ideas
• Start recording 5-10 minutes before the interview, so you don't forget among the initial hubbub
• Test to make sure you're computer's recording audio before recording an entire interview without it.
• If you're working with kids, be prepared to improvise a bit. You may want to test something they really aren't ready to use or understand, or you may use terminology that's beyond them.
• Try not to say 'um' more than the kids you're interviewing. Bad form, Chris, bad form!

Today I'm reviewing several screen-sharing apps to conduct some remote usability testing for Yoursphere. These particular tests have some interesting constraints:

• My subjects are parent-volunteered kids, 8-16 years old and all in the same family
• I'm on a mac, they're on PCs
• Tests should be about 45 minutes long
• If possible, I should be able to take over the screen to type in URLs

I tested on the following hardware

• Presenter: Windows XP with 1920 x 1200 resolution
• Client: Macbook pro running Leopard

Yuuguu

Pretty good, but not quite good enough for usability testing.

• Free
• Marginal refresh rate (I foresee missing some important clicks and mouse wandering)
• Requires registration of host and client for optimal use
• Very easy transfer of control to attendee
• Nicely integrated chat notifications (for passing URLs, for example)

TeamViewer

Pretty much pure awesomeness after looking at all the other apps.

• Not so free (free for evaluation, $39/mo or around $650 for full license)
• Awesomely spectacular refresh rate
• Transfer of screen control is more difficult than with Yuuguu, but not bad
• Presenter can run a small executable without having to install anything
• No account is needed on either side, however the presenter has to type in a 9-digit code plus 4-digit password to connect.

Mikogo

Very simple interface, but didn't quite cut the mustard. The Mac version has a permanently disabled 'preferences' menu item, indicating it's probably in beta whether they say so or not.

• Free
• Uber-simple interface
• Extremely crappy refreshing, probably totally unusable for a usability test
• Registration needed by host, but not client

ShowMyPC

Not too bad, except for that refresh rate. Darn you, TeamViewer, you've spoiled me!

• Free
• Surprisingly works for a mac
• No installation required
• No accounts required, just pass a (long) code (and don't forget to type the dash)
• Fairly dismal refresh rate
• Rather obscure interface (screen sharing button is a black box in a gray box in a another gray box)

Bosco's Screen Share

I *maybe* should have been wary of the playful doggy-centric identity, but I gave it a go anyway. Apparently you have to configure your router and firewall to get it to work. I don't think I'm going to start a usability test with a router configuration. No sir.

Persony

Pretty cool stuff, especially the 3D conference room motif </toungeincheek>.

• Free for 1-to-1 meetings
• Web-based, you just need to pass a code. You might need to install some Java though, which could really hang things up in a usability test.
• Absolutely the worst refresh rate of all the apps
• Setting permissions for different attendees is pretty nifty
• Lots of features, like voice and webcam sharing
• Did I mention the horrible refresh rate?

Conclusions

I guess we'll be using TeamAssist tomorrow on an evaluation basis. The other apps just don't compare in terms of refresh rate.

Sources

http://webtoolsandtips.com/freeware/screen-sharing-remote-access-web-tools/ - Good leads on several apps
http://www.masternewmedia.org/screen-share-top-25-best-screen-sharing-tools/ - Good list, not entirely accurate