@erinjo "Mr Watson, come here, I want you."

@find_erik Exactly! We're looking forward to you trying it out - our beta starts very soon.

We're at the #reclaimyourdomain hackathon this weekend. #indieweb in education is an exciting idea! http://reclaimyourdomain.org/

Here's how Known makes it easy to publish content on your own site & share it across all your networks: http://werd.io/2014/my-indieweb-life-how-my-site-gives-me-an-awesome

@misuba For now you'll need to connect your site with http://brid.gy. We're looking at ways to simplify this step.

@misuba Not yet, but we're thinking hard about how we can make the best possible community space for Known users.

David Wiley (@opencontent) wrote a lovely post on Known, the #indieweb, and education technology: http://opencontent.org/blog/archives/3393

Open Source User Research Tools

We’re trying an experiment at Known. We believe that user research is a valuable part of the design and development process, but it is often under-utilized or neglected, especially in open source projects. We also understand that it can be tricky figuring out where to start with user research if it’s not something that you’ve done before.

But we want to make it easier. While Ben and I plan to continue to make user research a key aspect of the work we do, as our community grows, we’d love for others to contribute research as well. We’d also love to help other projects and communities get started with user research.

To that end, we’ve set up a new section in our Github account, and we’re releasing some of the templates that we use for user research. Our goal is to make it easy to get started, even if you’ve never done research with people before.

I presented this at Open Source Bridge in Portland on June 25th. We’d love to grow our collection of resources and continue to provide useful information. Check out the templates we’ve posted on Github. An outline of my talk at Open Source Bridge is below.


DIY User Research for Open Source Projects

Why User Research?

In open source, there are a lot of software projects developed to scratch your own itch. That’s fine if you’re building for yourself or a small group of people like you. But if you want to build software that solves a greater need and is accessible outside a small circle of developers, you need to do user testing.

Even if your product isn’t directly similar to another major product or tool out there, your users are used to the experiences and interactions of companies like Twitter, Facebook, Dropbox, and others. So in a way, you’re competing with the experiences at these companies. Organizations like this have the resources for a lot of user research.

My goal with this presentation is to provide you with tools and information to run your own quick, cheap, and easy user research, even if you have a small team. It can be hard to get started with user research if you don’t know what to put together or where to begin.

We recently put our user research materials on Github for others to modify and use. I’d love feedback on what templates to add and what information people are looking for.

We’re a team of two, and Known began as a side project over a year ago. We became a company about 8 weeks ago.

The thing that made the jump to fulltime with Known possible was our acceptance into Matter. Matter is a startup accelerator in San Francisco with support from the Knight Foundation, KQED, and PRX. They focus on companies that aim to change media for good.

One of the ideas at Matter is that companies should focus on a combination of desirability, feasibility, and viability. Desirability covers things like usability, accessibility, and empathy for users. Feasibility is how able you are to build the product based on technology available and your knowledge. Viability has to do with the sustainability of the product as a business or (in the case of many open source projects) an active community.

I think a lot of open source projects are strong in the technical feasibility area but are necessarily as focused in the areas of viability and desirability. User research helps with desirability.

Getting Started

When should you test?

  • When the thing you’re making goes from “project” to “product”
  • When you want someone else to use the thing you’re making
  • When you’re building based on assumptions
  • When you don’t know the answer to something that you’re building for
  • And if nothing else, test early and test often

What should you test?

  • Start with the questions that you want answered
  • Decide what kind of answers you’re looking for
  • Choose the research methods that will get you closer to the answers you’re looking for

There are lots of types of research out there, but I can’t cover them all. Today I want to talk about three: surveys, interviews, and usability studies.

Let’s start with surveys.

When should you use surveys?

  • When you want to measure changes over time (such as satisfaction or uptake)
  • When you want to quantify characteristics of users
  • When you want to measure attitudes or intents
  • When you want to measure product usage or activity

We’ve put a section on Github for surveys. There’s some information on getting a significant number of responses, deciding what questions to use, and preparing the survey. I’ve also added a few tips.

If you’re just getting started, these are some tools you might consider for the survey: Survey Monkey, Wufoo, Typeform, Google Forms, and LimeSurvey. For quick and cheap surveys, lately I’ve been turning to Survey Monkey and Google Forms as my top options.

We’ve also included some starter questions on Github to customize and use for your own surveys.

If you’re looking for participants to respond to your survey, these are some options to explore:

  • Company email list / customer list
  • Listservs, mailing lists, interest groups lists
  • Craigslist
  • Intercept surveys or pop-ups on your site
  • Social sharing – post on Twitter, Facebook, LinkedIn, etc

With Known, we’ve conducted one survey so far. We got responses from about 50 people who were amateur bloggers and photographers. We wanted to find out more about the tools they used for blogging and for social media.

Moving on to interviews.

When should you use interviews?

  • When you are conducting needs finding
  • When you want to see how someone works in their environment (for in-person interviews)
  • When you want to assess needs, tools, workflows
  • When you’re gathering detailed info on attitudes, desires, experiences

We’ve put a section on Github for interviews. There’s some information on getting participants, as well as a general game plan and timing. If you’re just getting started, these are some tools you might consider for recording interviews: appear.in, Skype, Google Hangouts, join.me, GotoMeeting, Adobe Connect Pro.

We’ve added some templates for getting started with interviews in Github. Right now we have a template for screening participants, an email confirmation template, a sample script for the person conducting the interview, and some sample questions to get started.

If you’re looking for participants to respond to your survey, these are some options to explore:

  • Company email list
  • Craigslist
  • Social sharing – post on Twitter, Facebook, LinkedIn, etc
  • Word of mouth
  • Meetups, clubs, groups

At Known we started doing interviews before we were even an official company. So far, we had a session where we talked to people on our mailing list who were already interested in indie web and the platform. Then we talked to a set of amateur bloggers, writers, and photographers who post their work using blogging platforms. Finally, we’ve been speaking with PR and social media people who focus on creating content for organizations.

Finishing up with usability studies.

When should you use usability studies?

  • When you have a prototype or product
  • When you’re testing user flows
  • When you’re looking for frustrations or problems users might experience
  • When you’re testing navigation in your product or website
  • When you’re testing interactions in your product or website
  • When you want to see how a real person performs tasks

We’ve put a section in Github on usability studies. There’s some information on getting participants, figuring out your game plan, and timing or scheduling for the studies.

If you’re just getting started, these are some tools you might consider for recording usability studies (both remote via screen-share and in-person where you’ll want to record the session).

For a desktop device try: Silverback, Camtasia, Skype, join.me, GotoMeeting, Adobe Connect Pro, or Morae. For a mobile device try: Magitest, UX Recorder, a camera sleigh, or try mirroring your device to a desktop computer that you can record on.

If you’re the person conducting the usability session, you might want some tools for recording and notetaking, such as: a Livescribe pen and journal, Etherpad for online notes, a camera, an audio recorder of some sort, or a notebook and pen.

We’ve added some templates to Github for usability testing. They include a participant confirmation email, a consent form, and a moderator script.

If you’re looking for participants for your usability sessions, these are some options to explore:

  • Company email list
  • Craigslist
  • Social sharing – post on Twitter, Facebook, LinkedIn, etc
  • TaskRabbit
  • Hackathons, Conferences
  • Hostel common rooms
  • Co-working spaces
  • Coffee shops (buy them a drink)
  • Where people gather in public spaces: university quads (students), airports/train stations (bored travelers), farmers’ markets/craft markets/flea markets (shoppers)

We haven’t gotten a chance to do any formal usability sessions at Known yet, although we’re trying to get the platform in the hands of as many people as possible. We’re hoping to test user onboarding and account creation, posting with the different content types our platform has, getting the platform set up on our hosted space or on someone else’s server, the comments and social interactions that our platform has, and our browser plugin bookmarklet.

Analyzing the Data

One of the advantages to doing user research with a small open source team is that you don’t have to write up big reports at the end. Once a session ends, we debrief and quickly pull out trends and notable moments. We also go through the information from all participants and cluster with sticky notes to highlight trends and themes that emerged across the responses. We’re looking for things that we can quickly act on, as well areas that we need to explore further.

Wrap Up

My goal with this presentation is to give you tools for quick, easy, and cheap user testing. I want people to feel empowered to carry out user testing even if they have a small team, like the two of us. The resources on Github are evolving, so please let me know what you’d like to see. We want to continue to add useful information and resources over time.

We open sourced our user research materials! https://github.com/idno/User-Research If you're at #osbridge, @erinjo will discuss: http://opensourcebridge.org/sessions/1331

Here's some links that piqued our interest this week. #knowledgefriday

There were two awesome wins for democratic technology this week:

The House of Representatives voted overwhelmingly to reign in NSA backdooring this week:

The House of Representatives just overwhelmingly voted to rein in the National Security Agency. By a vote of 293 to 123, the House approved a proposal by Reps. James Sensenbrenner (R-WI), Thomas Massie (R-KY), Zoe Lofgren (D-CA), and others that would limit "backdoor searches," a method of spying on Americans despite legal safeguards designed to prevent it.

Meanwhile, the Supreme Court unanimously voted to strike down an abstract software patent:

Essentially, the Court ruled that adding “on a computer” to an abstract idea does not make it patentable. Many thousands of software patents—particularly the vague and overbroad patents so beloved by patent trolls—should be struck down under this standard. Because the opinion leaves many details to be worked out (such as the scope of an “abstract idea”), it might be a few years until we understand its full impact.

Over in Campaign, Russell Davis discussed why content can't truly blossom in walled gardens, and why the indieweb is important:

This little band of web idealists actually have quite a lot in common with some of the world’s largest corporations. They all embarked on the exciting adventure of building their own websites, experimenting with content management systems, blogs and Flickr accounts before being lured to the ready-made audiences offered within the confines of Facebook, LinkedIn etc. [...] And, now, everyone’s not quite so sure; these are precarious places. Do you want Facebook to decide how many people your "free" content reaches? As strange as it sounds, maybe corporations should be investigating the IndieWeb – they might be fellow travellers.

Teens don't necessarily agree. VentureBeat listed the apps they actually use:

Teens. Since the beginning of humanity, they’ve always represented what the future of humanity would look like. Now, thanks to a new survey, we have an idea of what kinds of websites, apps, and online services the future of humanity will enjoy– at least for the next few years of existence. Then, an entirely new crop of startups replaces the upstarts that recently rose to power.

Jake Solomon calls for empathy in civic service design:

This is how we interface with our government: We beg on our knees, we queue all night for shelter, and we get aggressive letters in the mail. Our services disdain those they are envisioned to help. [...] This disdain shows in huge, controversial, life-changing ways. We take 260 days to get disability benefits to our veterans. We spend $3,000 to room a family for a month while the family begs for $900 in cash instead.

And finally, we were enthused by the New York Times, the Washington Post and Mozilla teaming up on a community platform for journalism:

The collaboration among representatives of the three organizations grew out of conversations that began last October at an industry conference, according to Sasha Koren, the Times’s deputy editor of interactive news who is also part of the steering committee. [...] The project, which will have about a dozen members, will be led by Mozilla’s Dan Sinker, who heads the Knight-Mozilla OpenNews initiative, which develops digital-news tools.

Have a wonderful weekend, everyone!