On Monday, March 18, 2013, OLC attended Agile Experience Design's User Research for Agile Teams meetup featuring Andres Glusman and Brenna Lynch of Meetup, Anna Cory-Watson and Cameron Burley of AppNexus and Jodi Leo of CapTap.
Jodi Leo demoed first, introducing CapTap as "building a flexible lending platform for small businesses." She outlined her presentation as first logistics, tools and time. "User researched used to be hours of time-consuming logistics," Leo said. "It was the same thing in 1963 and even in 2003. Since 2007, there has been a ton of tools to figure out how people are using your product," she said.
Anna Cory-Watson and Cameron Burley of AppNexus presented their user research in Scrum. "Scrum is all about doing, but research is all about thinking about doing," Cory-Watson said. "Scrum is a form of agile development," she added. "There really isn't a way that research is incorporated in Scrum. It's the responsibility of a UX designer that puts research in. When AppNexus started, UX wasn't at the top of the list, but we've been working hard to change that. UX is research. One of the ways we've incorporated research is through discovery—interviews with end users and internal stakeholders with use cases and edge cases, workflows, workarounds and other tools. Next is ideation, where we collaborate with users and internal stakeholders for A/B testing and product design workshops. Validation follows that, where we do early testing with end users and internal stakeholders for rapid research with lightweight prototypes and paper prototypes. And we have finalization, which is tested on end users with Alpha/Beta testing and usability testing with a live site," Cory-Watson said. "We used to have a 'User Wednesday Program,' where we had weekly cadence, test designs in production, questions and dissemination of results, but now it's been rebranded as the AppNexus Allies Research Project, to make it more product-friendly, monthly release notes and incorporated new tools like rapid research programs. We prototype with Verify and Solidify, which is great for internal testing. Rapid research programs allows for prototyping at early design stages and can be appropriate for end users," she said. From here, Cameron Burley took over and he discussed getting started with Sprint 0. "You have to have a checklist when you start to do research," Burley said. "Think about the impact on stakeholders, the impact on business procedures, dependencies and assumptions, constraints and epics. Without it, your client will not have a design that is functional," he said. Burley talked about UX Staggered and he listed it as being one to two sprints ahead of the team and that the team can hit the ground running. "It also gives ample time to explore backlog," he said. Burley talked about UX Spike, which he described as, "An experimental test, exploratory, a chance or opportunity to think about how to solve the problem and what else is needed. Think about research alternatives, socialize alternatives and mock or ideate alternatives," he said.
Andres Glusman and Brenna Lynch took the stage to present design and UX testing. "We all experience the Malkovich Bias," Glusman said. "That's the tendency to believe that everyone uses technology the way you do. The cure is to watch people use the stuff you build. It shifts the conversation from, 'How I would use the website,' to 'How this person uses the website,'" he said. The core reason why Meetup focuses on user usability is to shorten the amount of time people at Meetup wonder how someone would react to something and actually watch someone react to something. Brenna Lynch briefly talked about testing at Meetup. "We test with a lot of people. We have up to 400 sessions a year," she said. "Everything user-facing at Meetup is tested and we test four times a week. Tests are set up prior to us knowing what we're going even test on and we recruit participants from the community, but from Craigslist as well. Anyone from any team can request to have something tested," Lynch said. "When we do test, we use GoToMeeting to broadcast the sessions to designers, developers and product managers. We follow each session with discussions and videos and notes," she said. Glusman stepped in and added, "IM conversations are vital to testing. It throws away moderator questions and you learn great things too." Glusman switched gears and talked about design, stating that design is very important to the makeup of Meetup.com. "Our sign-up process used to be five steps," he said. "It used to take five steps to create a Meetup group. One of our designers came up with a prototype that was along the lines of, 'What if it wasn't constrained by anything?' When we tested it on users, they were generally confused, but they were really engaged with the interface. Our developers iterated until they felt it was clear enough for end users. We carved it into a series of experiments and when we were confident enough, we ripped out the old UI and plugged in what we tested. We learned what was good and bad through this—it resulted in 50 percent lift in groups. Usability testing liberates designers," Glusman said. "It's all about problem solving."