UX Research and Accessibly
Introduction
In a world where technology infiltrates almost every aspect of life, how does one make sure the products are usable and accessible? The answer is through the practice of user experience (UX). User experience is a person or user’s overall experience when using an online product e.g. websites or online applications. Accessibility is a term used when describing the usability of an online product and according to Adobe it has two components: “…how users with disabilities access electronic information and how web content designers and developers enable web pages to function with assistive devices used by individuals with disabilities” (https://www.adobe.com/accessibility/gettingstarted.html).
I conducted an internship at Knowbility, INC in the spring of 2019. Here, I was able to learn the back-end of user experience research in the context of technology and accessibility, or the lack thereof, such as recruiting for studies, scheduling and sitting in on tech checks, watching live or recorded moderated remote usability studies, and timing and counting steps from recordings of usability-lab studies. My intentions and goals of working with Knowbility stemmed from the desire to bring awareness to the lack of accessibility in the world of technology and to get a foot in the UX door. I was fortunate enough to work with passionate people such as Sharron Rush, Knowbility’s Executive Director and Founding Member, and Dana Narveson, our Community Programs Director. This report will describe the organization itself, my role within it, projects that I worked on, and the broader anthropological applications to UX research.
According to the Interaction Design Foundation, UX research is the “…systematic investigation of users and their requirements, in order to add context and insight into the process of designing the user experience” (https://www.interaction-design.org/literature/topics/ux-research). UX research utilizes various qualitative and quantitative research methods including interviews, creating personas (creating a fake person based on a desired demographic that might use a website and including personal biases and preferences that they might have), and ethnographic research methods.
Usability studies and UX research are the keys to making sure that a web product is usable and has an intuitive flow that a user can follow. Optimizely describes usability testing as a “…method evaluating a website or app’s readiness for release by testing it with real users who are part of the target audience. Usability tests measure the relative ease with which users can accomplish a set of tasks that a typical user of the app or website would need to accomplish” (https://www.optimizely.com/optimization-glossary/usability-testing/). Testing a website’s usability is important to understand what needs to be changed and what works best for the user. If a website or web product is does not have a high functionality or easy user flow then users will choose to use another website. As time goes on, designs change and new technologies are created making usability tests a key component in staying relevant, usable, and accessible.
Here are four common forms of usability testing:
1. Card Sorting: using actual paper cards or online software tools to allow participants in a study to organize topics into categories that seem logical to them.
2. In-person testing: can take the form of individual interviews, focus groups, or moderated usability tests. This is where a moderator (usually the UX researcher) sits with the participant and gives them a task to complete without guiding them through the task.
3. Remote testing: this uses online screen sharing technologies that allow the user to complete studies in their home or natural environment.
4. A/B Testing: this is where two versions of something are shown to half of the same study group to determine which preforms better.
While working for Knowbility, I was able to participate in both in-person and remote usability testing.
Knowbility, INC.
Knowbility is a nonprofit organization based in Austin, Texas. Founded in March of 1999, Knowbility has a “…mission of improving technology access for millions of youth and adults with disabilities all over the world” (https://knowbility.org/about/). In 1999, Knowbility created its first community-building event, the Accessibility Internet Rally or AIR Austin, now known as OpenAIR. This event became the model for which Knowbility could engage “…the information technology and social services community in disability issues” (https://knowbility.org/about/). Knowbility works with different schools, businesses, and other government and nonprofit organizations in order to train staff on different accessibility strategies.
One of the many ways that Knowbility is able to help close the accessibility gap within the world of technology is by creating their own database. This database, known as AccessWorks, allows people with disabilities to register so that they may be notified of studies in which they may qualify. This database is a way to connect real people to UX researchers to gain real feedback and data on the accessibility of their products. When people register with AccessWorks, they need to register with a promo code provided to them and they need to have a PayPal account in order for Knowbility to pay them for their participation in each study. Testers also enter in their basic information such as name, phone number, email, age, location, as well as their disabilities and the assistive technologies that they use. In addition to testers and participants being able to register with AccessWorks, businesses can also sign up to have their websites tested! When looking for participants within the database, a Boolean search can be done to help find specific participants based on their disability and assistive technology in a chosen location(s).
Aside from AccessWorks and Knowbility’s other programs, Knowbility participates in Austin’s annual South By Southwest (SXSW) tradeshow. A really exciting part of my internship was getting the opportunity to volunteer at SXSW for Knowbility and work their information booth (Fig. 2)! I was deemed a coordinator and my role was to sit with some of Knowbility’s volunteers who were to demo their assistive technologies to promote awareness within the tech community and other individuals passing by our booth. I escorted both of my demoers to and from their rideshares into the tradeshow and to our booth. Unfortunately, our location within the tradeshow was tucked away in a back corner with other nonprofit organizations and we did not receive much foot traffic on the day that I was scheduled to work. While working SXSW was exciting and informational, majority of my time at Knowbility was spent working on projects with Google.
Projects and Stages of Accessibility Studies
During my time at Knowbility as an intern, I worked every Tuesday and Thursday for 8 hours a week remotely from home or a back office provided by my other on campus job (Fig. 1). I found that working remotely was the most efficient use of time for both Knowbility and myself because much of my work consisted of recruitment, calling people on the phone, sitting in on remote tech checks and simply just computer/desk work. Every Tuesday and Thursday shift began with sending Dana a message through Slack and either telling her what I would be doing that day or asking her what she needed me to do. This often included sending emails, recruitment, and working on previously recorded studies by either timing how long it took each participant to complete each task, how many steps they took and typing each step out individually into a spreadsheet.
Aside from Knowbility’s programs and conferences, it mainly works on client-sponsored research with large tech companies such as Google. I began working a week before the first round of Google studies had begun and was able to jump right in by creating a screen test, or screener, with Sharron to send out to participants. Due to confidentiality purposes, I am not permitted to discuss the products that were being tested in the Google studies, however, I can discuss what I learned.
When receiving a new project, there were certain steps that followed:
- 1. Begin recruitment from within the AccessWorks database for testers that fit the study requirement.
- If there are not enough qualified participants within the AccessWorks database, begin reaching out to outside organizations, schools and universities, Facebook groups and pages, MeetUp groups, and even Craigslist.
- 2. Create the screen test via Survey Monkey or Google Forms and add logic to automatically disqualify people if they do not fit the study parameters.
- A screen test, or screener, is “…the document or script used to qualify participants for focus group sessions or in-depth interviews” (Kuhn 2016).
- 3. Go through screener responses, and manually disqualify participants based on responses if they were not caught in the screen test.
- Clarify responses with the participant if they seem like they would qualify based on their responses.
A tech check is the last step before a tester can tune into a remote usability test. It consists of the tester successfully joining a Zoom meeting with Jayne based on the invitation they are sent via email from us and then performing certain tasks. These tasks include turning on or turning off their computer’s microphone, camera, and sharing their computer screen as well as other tasks that vary depending on the study parameters. When not recruiting for studies or creating screen tests, I was able to sit in and watch Jayne Shurick, Knowbility’s volunteer UX researcher who also works remotely, conduct tech checks with qualified participants prior to their specific upcoming remote usability studies. I was able to sit in on so many of these tech checks that Sharron asked me to write up a set of instructions for those that wanted to recruit and conduct the tech checks themselves instead of Knowbility doing both of these tasks for them.
For the first round of Google studies I was not a part of the recruitment, but I was in charge of qualifying people via their responses from the screener that I created, and Sharron sent out. I was placed as the point of contact in her email and as a result was responding to people interested in the study. Over 200 people were sent the screen test and as the responses were rolling in, only 43 were collected before I closed the Google Form screen test. Of the 43 people who responded, only 21 qualified but for a limited 12 spots. I had to send emails out to each person that completed the screen test that did not qualify, were not chosen, or responded to the email Sharron sent out too late and the screen test had closed. This was an issue that I did not come across again moving forward when recruiting for Google’s studies. For the actual usability-lab test, I arrived at a market research company’s facility, ThinkGroup, in Austin about 5 minutes away from Knowbility’s office. I met Sharron in the lab, which consisted of a study room where the testers and test facilitator sit in with a two-way mirror on a wall connected to the room that I was sitting in (Fig. 3). Inside there were five chairs on a tiered floor plan with snacks and drinks lining the wall and dimmable lights. There was also a tv in the corner with a live feed recording the test room that we could see into. I kept notes on the background information of the participant, the start and end times of each task they completed, and in addition to their ratings of satisfaction with the product and final thoughts. One thing that I found interesting was that the more icons that are used on a web page, the more difficult it is for some low vision and blind users to navigate a web page.
Moving forward in the next round of the Google study, recruitment proved to be increasingly difficult. They were looking for blind and low vision users that used their Google products on a regular basis who also knew advanced tools within the products themselves. This particular study was in a different state and we had only 4 testers signed up in that area in the AccessWorks database. Having so few users in the database meant we needed to go out and recruit people from different organizations, schools, etc. When all was said and done the study lacked about 4 participants due to lack of qualified testers. I contacted over 148 people, organizations, schools, Facebook groups, MeetUp groups, and a personal Facebook post asking my friends and family in New York to spread the word; only 6 had qualified for the study. Jayne had informed Google that the people they were looking for simply did not exist. The parameters and qualifications required for participating in this study were too rigid. I believe that based on our recruiting experiences, this recruitment should have highlighted that there are real barriers that are keeping blind and low vision users from using Google products and based on these findings, redesign their study. Even if they believed that these people existed, I still don’t think they should be recruiting them to test the accessibility of their product. Ultimately, the study proceeded without the desired number of blind and low vision users.
In order to make a product accessible, it should cater to those who know absolutely nothing about the product in order to truly test how usable ad accessible the product naturally is. If the product does not have a natural flow, then clearly there is something wrong. If highly proficient users are desired for a study then there should also be an additional group of users not proficient in the product being tested. This would benefit the company in search of the data by comparing the two results to each other to see what really is intuitive and what is not easily accessible.
Accessibility should be the ultimate goal of every usability study especially because people with disabilities are a large portion of the population and are a huge demographic that the technology community is leaving out.
“More than 56 million people in the United States are living with a disability, according to the U.S. Census Bureau” (Anderson and Perrin 2017).
According to Essential Accessibility, here is the web accessibility checklist:
1. “Alternative text. Written descriptions (“alternative text”) should be available for images, graphics and logos. That way, people who use a screen reader will know what’s on your website.
2. Online forms.. Ensure these are accessible with features such as labels for any drop-down lists or check boxes with text, and easy navigation with a keyboard.
3. Organized pages. Web pages should include headings and lists, for instance, to make it easier for people to make their way around the site, especially if they’re using assistive technology.
4. Captioned media. Videos, audio files and other sound files should be captioned or transcribed for people who are not able to hear them.
5. Hyperlinks. When hypertext links are included, the text should be self-explanatory, even if it’s read out of context (someone might, for instance, skip from link to link without reading anything else on the page). The links should also be clickable using the keyboard, not just a mouse.
6. Special content. When PDF or Word documents, Adobe Flash animations or PowerPoint presentations are included on your website, you must ensure they have been converted into accessible formats.
7. Color. Don’t use color to communicate information. (You shouldn’t, for example, use a red font to show which of your products are the hottest sellers.) You should also adhere to the accepted standard for color contrast so people with vision disabilities don’t have trouble distinguishing the text on your site.
8. Effective writing. The writing on your website should be clear, and your message should be easy to understand.
9. Assistive technology made available. To give a customer with a disability an inclusive digital experience, organizations should consider offering free assistive technology on their websites. With the technology, customers who have physical disabilities, like cerebral palsy, multiple sclerosis, paralysis, stroke, arthritis and others, can navigate the web or use their mobile device hands free” (https://www.essentialaccessibility.com/blog/web-accessibility/).
Conclusion
UX research is intrinsically anthropological simply based on the fact that UX research examines how people interact, feel, and use different technologies. Remote usability studies is the most ethnographic in nature since the study is preformed within the participant’s home or natural habitat. The UX researcher is able to see how the participants uses and interacts with the product in real time and can hear the questions and thought process of the user. Regardless of which usability method is being used, each method requires some aspect of ethnographic research. The UX researcher is paying attention to movements and mannerisms, asking questions, and observing the participant and their body language.
My experience interning at Knowbility was beneficial to gaining experience in participating in the busy work of UX research. I am excited to grow within the user experience field and I look forward to aiding in awareness of accessibility in technology. The point of technology is to streamline life and make it easier, how can this be true if there are whole demographics being left out of the design process? I hope to see great strides being made in the field of accessibility in the future.
References
- About Us
Knowbility. https://knowbility.org/about/, accessed April 21, 2019. - Adobe
https://www.adobe.com/accessibility/gettingstarted.html - Anderson, Monica, and Andrew Perrin
2017
Disabled Americans Less Likely to Use Technology. Pew Research Center. Pew Research Center. https://www.pewresearch.org/fact-tank/2017/04/07/disabled-americans-are-less-likely-to-use-technology/, accessed May 10, 2019 - Essential Accessibility
https://www.essentialaccessibility.com/blog/web-accessibility/ - Kuhn, George
2016 What Is a Recruitment Screener in Market Research? Market Research Company New York | Drive Research. Market Research Company New York | Drive Research. https://www.driveresearch.com/single-post/2016/09/23/What-is-a-Recruitment-Screener-in-Market-Research, accessed April 21, 2019 - Interaction Design Foundation
https://www.interaction-design.org/literature/topics/ux-research - Optimizely
https://www.optimizely.com/optimization-glossary/usability-testing/ - Rohrer, Christian
2014 When to use which User-Experience Research Methods. Nielsen Norman Group.