GRIT Toronto Test #1: StreetARToronto — The Map!

Marisa Bernstein

February 14, 2019

GRIT Toronto is a civic usability testing service that connects technology creators — including governments, startups and civic tech groups — with diverse and underrepresented communities to help test their products. Our mission is to make usability testing more accessible, inclusive and effective.

One of the ways we do that is by working in the open. These regular blog posts open up the black box of technology testing, introducing readers to some of our clients as well as key findings from our tests.

Users try out a preview of the StreetARToronto web map at the public launch event in November 2018.
Users try out a preview of the StreetARToronto web map at the public launch event in November 2018.

GRIT Toronto’s first usability test was done in partnership with StreetARToronto to assess the team’s new web map prior to its public beta release. The map features projects that have received support from StreetARToronto’s Partnership Program, and helps residents and visitors discover street art across the city.

| Read more about the development of the StreetARToronto map here.

The focus of the test was to find ways to make the map as simple and easy for users as possible.

We tested the tool with a small group of people, aged from their mid-20s to late 60s, with a diversity of backgrounds and tech abilities. We held four in-person, one-on-one sessions with the testers, with most taking place in coffee shops.

Testers were asked to complete a series of tasks using the web map on both a laptop and smartphone, including using the search filters and finding artwork completed in a specific year and specific ward. They then provided feedback on the process of completing each task, as well as comments on the look and feel of the website.

Results

Through this process, we uncovered a few bugs, and were able to identify potential improvements to the map interface.

Part of the current map interface, showing the updated filters.
Part of the current map interface, showing the updated filters.

Making filters more intuitive

The search filters are meant to make it easier for users to sort through all of the artwork available on the web map. However, the testers were confused about what the filters meant and how to use them. In particular, they were unsure about which buttons to click to apply the filters and update the results, as well as how to return to the original view. This made the testers less confident when trying to navigate the web map.

To address this confusion, we recommended making “select all” the default state and making the process of clearing the filters more obvious. One tester also suggested more direct messaging drawing users in and explaining the map’s purpose and features. An introductory screen with this information now appears when you first visit the map.

Additionally, there was a desire for more filters (e.g., filtering by artist), which StreetARToronto plans to add to the web map as its development continues. We recommended StreetARToronto continue to gather feedback as they expand and refine the filters, in order to learn how people are using them and what additional functions they would like to see.

Map points should be more discoverable and dynamic

Testers noted that the dots showing artwork locations were not visible enough. Our suggestion was to change their colour and/or size, as well as giving the dots a more noticeable change in appearance when selected.

Fixing missing artwork images

Some artwork images were missing or broken during the test. Part of the web map project involved migrating all of the images into a new, consolidated database that contains all of the artwork information — including images, descriptions and locations — which is now complete.

Lessons learned and next steps

This was GRIT Toronto’s very first test! It wasn’t just a test of StreetARToronto’s cool new map, but also an opportunity for us to test and validate our process and assumptions. It was really rewarding (and humbling) to get a product in front of our testers, and it led to two key takeaways: one personal and the other quite technical.

We found testers were put at ease and quickly engaged with the map if we asked them to locate a piece of street art near where they lived or worked. Once they saw themselves as a real user of the tool, their feedback became more personal. As we build out questions and guidelines for future tests, we’ll be looking for ways to help testers see what they’re testing not as an abstract product but as a tool in their daily lives.

The test also taught us — the hard way — to include bandwidth as part of our pre-test checklist. The StART map was being developed in partnership with Code for Canada’s Civic Hall Toronto program and members of the local civic tech community. Those of us involved didn’t foresee staging site bandwidth being an issue, but we actually exceeded the cap before we finished all our testing. This forced us to limit the number of testing sessions to four, as opposed to the six we had originally planned.

Finally, we’d like to give a special thanks to StreetARToronto for being open to experimenting with this new (to Toronto) form of usability testing and learning along with us.

Interested in working with GRIT Toronto? Become a tester or even run a test with us. If you have any questions, please reach out to Marisa Bernstein at marisa@cfc-dev.loafingshed.ca.