I created a test plan & template to stress test a new algorithm assigning clean scores to each room on iRobot home maps before the feature was fully built.
Role & Team: Principal UX Designer, iRobot
Tools: Mural
Skills: User Testing
Deliverables: Test plan with instructions & Template for recording results
I created a plan and template for the product team to stress test a new algorithm assigning clean scores to each room on iRobot home maps. A numerical score is assigned to each room based on robot sensed dirt events, type of room, and time since the room was last cleaned.
Ultimately, these numerical scores will be grouped into ratings (clean, average, dirty) and displayed as colors on the map like this:
However, while development was working through the final visualization, only the numerical scores were available in a list view for testing the behavior of the algorithm/scores, like this:
THE Challenge
This experience is one that happens over at least a 3 day time period (a room’s cleanliness decays over a 3 day period from clean to dirty). It is important for us to test over at least 3 days, while cleaning different groups of rooms.
This meant that testing would inevitably take time, so starting early was critical to gauging the quality of the experience ahead of release.
The SOLUTION
In order to start testing in tandem with development, I created a suggested test plan (tasks for each day) and a template to record list view results in a visual way, mimicking how they will ultimately show up in the map.
Below, you can see an example of how a tester might fill out the template with cleaning results. The idea was to manually create a quick and dirty version of the expected visual by translating the numbers into colors in mural. I created prebuilt components to make this easy for testers.
RESULTS
This test plan and template were used by our cross-functional product team over the course of several weeks to debug the system, find inconsistencies in the algorithm, and refine the experience to feel more accurate and organic in practice.