How we are A/B testing in USA

For our users to benefit as much as possible together with our client from USA we are A/B testing their app, trying out different solutions and searching for the best outcome!

polco-recording2

Starting with MVP

To start this history about testing we need to say a few words about the application itself. Over a year ago several guys from the USA came to us with this idea. They wanted to build a tool to help city officials communicate better with their citizens. Using simple publications, they would be able to find the answers to people’s needs. They were in desperate need of specialists who focus on user-centered design. With the help of our designers and developers they were able to create an MVP which they could show to certain groups of city officials, and even sell it at some point!

Polco previous homepage version

This MVP has become living proof of a bigger concept, which could sell the idea to a wider audience.

polco testing
Polco previous homepage version


Next stage

After that, the Polco Team began to bring the app to the users. They spent six months gathering extensive feedback from the community around them and the potential users of the software. Research showed that the spectrum of users was very wide, stretching from young students to middle-aged officials. They also checked out different operating systems and devices, like mobiles and desktops.

After this the team came back to us with another problem to solve. Or should we say an improvement to make? Because that’s what it really was.

testing polco
Research, research, research…

We had a specific problem to solve – newcomers did not understand what Polco is all about.

In this version we have for the time being concentrated on the homepage architecture and on working out how to improve it. It occurred to us that people did not know what Polco is about. After analyzing the report provided by the Polco team in depth, we could begin our work. We proceeded according to our detailed plan. At this point we could skip our product vision and moodboards because they were no different from the originals (1st version). But after receiving some new feedback from the creators of Polco we prepared two frameworks, each representing different approaches. The main goal of these was to introduce Polco to the users in the most effective manner.

polco testing
Polco new homepage option A

 

Option A was focused on the sending of quick messages. With this option the user can watch a short video. Using animation in order to explain Polco is a simple tool to win the users’ hearts and show how big an impact a simple solution like Polco can have on local politics. Without going into detail, it merely shows the “pure power” of the concept, and how easy it is to use.

 

polco testing
Polco new homepage option B

 

 

 

 

Option B was more sophisticated. We decided to use the storytelling approach here. Why? Because by this means we could introduce the user to three important things:

  • showing an actual example and its positive outcome (hey, it really works!);
  • clearly demonstrating the mechanics of the app and its ease of use;
  • showing the real impact of this simple app.

 

At this point we came up against a tough decision, and to be honest we were no oracles. How would we know which one would better suit the needs of our users? That is why we decided on A/B Testing.

Testing in progress

Now, after 3 months of gathering data we have started getting some results. We have connected Mouseflow, a tool that helps track users’ behaviours. We are testing our designs in around twelve US cities, all over the country. We are tracking real users, in their real environment. We have obtained nearly 3k recordings and over 6k pageviews. We have learned a great deal about our users’ behaviours, their profiles and their needs. At this point we are processing all the data we have gathered. As soon as we have some reliable data we can adapt our project to the needs of the users and make it change the world for real! We are currently working on updating the app itself, starting from the information architecture and ending with new and improved designs.

polco testing

 

Stay tuned as we will stream the progress of our work as the app evolves and let you learn from our experiences.

If you want to know more about Polco visit this link

Enjoy!