An UI in a CRO team world

It’s been a year since I joined a Conversion Rate Optimisation team started from scratch in fact that was one of my goals, working in problem solving, having data about that, think and discuss along with other team members and also design, UX and product teams possible solutions and new ideas.

One of the best things of working in a team like this, is having results and data about your work. How your work affect positively or negatively the business.

Let’s talk about what is a Conversion Rate Optimisation Team and it’s objectives

Identify problems — This isn’t(or at least it shouldn’t be) just a CRO team objective, but is part of our job, to gather the information we have from our users (either coming from pure BI data or UX research teams, Customer Service teams, or directly from end users .. ) list the major problems and start thinking and working in possible solutions.

Discuss possible solutions — At this point the collaboration is definitely the key. Discuss the problem with the team and along side with other teams. Especially with Design and UX teams, trying to achieve one solution that could solve the problem and be quick and simple to implement. This forces you to get out of your daily code routine and start thinking in the product as a whole.

Test the idea — Implement your team solution, and send it to live environment.

Getting results — Gather all the information that you can, the results can come from many sources, such as your Data team, from your Customer Service, from your Operations Team. The greater and the more varied details you have about your test, the best will be your decision about it.

Learn from it — As soon as you have gathered your results, comes the question: is the test a winner or a looser? Was there any event that could affect the results? Can you implement the logic behind that test, in other scenarios and fix other issues with that in mind? Again, collaboration could really help you here, involve persons from other teams and cross the business and let them share their thoughts and feelings about it.

Iterate — There are a few reasons why you should do this, your product is never finished, the world is always in change so as your users, maybe they start using some app like WeChat and they want to be able to make the purchase from there, or something that you did on the way ended up breaking something.

And what have I learned so far?

Maybe start by saying that, if you are expecting that the majority of your tests will be succeed, you should think twice. There is also no magic wand that can fix all your problems and there are no absolute truths either.

Do not get carried away by some myths like “if you do this, or that, you will have an absolute amazing uplift of X% in your CR”. The most probable is that will not happen. Remember that your users are not the same as other websites, your flow isn’t probably the same either, so things that have a huge impact in other websites, might not have a great impact in yours at all. This was probably the first lesson I learned since I’m working in CRO and with AB testing.

Keep your test simple. The more complex is your test, the more complex will be extracting results from it, and even more complex will be learning from that. And even if you want to do it, see if it’s possible to break it into small tests, and interact over it.

Collaboration between teams, not only development, but also from Design, User Experience, Customer Service, Finance, Operations etc etc etc … Your work have impact in all of your company, you can not just give a better experience to your end-users but also help other teams from the organisation. Involving different persons, from different business areas also help you to remember that all are working to the same goals.

Last but not least, do not lose track of your closed tests. Again, remember that as your product is never finished, you could end up doing something that for some reason could damage your results from your closed test.

Dealing with code

Dealing with many AB tests running at the same time, can cause a lot of issues, since performance, possible duplicated code from “control” and “alternative” version, new files added to the project that can be lost in the way, etc.. Also there is more than a way of dealing with those issues.

We as a team have decided to treat a CRO test just like other feature added to the project. This allow us, as soon as we have results, to make the few changes possible to put the test live or in case that the control version wins, put the code exactly like it was before the test as soon as possible in live environment.

With that in mind, we try to isolate all of our code from the other features.

In my particular case, for example all of the css needed to the test goes to an individual file, and only then will be imported to the component/page where are the rest of correspondent styles. In the case that you can’t isolate the code, you can always create a data-attribute that can show you if you are or aren’t in the test, and make your selectors with that. Remember that will have impact in your selectors performance, use that carefully.

This is one of the reasons why you should keep the test simpler, the more complex the test, the greater the likelihood of you ending up with mixed code between ‘control’ and ‘alternative’ versions. And when you need to add some new feature in that component, you are increasing the probability of breaking something or even forget to add that in the two versions.

Also if there is more that a team sharing the same project where you are doing your tests, it will be a good idea to keep them updated in what you are working, what new files you add to the project, and to offer your help in case they are having problems in a merge conflict. It can save you a lot of time debugging ‘ghost’ problems.

Please feel free to add your thoughts and/or share your experience in this AB test world.

Also if you want, you can contact me on twitter

This story is published in Noteworthy, where 10,000+ readers come every day to learn about the people & ideas shaping the products we love.

Follow our publication to see more product & design stories featured by the Journal team.