This week, whilst struggling through "man flu" I was listening to Inside Health on Radio 4 (the hypochondriacs choice!). One item was a study into the use of paracetamol for moderate lower back pain which suggested it was only effective for 1 - 5% of sufferers. They followed this up by referring to a study in 1977 that showed better outcomes in heart attack victims that were not given oxygen in comparison to ones that received the standard treatment. In this cold and flu season, I have seen many articles questioning the efficacy of over the counter remedies such as cough medicines. Not surprising if you look at the ingredients list that they only contain what is effectively sugar syrup.
In medicine, the gold standard in proving the efficacy or otherwise of a treatment is the double blind placebo controlled trial. Essentially subjects are assigned either to a group that will receive the actual treatment or to a placebo group given an inactive substitute. No one, patient or doctor, will know which group they are assigned to until the end of the trial. The data is then processed to see if there is a statistically significant difference between the groups. At first glance, it would appear to be immoral not to provide a potentially beneficial treatment to patients. However, without testing, we will never know if the treatment is effective. It's surprising the number of treatments that are in common use have never been tested using this method and yet still practiced and regarded as effective.
Consider the tactics employed in your business. Are they effective? Could they be more effective or are there other approaches that could be more impactful? Without robust testing, you will never know. Replacing or adapting a tried and tested tactic is a risk but without doing so you will never know if they are as effective and efficient as they could be. When you consider testing of any tactic, online or offline, design the test taking the following into account:
Measuring a Baseline – often new treatments are tested both against a placebo and the current best practice. In your business, you need to establish a baseline. What are the results from the current tactics; leads generated, conversion rate, typical spend etc. Knowing this you can quickly identify if a new or adapted tactic is an improvement.
Accuracy – how close your measurements are to the actual value. There has been controversy around some clinical trials where negative results were ignored. The “double blind” part of a clinical trial ensures that, since neither patient nor clinician knows whether the treatment given is active, then there can be no interference with the data. It’s vital to record all results so that you have a full picture and any preconceptions of the success or otherwise of the new tactic colour your data
Reliability – refers to the repeatability of results; if you ran the test again would you have the same results. The first consideration here is in controlling as many variables as possible and changing and testing only one feature of your tactic. In the case of a sales letter or flyer, the headline only being changed. The second consideration is the numbers of results, with few numbers in the test results differences can be due to random factors but with significant numbers of both versions, you can be more certain that a result is meaningful.
Validity – measuring what you intend. If you decided to look at a lead generation tool that was sent out via email and measured open rates to gauge success. However, as with all activities, it is the return on investment which is the acid test. In the email example maybe more are opening but not taking the next steps and becoming a customer. You could be generating more leads and converting to customers but these have a lower spend than you have measured as a baseline.
Testing boosts the effectiveness of any of your client acquisition tools and can ensure every stage of the customer journey is as efficient and effective as possible. Without testing what you are currently doing you are almost certainly leaving money on the table.
Testing is at the heart of the 6 Challenges which have been developed to help business look at 6 areas that impact your business. By reviewing and testing you can ensure each is as effective and efficient as possible.
PS Feeling much better now without any snake oil or quackery, isn't the human body wonderful! Don't rely on a wait and see approach for your business. Try out the challenges here.