Tag: A/B Testing


How to Gain Customer Insight… The Right Way

Many mid-sized businesses correctly worry that mistakes in new product introduction or revisions can be very risky. Customer dissatisfaction often results in shifting market share.

The price of being wrong is very high.

When faced with the challenges inherent in customer perceptions of new product introductions or revisions, companies, especially mid-sized companies, have few good choices:

Make a well-informed guess about product features and launch; use online surveys and their inherent weaknesses; or use expensive, costly, and time consuming focus groups.

We’ve developed a new way to effectively gather information that is accurate, inexpensive, and dynamic to the day to day shifts in your business. We would be glad to demonstrate in a no-obligation conversation and assessment how this may work for you.

Aside from being expensive, traditional focus groups reflect a reaction to a snapshot in time, and only get responses relevant to that snapshot.  The participant interaction has to be well managed to avoid a domineering personality and group makeup is limited to a geography and/or a collection of people willing to devote time to the process.

Real time moderator skills are essential as the group only persists for a couple of hours and nuanced followup is often not possible.

This “single-shot” qualitative approach can provide impact, but is limited in that insights that were relevant at a time can quickly become obsolete, follow up is not possible, and participant segmentation is not easy.

At Keane Consultants, we turn the traditional model on its head. Through dynamic, virtual online focus groups we give businesses the ability to rapidly test and reiterate concepts.

Groups are built on specific participant criteria.

Members dialog over a period of weeks on specifically assigned topics and interact with each other in a thoughtful, persistent way.

As results come in, a moderator can ask for direct feedback on the concept and re-test based on what the group produces. This constantly iterative process employs the type of analytical strategies that have only previously been available in quantitative research settings. The new qualitative model marries the voice of the customer with sophisticated analytics, while following the constantly changing state of a company over time.

Here’s an example:

One of our clients sells to elementary schools. We gathered a group of teachers in our best performing, previously identified segment (based on our customer analytics algorithms), and asked them to commit to a month-long engagement in an online focus group.

In the first week we gave the teachers a survey and discovered that the product we are selling needs more teacher instruction on how to use. That week, the company created an infographic that helped explain each of the steps and what was required of the teacher. In week two we showed them that infographic and asked them to comment on its individual aspects. By week three we we had a refined, teacher approved infographic that we immediately implemented as part of the product.

Under traditional circumstances, the company would have only realized the initial insight that it needed more instruction. At a fraction of the cost, we were able to take that insight and produce multiple iterations of its solution while, at the same time, asking our group a series of other questions about its preferences.

The New Qualitative Model

Recruit It is imperative that a company uses a segmentation model to guide its recruiting processes. The most relevant segmentation method will vary, but it is absolutely crucial to the recruiting process in order to marry the results of the conversations to the eventual implementation of the concepts.  We build these segments from both specific customer behavior and market data.

Beyond careful selection and random sampling, the number of participants must be significant. For the average group, twenty is a solid number. A good moderator will encourage participation over the duration of at least a few weeks. By offering compensation – usually no more than $50 depending on the product and title of participant – group members will remain engaged.

Iterative Questioning and Discussion – The content of each activity is obviously exclusive to each company, but a consistent engagement over several weeks testing multiple iterations of a concept is universal. The varying methods of discovery at this phase include, but are not limited to, surveys, content discussion, and product and messaging ranking.

After each wave of activities the company can internalize the information, make adjustments accordingly, and test the new ideas in the marketplace.

Quantified Reporting While immediate insights are digestible at a granular level, week to week, activity to activity, the overall pattern of responses at the conclusion of a group’s engagement tells a story that can transform the overall marketing trajectory of a company.

Depending on the format of the group, it is easy to track each of the participant’s responses and tag them with the according segment and category of response. Some examples response categories could be “emotional,” “functional,” or “aesthetic.” Paired with participant segment labels, the aggregation of these descriptive tags tell a story that goes beyond an informative response to specific material. It gives companies insight on the behavior patterns of specific segments. It’s possible that young men in large cities with median income respond in a drastically different way to the overall stream of questioning than elderly women in the country. The ability to pinpoint what these differences are, and act on them not only on specific projects, but throughout company-wide targeting strategies can be transformative.

By implementing this model at Keane Consultants, we have seen the type of transformation that is possible. It can at first seem astounding what this level of engagement and reporting can do for an organization, but the logic of it quickly becomes obvious.

Contact us for a free assessment at sean@keaneconsultants.com or at (414) 737-3644. To hear what our clients have said, check out keaneconsultants.com.

 

All Growth Strategy is Quantitative

All Growth Strategy is Quantitative.  You can’t grow what you can’t predict.

I was in Ireland not too long ago in a meeting of a company board of which I’m a member.  The company  had a very good year – a great one, really – based on a executing a good strategy very well.  Success has been several years in the making, and their efforts have now paid off for them.

In most companies, growth challenges are greatest at sales of $10MM to $15MM.  Growth is impeded by a company’s inability to extend its reach and effectively attract resources required for growth.  The big question on the table is how to move from efforts based on great experience and instinct to  systems, processes and structures that allow growth to continue as more people and resources are added to the business.

In other words, become more quantitative.

This means moving to rely on accurate factual data to teach managers how to make key decisions that incorporate both their best instincts and and quantified facts.

Examples include marketing analytics, sales forecasting, and disciplined unit economic analysis (cost accounting) for all non-sales related activity.

In this introduction to managing by analytics, I cover three topics: Segmentation, A/B Testing, and Comparables. In this short blog, you will learn how to segment customers, how to determine an optimal approach for your target, and how to compare your results.

Segmentation

In the ecommerce world, there’s lots of talk about “growth hacking.” Whether you like the title or not, the point is that solid, quantitative discipline will produce much better results than any other process.  Any company who regularly acquires new customers, or would like to, can use these same quantitative disciplines whether they sell by e-commerce exclusively or not.

Today, tracking acquisition – where each incoming sale or lead came from – on your website isn’t only easy; it’s an entry level requirement.  This is a preliminary step in the segmentation process — describing in some way the commonalities and differences in each group of new leads/sales/customers.

This leads to several questions – what are the better segments (meaning ones who produce more growth per dollar spent)?

The next question, tho, is critical – how can you generate more from that source?  How do you get from these “better segments” to finding more of them in the marketplace?

One way is to use the same sources you’ve been using — SEO, advertising, social media, adwords, websites, and the like. This works and can be quantified in cost per lead/sale/customer.

However, if you want to proactively solicit potential prospects, you need to be able to identify them – and reach them – you need to be able to identify them by findable descriptions that can apply to the entire market.

Segmentation is based on being able to apply the segments to the universe of potential customers and economically find the segments your product appeals to.  This may require a combination of social, SEO, adwords, and other tactics that, when combined, result in a formula that is predictably repetitive.  Or it can be a description, along behavioral or demographic lines, that point to people or sources that can be “found” in the marketplace.

Good information like this will increase the velocity of growth, as well as the predictability of future results.

Analytics make growth repeatable.

 

A/B Testing

What offer or product description appeals most?  This is basic A/B testing.  Every  program or campaign – whether inbound or outbound – ought to present, randomly, many different approaches.  (We often set one up to be what we currently see as the winner, the “control,” and others, served to a smaller percentage of incoming leads, “the tests.”)

Think about the power here.  The presentation, the copy, the method of presentation – all can be tested – and pretty much in real time.  Results are evident in hours and days, and the “winner” quickly becomes the control.

What would happen if…

The A/B test means you can finally know:

  • What would happen if we changed the pricing to volume-based?
  • Is the case study or the benefit description more powerful?
  • Do we ask for a number to call, fill out a form, institute a realtime chat, or just let people call us?
  • What would happen if we incorporated a free trial/test/offer/sample?
  • Will more customers buy a bundled offering?
  • What if we target inquirers with an “only for them” offer each day that expires in 24 hours?
  • How frequently can we communicate with customers?

These kinds of questions can be answered in a day or two and no longer are the subjects of endless meetings, debate and hypothesizing.  You can get an answer and move on.

Comparables

We use comparables every minute in our lives.

A good comparable is a standard – the best margin, the highest sales rate, fastest result.  More important it tells us what the best possible has been in a category.  (I’ve seen many business plans that rely on results that are two or more times better than the best historical comparable – and the author didn’t know it.  While it’s not impossible to achieve, it’s imperative to know what the height of the bar is.)

Good quantitative growth management comparables mean we know:

  • what a good lead/sale/new customer (as defined by our segments) costs and what others are paying for it;
  • what we and our competitors are spending in marketing per dollar of revenue;
  • what our customer acquisition costs are and what range theirs are in;
  • how our product costs line up with theirs.

Sources of this information run from published reports to trade associations, web data from collection and reporting sources; competitive spending on AdWords and other bid-based products; direct customer feedback.  Every growth system must systematically track comparables performance.  These comparables can come both from our own “best” historical examples and from examples in the market as a whole.

In many pre-growth companies, measurement is budget-based.  Budgets set goals that the business has to achieve but don’t provide any information about comparable performance.  Without comparables being included in the budgeting process, both on the revenue and cost side,  it’s hard to tell if the business is achieving more than it should, or less.

———-

Every business should know the cost – in time and money – to acquire a customer, how much that customer will produce in revenue, and for how long.  This gives them the ability to identify customer segments and target acquisition cost, budget for results, and measure performance.

Unfortunately, most don’t know the cost of customer acquisition.  

We do.  When you partner with Keane Consultants, you get a proven track record of success in bringing costs down and driving win rates up.   www.keaneconsultants.com