’Tis the Season to Optimize

It’s almost the most wonderful time of the year! And we know it’s also retail’s busiest, so we’re here to help you get in the holiday spirit and also make sure you get the most magic (and ROI) out of your personalization platform. As holiday shopping is on everyone’s mind, RichRelevance’s very own Kris Kringle, Alex Ciorapaciu, shares some tips and insights on how you can maximize on personalization (and cheer) this holiday season.

In this series of short videos Alex has personally selected some optimization tips to help set you up for success this holiday season.

Videos in this holiday series include:

Boosting Product Attributes
With days dedicated to limited time offers and promotional pricing, consider creating holiday or even ‘deal’ specific rules to ensure promoted products and categories are top of mind across your site. Alex’s step by step guide shows you how to create holiday rules within minutes right in the RichRelevance dashboard.

Customer Preference Center
With Customer Preference Center (CPC) enabled, shoppers can explicitly tell you what products, categories or brands they like and dislike and RichRelevance harnesses that information to influence the weight algorithms place on those attributes or categories. CPC helps make personalization more robust, but we never want to confuse personal purchases with gift purchases. Watch Alex’s video to uncover a quick way to eliminate this concern.
If you don’t have CPC enabled and want to learn more about it please reach out to your account manager discuss setting it up.

Holiday Strategy Messaging 
Personalization comes in many forms and in many ways what you say is as important as how you say it. When it comes to recommendation messaging customers say that when brands use their own language they’re 35% more likely to engage with a recommendation placement because it feels more genuine to the brand. Customizing your strategy messaging is easier than ever within the RichRelevance Dashboard, Alex shows you how.

King of the Hill
Ever wondered how RichRelevance algorithms and strategies really work? How they know which strategy to show to which shopper? In Alex’s quick video he explains how King of the Hill eliminates the need to worry about manually manipulating recommendations.

Dynamic Landing Pages
With RichRelevance you can create dynamic landing pages for any segment you dream up. Alex walks through how you can ensure you’re getting the most money from your holiday search engine advertising spend with custom holiday landing pages by creating rules with specific date ranges or referrer URLS.

Ready to unwrap your holiday tips? Watch the video series here.

Decoding Your Recommendations Performance

Product recommendations, also known as “recs,” are a cornerstone to an effective ecommerce merchandising strategy. When fully optimized, recs typically increase retailer revenues by up to 5%. That’s a substantial contribution when extrapolated across annual sales in the hundreds of millions or even billions of dollars. However, one thing I’ve noticed while navigating the world of ecommerce personalization is that the word “performance” is often misused in the context of recs. The reality is that there are multiple meanings to this word, and for each meaning, there is a discrete method for assessing its relationship with your shoppers and your revenue. So, let’s decode your recs performance.

At RichRelevance, we group performance into three analytical categories: value, engagement, and cohort analyses. While each type is useful, each tells us vastly different things. Misappropriating the type, like confusing an engagement analysis for a value analysis, can lead to disastrous business decisions. I’m here to protect you from making such mistakes.

First, let me summarize the three types of analysis:

  1. Value Analysis: this indicates the incremental revenue impact of recs, and is usually the report of most interest to retailers. The key metrics are typically revenue per session (RPS), conversion, average order value (AOV), and the lift contribution is determined by running an A/B or multivariate test that compares key performance indicators (KPIs) of populations based on their exposure to recs.
  2. Engagement Analysis: this quantifies shopper utilization of recs, and can be a signal for relevance. The key metrics are recs sales and clickthrough rate (CTR). Recs sales represents the sale of items clicked on in recommendations.
  3. Cohort Analysis: this profiles the spend patterns of shoppers that choose to use recommendations vs. those that do not—comparing the RPS, conversion, and AOV of shoppers based on their engagement with recs.

Most retailers stumble with using engagement or cohort analyses interchangeably as indicators of value. I get it; these reports are readily available, whereas conducting a true value analysis requires an A/B test which can take weeks and expose the retailer to undue opportunity costs. However, the reality is that these reports don’t convey the incremental benefit of recommendations, and here’s why:

Disassociating Engagement and Value Analyses
An engagement analysis tells us how much shoppers use recs and, to some extent, indicates relevance. If the recommendations are random, shoppers won’t click or buy from the retailer. That said, striving to maximize recs sales or CTR is equal to saying that recommendations are the most important piece of content on your site—and we all know that they’re not. What’s most important is getting shoppers to convert with the highest level of spend. Recs merely support that objective.

In fact, certain kinds of recs engagement can have a neutral or negative impact on your business. It’s the responsibility of the recommendation technology and how it’s configured to mitigate these instances. To shed more light on the matter, here are three events you should be aware of that explain why value is not a function of recs engagement or sales:

  • Overlap: Sale of items that would have happened anyway, even in the absence of recommendations. Frankly, the vast majority of sales that happen through recommendations aren’t incremental, so indiscriminately increasing recs sales does not guarantee more cash in your coffers.
  • Cannibalization: Reduction in AOV due to the presence of recs. If recommendations cause a shopper to buy a cheaper SKU than they otherwise would have purchased, that takes money out your pocket as a retailer.
  • Shopper Distraction: Reduction in conversion due to recs. If recommendations are optimized to attract as many clicks as possible, they can bait shoppers to click in perpetuity until they become fatigued and leave your site without converting. No one wants this.

And of course, we have copious amounts of data to demonstrate that maximizing recs engagement has a detrimental impact on revenue. We’ve plotted recs CTR and recs sales against RPS lift for a multitude of RichRelevance A/B tests, and the resulting scatterplot shows no positive correlation between engagement and incremental revenue. In fact, the data suggests that at extreme levels of recs engagement, RPS lift can be severely compromised. For more details, view this interesting TED Talk on the “paradox of choice”, which implies that oversaturating a shopping experience with product options and baiting shoppers into excessive exploration can result in non-conversion.

Quite simply, it’s not about maximizing recs engagement; it’s about driving the right level of engagement that minimizes the aforementioned events and maximizes your per session revenue.

Disassociating Cohort and Value Analyses
A cohort analysis tells us the RPS, conversion and AOV of shoppers that choose to use recs versus those that do not. Most often the shoppers that choose to use recommendations have substantially higher key performance indicators (KPIs), which can lead one to presume an extreme level of benefit from recs. That said, please consider that although recommendations do generate substantial revenue lift when properly optimized, a cohort analysis does very little to quantify that impact.

In the business-to-consumer world, shoppers that utilize recommendations tend to have higher spend propensities, so it’s not uncommon for recs users to have 2x the RPS of non-recs users. Intuitively, this makes sense. When dealing with household consumers, willingness to engage with your catalog is a primer for purchase intent. In the business-to-business (B2B) world, the inverse is true. B2B buyers have predictable requisition lists that they rarely deviate from. Consider an office manager that’s restocking his company’s office supplies or an IT manager that’s purchasing specific laptop models for her business. These professionals know exactly what they want, order in bulk, and do not use recommendations.

So, these non-recs users have high RPS, conversion, and AOV. Conversely, the recs users represent those shoppers that are less committed—those that may or may not know what they want, and if they do convert, it’s with much less spend. For sites like Staples, Office Depot, or Dell, these are most usually your household consumers, and a cohort analysis would report much lower KPIs all around. Therefore, due to a self-selection bias, whether a cohort analysis shows higher or lower KPIs for recs users is independent of the value the recs technology actually delivers. Rather, it’s indicative of the types of users that decide to use recommendations.

Are recs impacting your best shoppers? Do recs help monetize individuals least likely to convert, or do they have broad resonance across all levels of spend in your shopper base?

So, there you have it—your performance analyses decoded to help you learn a bunch more about your recs and answer these questions: Who is using recs (Cohort Analysis)? How much are they using it (Engagement Analysis)? What incremental value does this utilization provide (Value Analysis)?

Book People Case Study

With one of the largest direct competitors, Amazon, Book People needed a solution to wow customers through creating innovative and personalized experiences, without presenting the customer with too much choice.

Read the case study to learn how Book People implemented both Recommend™ and Engage™ to improve conversion rate and remain competitive.

Download Case Study

Leading Marketplace gives RichRelevance Top Review

I wanted to highlight a particularly interesting case study of RichRelevance customer, PriceMinster. The second most visited ecommerce site and leading marketplace in France, PriceMinister, part of the Rakuten Group, was looking for a personalisation platform to help improve their commercial performance and give them a competitive edge.

PriceMinister turned to RichRelevance in 2012 because they needed a scalable and robust personalisation engine to cope with one of the broadest product catalogues (up to 200 million products) and 15 million visitors per month to their website.

After A/B tests showed a much greater engagement of visitors with the RichRelevance personalised recommendations verses their incumbent provider, plus a 1.5% revenue lift, PriceMinister implemented the RichRelevance Recommend™ solution.

Wanting to ensure they were practicing as much personalisation as possible, PriceMinister went on to utilise the help of one of the RichRelevance personalisation consultants to run optimisation tests in order to further improve performance. As a result they’ve maximised the amount of personalisation opportunities that can occur across their various customer touchpoints (web, email, mobile, etc.).

To find out how PriceMinister got on with their optimisation project and for more results, download the full case study.

Better recommendations are worth $500M

They say you can’t put a price on love, but Netflix CPO, Neil Hunt, recently put a price on the value of better recommendations for his customers, and that figure is half a billion dollars.

In his RecSys 2014 keynote, he said that recommendation systems should not try to be an oracle asking users to “trust me, you’ll love this” but rather advise users that “based on your watching title X, theme Y, actor Z, you’ll probably enjoy this.”

Neil Hunt is pretty spot on, you don’t want to be preached to. The most powerful recommendations are those you inherently trust because they reveal their underlying logic. Many systems leverage black box technology driven by complex algorithms or based on simple rules. Black box applications don’t allow your intuition to complement machine computation and result in irrelevant “You may also like” recommendations, while the basic approach feels forced and overly simplistic.

Combining sophisticated machine learning technology and intuition, leveraged the flexibility of the Relevance Cloud to implement unique selling strategies that directly combat the pitfalls associated with black box and rules-based systems. After implementing RichRelevance recommendations across their site, they noticed that some of the unique qualities about selling and buying wine also make it slightly more complicated to recommend. For example, many people repurchase the same bottles and often purchase in large quantities; upsell/cross-sell strategies tend to recommend popular products since there really is only one product to sell – wine.

With this in mind, began to develop strategies to address the particular nuances of their product set. Starting with a ‘Similar Products’ strategy, they offered recommendations that keyed off attributes unique to wine, like varietal and region preferences. With strategies like these and other personalization efforts, has generated a 15% increase in average order value and $5 revenue per click [View the case study here]. Cam Fortin, Senior Director of Product Development at shares how he implemented his own strategies in this webinar.

When it comes to what works, everyone’s path to personalization is unique. Many vendors offer canned personalization that can be simply turned on. In today’s competitive marketplace, it is imperative that you are empowered to build a relationship with your shopper that is best-suited to your unique business and affords you the ability to test and implement creative personalization tactics. You need to innovate with agility, which is why we brought tools like “Build Your Own Strategy” to retailers like you, eliminating the need to wait on someone else to do the work for you.

Creating the most personalized experience possible requires adherence to one simple principle: respect the shopper. A modern personalization platform should give you the control over your recommendations to deliver experiences that are unique to you and respectful to your shoppers. With an open platform technology, you’re able to analyze and activate your own data in real time to develop more relevant recommendations that drive higher engagement, stronger relationships, and ultimately, revenue.

Contact us at to learn how you can bring human intuition into your recommendation strategies.

Webinar: Build Your Own Recommendations

Are your recommendations as mysterious as a magic 8-ball?

It’s time to rethink your recommendation strategies.

A modern personalization platform should give you control over your strategies to deliver the right upsell and cross-sell opportunities. You should be able to analyze and activate your own data in real time to create relevant recommendations that drive higher engagement and revenue.

Black-box applications don’t afford you the opportunity to use your own data to articulate clear strategies. Instead, you’re stuck with static “You may also like” recommendations that are sterile and lack transparency.

Consider an agile, flexible approach that gives your team the tools to tailor each strategy to reflect your customer data and brand promise. We have put together a webinar on Building Your Own Recommendations to help you learn how.

In this webinar, Cam Fortin, Senior Director of Product Development at, will share how they built their own, highly effective recommendation strategy that resulted in 15% increase in average order value and $5 revenue per click by utilizing the "Build your own strategy" offering within the Relevance Cloud™.

Join us to learn how you can translate your brand promise into tailored customer experiences.


Register Now

Build Your Own Recommendations
Thursday, March 19, 2015

9:00 a.m. PST / 4:00 p.m. GMT


Cam Fortin

Sr. Director of Product Development,

Brad CerenziaAn e-commerce industry veteran, Brad Cerenzia has more than 15 years’ experience as an innovator, designer and engineer working at companies like Amazon and Redfin. Currently, Brad is the Director of Data Innovation at RichRelevance, introducing proofs-of-concept with top retailers eager to establish themselves as market leaders in adapting personalization to such areas as mobile shopping, sales associate on-floor tools, fitting room technology, POS marketing, cross-channel data technologies and assisted personal shopping.

Jolie Katz

Product Marketing Manager, RichRelevance

Jolie KatzJolie Katz is the Product Marketing Manager for the Recommend and Discover products at RichRelevance, creating and delivering go-to-market assets that drive demand and generate pipeline for the business. Prior to RichRelevance, Jolie worked in the Consumer Insights division of the Estee Lauder companies, where she was responsible for analyzing global market and consumer trends to deliver strategic recommendations and insight to a broad range of internal departments. She received her BS in Organization Leadership from the University of Delaware.

1 2