Retail Personalization Index Methodology

The purpose of Sailthru's Retail Personalization Index is to quantify how leading retailers are personalizing their customer experience, which personalization methods are most effective, and the relationship between a personalized experience, customer satisfaction, and customer retention.

Research for Sailthru’s 2nd annual Retail Personalization Index study was conducted between June and August 2018.

  1. Scoring the Sailthru 100 on how they personalize the customer experience
  2. A survey of customers for each retailer to quantify how those customers felt about the brand and their experiences

Choosing and Scoring the Sailthru 100

Two hundred and fifty brands were selected for inclusion in the study based on a number of criteria, including brand prominence, retailer type, and known leadership in customer experience innovation and personalization. Selected retailers include those with a global presence and those with specific presence in the United States and/or United Kingdom.

Sailthru’s team of personalization experts, including individuals with backgrounds in email marketing, website optimization, mobile messaging, business intelligence, analytics, and broader digital marketing strategies developed a list of criteria on which to measure the selected retailers against. The selected criteria, outlined below, included a variety of brand-side and consumer-side activities either in digital channels or using digital data that would be visible to customers shopping with the brand either online or in-store. The criteria list was broken into four sections or channels: Site, Email, Mobile, and Other.

An internal team at Sailthru was assigned to review each retailer in a process called “experiencing the brand,” in which the team would go through the retailer’s site, app, and email experiences as if they were a new customer, keeping track of the efficiency of the customer experience and how well their experience was personalized.

The customer experience and personalization criteria against which brands were evaluated include:

Category: Site

Does the retailer’s site allow customers to create a profile?

  • Do customers on the site see/receive personalized offers based on their buying habits or data profile?
  • Does the site use/display the customer name?
  • Can customers star/rate/like/favorite products?
  • Can customers create wish/watch lists?
  • Can customers get notifications for out-of-stock or preview items?
  • Can customers view in-store availability for online items?

Does the site have a preferences manager?

  • Can customers set preferences for products and recommendations?
  • Can customers set preferences for messaging (quantity and cadence)?
  • Can customers opt out of messaging through the preferences manager?

Does the site have product recommendations?

  • Are there attribute-based recommendations? (e.g. related/similar products, “complete the look”)
  • Are there collaborative filtering-based recommendations? (e.g. “customers also bought,” “people like you bought”)

Can customers chat, either with a bot or a live representative?

Does the retailer have a digital personal shopper or customized product recommendation engine?

Does the retailer site have non-product content?

  • Is that content curated or recommended to the customer based on their purchase habits, reading habits, or data profile?

Category: Email

Does the retailer send a welcome email to new customers?

  • Do they have a welcome series, sending multiple emails?
  • Do they acknowledge the channel the customer arrived from?

Does the retailer use the customer name in email communication?

Does the retailer use collected customer data to personalize messaging to customers?

  • Are the products shown in email messaging personalized?
  • Are the offers or sales shown personalized?

Does the retailer acknowledge the customer’s birthday in some way?

Does the retailer send a message/email for an abandoned cart?

  • Does the message show the items left behind in the cart?
  • Do they show custom discounts or offers, or remind the customer of discounts they have access to?

Does the retailer send emails based on timing or behavioral triggers?

  • Does the volume of retailer email scale with customer engagement?
  • Does the language in the emails reflect the timing of the messaging?

Category: Mobile

Does the retailer have a mobile-enabled website?

Does the retailer have an app?

  • Does the app share a customer’s profile with the site?
  • Does the app share a cart with the site?
  • Is the customer’s name used in the app?
  • Does the app have in-app notifications and rich messaging?
  • Does the app include push notifications?
  • Are these triggered by behavior and/or timing?
  • Do they contain personalized products, when products are shown?
  • Do they contain personalized offers, when offers are shown?

Category: Other

Does the retailer use customer profile data to personalize the in-store experience in some way?

Does the retailer send email or other digital messaging after the customer makes an offline purchase?

Does the retailer have a loyalty program?

  • Does it include a rewards program?

Does the retailer have a referral program?

If the retailer sends direct mail, does it use the customer’s name?

Does the retailer use digital data/purchases to personalize direct mail communications in some other way?

Does the retailer have a presence on major social channels? (e.g. Facebook, Twitter, Instagram, Snapchat)

  • Does the retailer respond to or engage with customers through these channels?

Does the advertiser retarget site visitors/cart abandoners through advertising?

  • Are the products in this messaging personalized?
  • Is the messaging itself personalized? Does it include personalized offers?

Each retailer was assigned a value of 1 or 0 according to whether they met the listed criteria, with notes kept on notable examples.


The Customer Survey

To accompany the scoring process, in 2017 Sailthru conducted a survey of 13,260 adults ages 18+ recruited from a panel by Qualtrics. These respondents were in the United States and United Kingdom and reported shopping with one or more of the brands evaluated in the prior 12 months. Respondents were asked to describe and rate — on a scale of 1 to 5 — their most recent experience with the brand, as well as how well they felt the brand personalized that experience. Respondents were also asked about their likelihood to recommend the brand to a friend or colleague (to obtain Net Promoter Score) and their likelihood to make another purchase from that retailer in the near future.

Creating the Scoring

Once the brands had been rated as listed above, a series of regression analyses were run using data from the customer survey on NPS, experience ratings, and personalization ratings as the dependent variable and the personalization attributes (scored 1 or 0, as noted above) as the independent variables. From this, Sailthru’s research and insights team was able to identify the relative impact of each on the brand’s customer-given scores and used this magnitude to assign a score to each category. This also informed the design to weight the 4 broader categories unevenly, as the analysis suggested that site personalization efforts had a larger impact than the other categories. The final top 100 list ranks the 250 brands that were analyzed in the study based on this scoring methodology.

See Something Wrong?

If you are a retailer included in the survey, and you believe that we have overlooked some of the criteria above, please contact us: we will investigate the error and update our scoring accordingly.

Want To Be Included Next Year?

If you are a retailer not included in the survey, and you believe that you offer an exemplary or innovative customer experience and use of personalization, please contact us to be considered for the 2019 report.