The New Gmail: +1 for Revenue, ‑1 for Opens

Many colleagues in our space have relied on sequential analysis (i.e. before the tab roll-out, after the change) to analyze fluctuations in Gmail open behaviors since the new inbox launched, and the assessments have been fairly consistent: no dramatic changes just yet. We took a road less traveled in our analysis, and the end results suggest that the inbox change most definitely deserves some further scrutiny.

Numerous Sailthru clients are commerce companies, and with those businesses come two important realities: 1) seasonality is a very material issue, meaning that it in no way would it be accurate for us to compare July/August data against sometime before Memorial Day, and 2) at any point in the year, open and engagement rates vary significantly week-over-week (WoW) as a result of changes in available inventory, special offers, sales, etc. We see similar, though less marked, variations with our publishing partners.

So, what did we do to try and understand how the new Gmail inbox is affecting brand communications?

We culled over 70 million campaign messages to look at changes in Gmail trends vis-à-vis the three next populous ISPs: Yahoo!, Hotmail and AOL, and focused on two weeks in particular: the week of 7/15-7/21 (week prior to the widespread roll-out) and the week of 7/22-7/28 (first week with the new tabs); our understanding is that penetration of the new tabs was quite low prior to the July 22, so we opted to omit the early weeks of the roll-out.

Our primary metric was the degree of difference between Gmail engagement deltas and engagement deltas for the other ISPs. For commerce clients, we used a revenue/open metric whereas for media clients we used a pageviews/open figure (ultimately a proxy for ad revenue). We also looked at how the value of any given open changed with the new tab treatment to try and answer the following question…

Are people who open via the new Promotions tab treatment more valuable/engaged because they have higher intent when they do open?

The results:

Commerce

• While we saw a 1.1% increase in Gmail open rates, for the same time period and same messages, we saw a 3.7% lift in opens for Yahoo, Hotmail and AOL in aggregate, meaning that the week-over-week delta for Gmail opens was actually 70% worse than it was for the other ISPs; this is definitely a number we’ll be monitoring more closely.
• Revenue/open was down WoW for all ISPs (again, probably due to inventory or sales), but the engagement dip for Gmail was almost 10% less severe than for the other ISPs, implying that the value of an open may in fact benefit from the change.

Media

• Open rates were down across the board (similar to commerce, could be a function of trending stories, etc.) and the decline in Gmail opens was 94% more pronounced for Gmail users than other ISPs.
• Whereas PVs/open just slightly declined WoW for the other “big 3” ISPs, the value of a Gmail open actually increased by 5.9% (or a 700%+ improvement over the others). Once again, this value metric gives us faith in our hypothesis that the new tab treatment might actually result in increased engagement per open.

In short, based on this ISP-oriented analysis, we have strong conviction that the value of an open will increase with the new tab layout, despite the fact that opens may indeed be suffering – meaning we’ll all need to think hard about the balance of open quantity vs. open quality. With that in mind, until we have a more robust analysis, we recommend email marketers apply their focus to gross revenue per send.

Given the very recent nature of the widespread roll-out of this change, this analysis is obviously extremely preliminary and somewhat provincial, but we thought our approach was worth sharing. That said, we’ll be keeping a close pulse on changes to opens and more importantly, on changes in engagement patterns, and will be updating our analysis accordingly; we’ll certainly keep you all posted about what we see.

—

Cassie Lancellotti-Young is the VP of Client Optimization & Analytics at Sailthru where she is leading the revolution around 360-degree customer marketing and helping our clients maximize their marketing initiatives.