In today’s study, you will see the results of a large-scale investigation we did into AMP. I set out to learn as much as I could by speaking with 10 different companies about their experiences with AMP. These were all companies that had released an AMP implementation on a major section, or nearly all, of their pages.
The TL;DR, for those of you who want to avoid diving into all the details, is that for companies that followed through with a complete implementation of AMP, pretty much all had good to very strong results. However, how I’m defining a “complete implementation” might not be what you expect (you’ll have to read on to see what I’m talking about).
In our study, each participant shared a lot of details about their experiences with AMP, including implementation issues, and the resulting metrics. As a result, this study should help you understand:
- The potential benefits from implementing AMP
- How much effort it will require
- Solutions to some of the challenges
- What your potential ROI might look like.
Recap: Why Speed Is So Important
You’ve heard it many times: Speed on the web is important. Many people don’t fully internalize that for most mobile devices, the challenges with page load time are materially greater than they are for most desktop environments. One of the core goals for AMP is to level that playing field. What are the practical advantages of faster sites? Let’s have a look:
Clearly speed is a big deal. You can already see how much this matters to users from that data alone. But people are also beginning to recognize the AMP lightning symbol in the search results, as shown by this poll running on 9to5google.com:
AMP indeed does make sites quite a bit faster. I go into detail about why this happens in my recent article on whether your website should adopt AMP. So, in theory, AMP should offer significant benefits for people who implement it.
Looking beyond speed, why AMP works for news sites is pretty clear: there is a huge incremental traffic opportunity created by the AMP News Carousel (also known as Top Stories on desktop devices). Placement in that carousel is available only to AMP pages, and it represents a huge UI change from the normal SERPs, as shown here:
The results that follow from the news carousel are compelling. Here is what we saw for results across three major news sites:
Thrillist is an online media brand covering food, drink, travel and entertainment. The company was founded in 2004 and is based in New York City. They converted all of their article and venue pages, representing 90 percent of the pages on the site.
Their first pass at AMP took one week to do, but that had only very basic styling. The second pass took three weeks, which included a lot more styling, and implemented more complex things like event tracking and social sharing. This second stage took place in November of 2016.
Unlike other early adopters, they did not have major issues with ads, but this was helped by the fact that they sell their own ads. For historical background, other companies had issues with availability of advertisers because of the AMP requirement for everything to run https, which some ad networks were slow to support. That issue is largely gone now.
Overall, Thrillist has seen a 70 percent lift in traffic, with 50 percent of that growth coming from AMP. Needless to say, they are very happy with the ROI on their AMP investment. A detailed case study of the Thrillist experience with AMP can be found here.
Another large media publisher (an anonymous participant in the study) also implemented AMP on a custom platform. They ended up converting 95 percent of their articles to AMP. There were some that they could not easily change to AMP because they contained more complex interactive components that weren’t easily converted.
This publisher is highly metrics-driven, and revenue per page is something that they monitor closely. In the early days, the revenue per page was lower, because different ad formats and advertisers were slow to become AMP-ready, but this is no longer that much of an issue.
One of their larger sites saw a 20-40 percent increase in Google organic traffic, and the other one saw a lift of around 67 percent. Overall, the investment in AMP for that publisher has really paid off.
AMP Myths
So, is that it? AMP is only for news sites? Or should we all do it because Google tells us to? Do I have to live with a stripped-down version of my site? OK, it’s time to explore four major myths about AMP.
Myth 1. AMP Only Benefits News Sites
The initial versions of AMP placed a heavy emphasis on news sites, and the AMP News Carousel (a.k.a. the Top Stories block on desktop devices) gave them a major traffic incentive to participate. But it turns out that many other types of sites can benefit as well.
One example is the story of NoBroker, a company based in India that matches up renters and potential tenants, without requiring the use of a broker. They launched their AMP implementation in December of 2016. As a result, the pages per session and session time on the site are both up 10 percent, and their bounce rate is down 18 percent.
Perhaps more importantly, the number of connections that they make between tenants and renters is up 77 percent. This is an awesome result! You can read the full story on the NoBroker experience with AMP here.
The Lesson
AMP can work for far more than just news sites. It’s a great thing to do for commercial sites as well.
Myth 2. AMP Can’t Be Used on E-Commerce Sites
One of the big issues with e-commerce sites is that they typically require a great deal of interactivity. The fact is that this used to be a legitimate complaint.
But the arrival of the AMP Live List Component and the AMP Bind Component changes all that. This has enabled a new world of interactivity and dynamic components that were previously not possible.
Consider the example of Myntra, the largest fashion e-tail site in India. They did their AMP implementation back in May of 2017. It took them six to seven days to convert the main product listing (category) pages of their site, and three to four more days to convert their home page.
The product listing pages account for two-thirds of their organic SEO and SEM traffic, and the home page is responsible for another 20 percent. In addition, they built out a progressive web app (PWA) implementation for the rest of the pages of their site.
This plays an important role in their site speed story, because PWAs include a component called a service worker that preloads pages in the background, before a user clicks on a link to request them. As a result, when the user request for a new page comes through, that page is already partially or completely loaded, and this also offers significant performance benefits.
The load time of the product listing pages has dropped by 65 percent, and the bounce rate on those pages is also down 40 percent. The overall mobile revenue contribution is also up.
The AMP-bind component was a big deal for them, because it allowed them to implement robust sorting and filtering of products. Myntra is continuing to expand their use of this component. In addition, as of late September 2017, they launched a PWA plus AMP (a.k.a. PWAMP) implementation, so their PWA pages are now also all AMP compliant.
Overall, the results for Myntra were very strong. You can read more about the Myntra AMP case study here.
The Lesson
AMP can work really well with e-commerce sites. In fact, it can offer strong conversion advantages. Users like speed. Faster e-commerce sites will convert better.
You may be tempted to say, sure, it works well for Myntra, because many users in India are on 2G phones. But remember, people like faster sites.
I also got to speak to Pete Dainty, Senior Director at Ebay, and while we weren’t able to complete full profile for this study, he did offer this comment:
Ebay has invested heavily in AMP, and remains fully committed to it. (We are) focused on rolling AMP out to more of our site, and we know that speeding up pages helps improve the customer experience leading to better e-commerce conversion.
Think that UI compromises might still be an issue? Read on to the next myth.
Myth 3. Build it and They Will Come
There are people who look at it this way. They want to jump on the bandwagon early, because they want to ride any significant wave that Google sends out.
One of our study participants, Beach House Center for Recovery found that out the hard way. Their site is on WordPress, and they did an initial AMP implementation with the AMP Automatic plugin. That did not offer the level of customization they were looking for, so they switched it over to the AMP for WordPress plugin.
The initial metrics resulting from this were not good:
However, it turns out that this was due to two things that were not right with the implementation, and these easily explain why the metrics looked weak:
- The analytics setup did not include the “session stitching” fix, which essentially means that the tracking was simply wrong. You can learn more about what the AMP analytics problem is, and how session stitching fixes it, here.
- The main menu on the AMP page was broken and was showing an illogical set of pages. Hence users who clicked on the hamburger menu were not likely to click on anything there.
- The pages had no access to the sidebar menu or any CTA at all.
These three issues easily explain the reason for the poor initial metrics. We worked with Beach House to help them resolve all of these issues. In the near future, we should have updated metrics to see how the changes we implemented will help improve the overall metrics. You can read the full story on what went wrong with the implementation, and how it was addressed, here.
The Lesson
If you’re going to implement AMP, you MUST follow through to get the full implementation completely correct. In the case of Beach House, the session stitching fix was not even available when they initially built out their AMP pages, so it meant that accurate tracking of what was happening on AMP pages was simply not possible. Now that the fix is available from Google, and it has been implemented, we should be able to get better stats on it.
Myth 4. AMP Sites Compromise My Site Design
This is one of the most popular myths. In fact, look at the 9to5Google graphic I shared above again. The wording of the first response is fascinating: “Yes, I prefer the stripped down versions of websites when reading something”. Stripped down, really? Let’s explore why this does not need to be the case.
Here is where there is one element of truth to this myth: If you do use non-AMP elements in an AMP-iframe (this would typically be some sort of interactive element), you must place it below 75 percent of the viewport height, unless you have a placeholder, and then you can put it anywhere on the page.
One reason why this myth is perpetuated is that the WordPress plugins DO give you a stripped down version of your pages. Here is an example of an AMP article, generated by the plugin, on the Stone Temple website, compared to its AMP page:
So yes, it looks stripped down. But is that an artifact of AMP, or an artifact of the plugin? I decided to find out. Working with our web developer, we were able to generate a page that looked a great deal like our mobile responsive page:
You will see some minor differences between the two pages, but all of them could have been resolved with another hour of two of effort. In creating this page, we proved the point: There is no need for a UI difference in your AMP page. In fact, I’d argue that the “Toggle Sidebar” feature we added to the AMP version makes it a superior UI. Better still, the page we generated by hand was fully responsive, so we can even use this page as our desktop page if we want. Last, but not least, here is how the sizes of the three versions of the pages compare:
As you can see, the size of the handcrafted page compares well to the other two versions of the same page.
The Lesson
There is no reason for your AMP page to look different than your mobile responsive page. It’s just a question of the level of effort that you put into it. And here is the punchline: If you have two versions of a page that look identical, and one is much faster than the other, which one will users like better? No, it’s not a trick question. They’ll like the faster one.
Summary of Findings Across All Participants
Having looked at the implementations for 10 different third-party sites, here are our aggregate findings:
The items with a yellow background showed negative results, but both of those were caused by implementation problems. One was marked red because they have not managed to get the pages launched yet. They launched an initial version before getting the implementation completed with their full UI, and the results were not good, so they pulled it back.
For this particular site, getting the right implementation is proving to be hard. However, they’re still pursuing it, and they’re expecting to relaunch the pages soon.
The other “negative” example is Beach House, and I noted earlier, I believe we have helped them resolve the issues, and they are well on their way to deriving positive results from their AMP experience.
Every other site publisher we spoke to had derived something from good to great results. There is one other site that believes that the return hasn’t been great, but per their own admission, this is because they were very early adopters, and the investment they had to make to get there was high. They have actually gotten good results in the end, but they went through a lot more effort because they were working with AMP when it was not as developed as it is today.
What It Takes to Be Successful with AMP
Having taken on the experience of coding AMP by hand, and our overall experience in helping companies with AMP implementations, as well as what we learned during this study, here are some of my key observations about being successful with AMP.
1. Get the Analytics Right. If you are using Google Analytics, session stitching is a critical issue. Without it, your analytics will simply be wrong. In a nutshell, Google Analytics will see the visitors to your AMP pages as being on a different site. Your AMP pages will show poor user engagement metrics as a result. To measure how you are doing with your AMP pages, you need to get accurate data!
2. Perform a Complete Implementation. Depending on your site, this may be a difficult proposition, but it’s worth it. Let me explain it with a simple thought experiment. If you have a mobile responsive page that loads in six seconds, and an AMP page that has an identical look and feel that loads in one second, which one do you think will get better user engagement?
3. Be Prepared to Deal with Exceptions. In working on the Stone Temple site with some outside help, we were able to come up with a basic template for one of our blog post pages pretty quickly. Then I worked on building out a program to replicate that template across other blog post pages, and that’s when I started running into one edge case after another. One page had a video, another page had a special “As Seen In” image block in a table, yet another had block quotes—all things that weren’t standard on our pages. I have many other such examples. It rapidly became clear to me that this was where a lot of time could be spent on some sites.
4. Learn How to Integrate Dynamic/Interactive Components. If you’re dealing with a lot of interactive or dynamic elements on your site, be prepared to learn the various tactics for dealing with this. Some things you may need to look into include:
This set of tools gives you a wide range of approaches to deal with many complex website interactions and dynamic behaviors.
5. Validate Your Pages. Take the time to validate your pages before pushing them live. I prefer using the AMPproject.org validator, as it’s highly interactive.
You can also use the one in Chrome Developer Tools if you’re more comfortable with that. You can reach that by clicking on “More Tools” and then “Developer Tools” in the Chrome Menu.
Recent data from SEMRush suggests that most AMP implementations have validation errors, and that basically means that those pages will not get accepted into the Google cache, and therefore won’t show directly in the search results. As a result, the effort to implement AMP pages ends up being totally wasted. So, DO take the time to validate!
Summary
Overall, I believe that AMP will offer very strong benefits to everyone who takes the time to truly do it right. And if your implementation is done with care, it can be made pretty scalable. As you’ve seen in the discussion above, the parties involved really did not spend a tremendous amount of engineering effort on AMP. Most of you who are reading this probably won’t require a ton of effort either.
