Transcript of Podcast with Eric Peterson
Podcast Date: September 4, 2007
The following is a written transcript of the September 4, 2007 podcast between Eric Enge and Eric Peterson:
Eric Enge: Hello listeners, I am Eric Enge, the President of Stone Temple Consulting. You can see our website at www.stonetemple.com. I am here today with Eric Peterson, the CEO of Web Analytics Demystified, and we plan to talk about what organizations need to do to be successful in web analytics. You can see the Web Analytics Demystified website at www.webanalyticsdemystified.com. Hello, Eric.
Eric Peterson: Hello, Eric. How are you today?
Eric Enge: I am doing great, how are you doing?
Eric Peterson: Excellent, thanks very much.
Eric Enge: Hey, it looks to me like things are going well for Web Analytics Demystified at least as an external observer. Can you talk about how things are going?
Eric Peterson: Yes, things have been going really, really well. I am tremendously excited about having my own company, about being able to pick and choose my own clients, and can really start to do some of things for the web analytics community that I've wanted to do for years. And, I have just been either time limited or limited by what my employers really wanted me to do and where they wanted me to focus. So, at just about 100 days into it now, the decision to leave Visual Sciences and start Web Analytics Demystified Incorporated has been just great. So, thanks for asking.
Eric Enge: Sure, I noticed since you mentioned the thing that you wanted to be able to do serve community that you didn't do before. One of those things was that free analytic survey that you put out there not too long ago.
Eric Peterson: Yes, and I am about to do a second round of that. I plan on doing those surveys twice a year, one time looking more at attitudes, which was the March survey. And then, in September we are going to look more at tool usage. That's the kind of thing that none of my previous employers, certainly not Jupiter Research, would have said go ahead, spend your own money, conduct research, do the analysis, write this up, get it edited and put it out there for free. Nobody would have ever given me the ability to do something like that, but I have always wanted to do that. At Jupiter Research, I wanted to focus more on web analytics and more of the kinds of problems that people at ground level, like real practitioners, have. But, I had a research program, and not to say anything bad about Jupiter Research, because it was a good research program, but, I can now make those decisions, let's focus on this; let's look at that. And, it's very, very satisfying as I am sure you know at Stone Temple.
Eric Enge: Absolutely. One of the things I saw in that survey is that you came to the conclusion that people who use the free web analytics tool are less likely to dive in and get the deeper value out of the analytics experience. Is that a fair assessment?
Eric Peterson: I have a second piece of free research I put out titled the problem with free analytics.
Eric Enge: Yes.
Eric Peterson: The data suggested that companies who primarily using free web analytics solutions were not really taking advantage of the technology the way they could. More adhoc usage of analytics as opposed to regular programming, a regular program for conducting analysis, employees to manage that or a process streaming approach, certainly companies deploying free solutions were less likely to be paying people to manage those solutions. So, some of the really important things, right, you know this, I know this, Eisenberg and Sterne and Barbie and everybody knows this, that you have to dedicate people. You have to have somebody whose responsibility it is to do web analytics, and that just didn't shine through in the data. So, I speculated that this may result in a very substantial number of companies who are not really getting the benefit; that was the essence of that report.
Eric Enge: Right. And, it doesn't mean that somebody couldn't use a free analytics tool and get the benefit. It just suggests that there is a correlation between those who do use one and don't pursue that level of benefits.
Eric Peterson: Exactly, so not only does it not suggest that you can't be successful with free analytics tools, it doesn't say anything about the value of those free analytics tools. I think of Google Analytics as being the prototypical free tool, and I think Google Analytics is great. I use Google Analytics; I get a lot of value out of Google Analytics. Certainly there are things that I don't like about it, and I fill the gaps there with other web analytics tools. But, it's not about the technology itself, it's not about the tool, and not even really so much about the people.
It's how the people are using the tool, you have to be committed to doing this, you have to be committed, you have to have every intention of using web analytics tools whatever you have, to better understand your audience and better understand your online marketing efforts. It's really that simple. Some people didn't take it that way; some people said that I was bashing free tools. Some people actually said that I was bashing for free tools, but again it comes down to reading the documents and really thinking about what the data says, what the data can tell you about how people are using the tools today.
Eric Enge: Right. Now, I took it the same way that you've just expressed it, but yes you had to dig in a little bit to make sure that you were really reading the whole document. Well, let's dive in a little bit. One of the things I have seen you write about or do presentations on is, it seems like a lot of organizations dive into analytics, and then assume that a continual improvement process will be sufficient. But, I have seen that you made the argument that it's not sufficient, can you explain that?
Eric Peterson: The continual improvement process is sufficient if it is truly a process. What I have seen and I talked about this in San Francisco a little bit at the Emetrics Summit, and I have been talking about it since then. What I have seen is that there are still a fairly substantial number of companies that seem to nod their heads, they bow their heads yes, yes we get it when you talk about continual improvement, but they haven't gone so far as to implement the actual process part of the continual improvement process. One of the most important things to continual improvement is having a testing platform; right is that AB testing or controlled experimentation or multivariate analysis, whatever you want.
Just having the ability to run parallel tests is fundamental to continual improvement done right. And, there are still a lot of companies that haven't deployed those kinds of technologies, maybe they are doing things in serial, but maybe not. I think a lot of times web analytics stops for companies when the reports have been generated, and they don't take it far enough. They don't get to analysis, they don't take analysis to multivariate testing; they don't do the processes behind web analytics. And so, I think that maybe there is not a limitation in understanding, but there is a limitation in use.
Eric Enge: Right. So, what about the management process used that are necessary to take this a step further?
Eric Peterson: Well, the management process used, and I wrote about this fairly extensively in a white paper, again a free white paper that I provide at webanalyticsdemystified.com, talking about the role that management needs to play. It's real common and certainly at Jupiter Research I experienced this and also at Visual Sciences, it's real common to go in to a situation where a company has spent significantly on web analytics tools technology, but don't have a named, assigned senior owner. Right, a Senior Vice President or a Vice President or an EVP or somebody whose responsibility includes web analytics, who includes making web analytics actually successful in the organization and deriving positive return on investment from Omniture or WebTrends or Corematrics or Visual Sciences or whatever they have got, and their people. There are series of management processes that often get forgotten and it's unfortunate, because where you end up is with well-intentioned, well-meaning, right people actually doing web analytics, but nobody is taking advantage of their analysis. Nobody is actually taking it to continual improvement, and taking it to the next logical step of let's do something with this data.
Eric Enge: All reports and no action?
Eric Peterson: Very common, very common: all reports and no action. It's actually, I don't if you attended the Web Analytics Association Webcast that I did last week, but I have just started talking about something called RAMP. I have always been looking for what is the most memorable; what is the most simple way to communicate, how to be successful with web analytics? And, I think it is this, ramp is resources, which is technology and people, ramp is analysis, ramp is multivariate testing, and ramp is process. You take first letter of all those, you get a clever little acronym, because everybody wants a ramp that goes up and to the right.
But, it's all of this, and it's management buying into the RAMP, and it's IT buying into RAMP, and it's analytics and it's everybody saying we can use web analytics and website optimization ecosystem of technologies to be very successful in the online channel as long as we are committed and as long as we have a roadmap, as long as we understand what we are going to do, when we are going to do it, and why we are going to do it. But, the evidence of that, the ability to be successful is everywhere, you know that, it's your clients, it's my clients. It's Jim Sterne's clients, and Semphonic's clients, it's all the vendors' case studies. The evidence for being successful with analytics is absolutely overwhelming, so it's not the technology, and it's not the people that are holding everyone back. It's really I think about the process and about being well-intentioned, and really trying to make this stuff work.
Eric Enge: Right. So, you can think of it as creating a data driven culture?
Eric Peterson: Creating a data driven culture or creating a data driven culture within a larger culture. Talking to Tom Davenport a little while ago and asking him if companies are not data driven from the top-down; are they just doomed, are they not going to be able to compete on analytics? And, what we eventually came to is you can't compete on analytics, but you can compete on web analytics at the microscopic level. So, one department, one group in the organization, will you be more successful if the whole organization is data driven? Probably yes, but it doesn't mean that if you are not data driven from the top-down, you can't be successful with web analytics. It just means you might have to work a little bit harder to not be led by the data, but to use the data to your advantage.
Eric Enge: Right. It's interesting to think about if you are dealing with the person who is currently not particularly sophisticated, and that's probably the wrong word. But, just not knowledgeable in the area of analytics at this point, and they are getting into the RAMP, and thinking about it for the first time. They are looking at a real investment, there is the buying of the tool; there is building up the organization with all the people. It's the cultural thing as it needs to happen and as Avinash Kaushik, famously said that the tool should be 10% of your cost, and the people 90%, and whether you agree with those numbers or not, you are really looking at taking a big step. It's fascinating to me, because you and I are both seeing what the returns are when you do that successfully, but how do you go about educating someone who doesn't have the knowledge and background as to what they can hope to get in return for their investment?
Eric Peterson: That's an excellent question, it's an excellent question. It's something I have been talking about a lot more lately. It really is how you get to it. Web analytics is hard, right. But, I don't know, we should probably have discussed this before we started recording the call, because I don't know how you feel about this. But, I think the web analytics is hard, right. I have a presentation where I go through 20 different examples from the vendors and from authors and even stuff that I have written and stuff that Avinash has written, and stuff that Google; a bunch of stuff from Google that says web analytics is easy, web analytics is easy, Google Analytics makes web analytics easy.
I don't think that's right, I think web analytics is hard, and I think the assumption that web analytics is easy or it is supposed to be easy is actually hurting our industry. Its hurting individual practitioner's ability to communicate to the large organization what has to be done to take advantage of web analytics. When you walk in the door and you say this is easy, we are all going to get it. Then when inevitably people struggle with definitions, when they struggle with data inaccuracies, when they hear about cookie deletion; when they see these new archaic terms, when they look at these interfaces, they think to themselves well this supposed to be easy, but this isn't easy for me.
Eric Enge: Right.
Eric Peterson: I think the recognition that web analytics is hard, and it's something that requires an investment of time and energy and resource is how you get organizations to buying into it. You put together processes, how are we going to educate managements in the organization about what you can and cannot do with web analytics. How are we are going to educate senior managers about the terms, the definitions that they need to know to take advantage of the reports that they are getting, and more importantly to take advantage of the analysis that they should be getting? If so it's easy, and it's going to be a slam dunk for you. I don't think you are going to give it the attention that it deserves. Again, this goes back to the free versus fee conversation we had moments ago. If web analytics is easy, I don't really need to spend Avinash's $90, right it's easy. So, we will just all be able to pick it up, right?
Eric Enge: Right.
Eric Peterson: It's not easy.
Eric Enge: Yes, I agree completely. I mean it's like I could spend a dollar, and maybe I will get $2 back, or I can spend $10 and I am going to get $50 back. Well, getting yourself to spend the $10 might be harder to do or take little more thought upfront, but I really think the web analytics situation is like that. The ROI grows as the expense grows assuming of course that the expense is being spent in a smart way, but I think the ROI grows as you ramp up the effort.
Eric Peterson: Yeah, I agree. But, it's about getting people to ramp up the effort and having the right expectations.
Eric Enge: Alright, did you mean to use your acronym there by the way?
Eric Peterson: That's what I want, RAMP, right let's RAMP it up. It is an important thing; I got some comments back from the WA Webcast which I will be making freely available through webanalyticsdemystified.com. I think towards the end of next week, thanks to the good graces of the folks of the WAA. One of the comments I got back was I saw this in the Yahoo group that Peterson wasn't talking about anything groundbreaking or revolutionary with RAMP. And, I am not, we all know this.
The problem is we are very insolated little circle of individuals, the web analytics blogers, the people that go to Emetrics, and we need ways to take web analytics out to the masses, to everyone in the organization and people outside the organization. So, RAMP is one way to do that, right it's not to simplify it so far that it makes it useless, but to communicate it more effectively. And, multivariate testing, right you don't get to continual improvement done well until you get good at multivariate or AB testing, I mean this is just the reality of it. So, feel free to talk about RAMP all you want.
Eric Enge: Sure. So, another thing I noticed is that you are really into detailed diagramming of every part of the process, and maybe you can talk about why you feel that's so important?
Eric Peterson: Yeah. I don't know that I am a huge proponent of pedantic diagramming of everything. I don't know that I am that big a proponent of getting up the pens and papers or SmartDraw or Vizio or whatever in diagramming everything. But, I am trying to convey with this point that there has to be more attention paid to the detail, because people say we have web analytics integrated into all of our campaigns and page deployment processes, web analytics is the strategy part of our business. And then, I say okay well, so that means that you never forget to tag a campaign and you never forget to tag a new page and you haven't deployed Web 2.0 Applications, such as Ajax or Flash, or a podcast, or an RSS feed, that every time you deploy something new there is always analytics baked into that, right. I say that to companies and I get this funny look back, and then somebody in the back then goes no, actually we forget to tag campaigns all the time, and we just launched a brand new Ajax application and it doesn't have any tracking in it at all.
Eric Enge: Right.
Eric Peterson: And, I say it's because it's not part of the process. You have not diagrammed web analytics into the process; you have not considered the importance of web analytics. And, so then it comes in at the 11th hour, and then you fall behind, you get busy and it just drops out. I mean how frequently does web analytics drop out of sight or campaign or content deployment process, it is still very common. So, this diagramming is simply an exercise, this is something that I go out and do with Web Analytics Demystified clients.
We sit down and we say let's talk about how you deploy a new campaign, let's draw out all the steps and look at where measurement should be, and let's talk about whether or not it's there or not. Simply the act of creating those maps, of creating those checklists gets people to think more carefully about the value of measurement and how measurement has to be in there. So it's, I mean it's not an end, it's a means to an end. This diagramming is something that, you do a couple of diagrams you get the gist of it, you say yeah, yeah, we have to remember to do web analytics, we have to remember to insert measurements into these processes. So, it's not something you have to do forever, you don't need big binders, process binders for your web analytics integration, it's everything else you do. But you just, you have to think about it that way and I found that to be the best tool for getting my clients and my customers and my friends to think about it.
Eric Enge: Right. What I find is in terms of, if somebody rolls out a new section of a site or like you say an Ajax application or something like that, what happens in a lot of companies, is that somebody goes and tries to pull the data, and that's when they find out the analytics is missing.
Eric Peterson: Data is not there, I mean I just stop counting the number of times in sort of the explosion of Web 2.0, which I think is great. I mean I think Ajax and all of this stuff is fascinating, I love my iPhone. I spend more time then is absolutely necessary on the FaceBook application on iPhone, I am really into Web 2.0 and Web 3.0 technologies, but they need to be measured, right. It's not responsible to create these measurement black holes, and I just stopped counting the number of times I would sit down with companies and say hey, this is a great Ajax application, this is really engaging. How do you measure visitor engagement with this? And, they go well, we are not really measuring that much, and I go well, at least you are measuring like the conversion rate, i.e., the people are using this application to complete the critical conversion process, right? Then they go no, we are going to hope to back that in and like the third version of it or something like that. And then, I am just sitting there staring at them. I am like okay, I am the guy who wrote 3 books on analytics, are you just admitting to me that you spent $200,000 building this cool Ajax application, and it's got no measurement in it. I mean it's so uncomfortable conversation there for a couple of seconds, and then we move on.
We start to talk about how do you measure Web 2.0, how do you bake the tagging or the click tracking or whatever you need into it while it's being developed, and test that that works and think about, what do we want to know? You don't need to know everything, you don't need to know every drag and drop and zoom and click and all of that stuff, but you need to know some of it. How do you know what you need to know in advance? And so, in some ways Web Analytics 2.0, which is the subject of my longer talk at Emetrics in Washington this year, Web Analytics 2.0 is a lot like Web Analytics 1.0 was, years ago. We are going to have to relearn a lot of these things, only more money is being spent and more peoples' necks are on the line now.
Eric Enge: Yeah, ultimately you expect that the tracking mechanisms will follow the way that people make money on the application in some fashion. So, that's the thing that absolutely must be measured, right?
Eric Peterson: Yeah. No, I agree. The funny thing is people ask me, the Wall Street guys specifically will ask me who else is out there, like who is going to just do a great job of measuring Web 2.0 stuff, like Ajax. So I just, don't think it's somebody from left field, I think that the Website Optimization ecosystem of technology is right. This is Web Analytics Technologies, Customer Experience Management Technologies, Voice of Customer Measurement Technologies, the forsee Results and Tealeaf's of the world. I think the stuff we need is already out there, it's just about using it the right way. I think it's just about understanding how the technology should be used, what you hope to measure and how you hope to use that data. I think it's really that simple, it's just it's not playing out that way.
Eric Enge: Indeed. So, can you summarize for us then what are the keys to success in Web Analytics today?
Eric Peterson: Yeah, sure. First key to success is to recognize that web analytics is hard, right. It is hard; it is something you are going have to work out. You are going to need people, you are going to need resources; you are going to need time and money. Web analytics is not easy, and when people tell you that web analytics is easy you should question their motivation, are they trying to sell you something? Are they trying to sell you on something, do they want you to buy a book or read a blog or something like that? Web analytics is hard; the second thing is RAMP; resources, analysis, multivariate testing and process. You've got to have all four of these things and you got to have, you got to understand how all four of those outputs or inputs work together to drive your businesses success. Resources' is technology and people, analysis is the desired output, reports are just reports, right. Reports are only good if you know what they are telling you, but analysis and recommendations is the desired output from Web Analytics projects. Multivariate testing we've talked about a fair amount, process we've talked about a fair amount. You have to consider all four of these things to build a ramp that will ultimately increase the success of your online business.
Eric Enge: Well great, thanks for taking the time to talk to us Eric.
Eric Peterson: Absolutely. Thanks for asking me Eric. I wish you all the best at Stone Temple and I got to say man, I am really, really enjoying the podcast and the interviews that you've been conducting up there, just great stuff this global interview. You've really managed to grab onto an idea and talk to some really great people, and then cover some great information. So, I want to just say I very much appreciate that.
Eric Enge: Well, then thank you and I am pleased to have you in the list of those people I've been able to talk to.
Eric Peterson: Excellent.
About the Author
Eric Enge is the Founder and President of Stone Temple Consulting (STC). STC offers Internet marketing optimization services, including SEO, Social Media and PPC optimization, and its web site can be found at: http://www.stonetemple.com.