Transcript of Podcast with Dennis Mortensen
Podcast Date: March 9, 2007
The following is a written transcript of the March 9, 2007 podcast between Eric Enge and Dennis Mortensen:
Eric Enge: Okay. So, hello listeners, this is Eric Enge. I am the President of Stone Temple Consulting; you can see our website at www.stonetemple.com. I am here today with Dennis Mortensen; the COO of IndexTools, a leading Analytics Vendor. And, we plan to talk today about web analytics. You can see the IndexTools website at www.indextools.com; so, good morning, Dennis.
Dennis Mortensen: Good morning, or actually, Good Afternoon here in Europe.
Eric Enge: Let's start by talking about the various approaches that analytics packages use. In particular, when we talked before, we talked about the difference between a reports-based approach and a metrics-based approach.
Dennis Mortensen: I think the first conclusion we have to make here is, where in the market that the different analytics packages are. I think that's a line in the sand in the sense that if you want to provide a keyword on Google for a $5 a click, you have to provide certain information with that, and Google, Yahoo, and the other major engines actually do provide that. In Goggle's case it's called Google Analytics, so that's a line in the sand. And, then we have analytics packages above that, and you know I would probably call those enterprise analytics pages, you would find ten perhaps, even fifteen players in that field.
Those players are in, again in my world divided into two sections; what I call report-based analytics tools and metrics-based analytics tools. And, the difference is not so much just in the use of it, but also in the way that they have developed their technology, and their approach to the market. And, the way actually a report-based tool, just as for example, a Google Analytics, or even the middle packages such as WebTrends is that they tend to focus on providing you reports.
This is a report on a specific set of information that you have been collecting over a specific time period. I am not saying that metrics based reporting is the better direction, but it's more an investigative approach to your data, and does not really provide you with that many reports. But, what it does do is give you as the user, the web analyst, the opportunity to investigate your data, so that means that you have a lot of metrics, and you have unlimited possibilities to compare, slice, dice, and segment that data, to answer the business questions that you have.
Eric Enge: Right. So basically, it's along the lines of providing a toolkit to the analysts to do their job.
Dennis Mortensen: It's more about business intelligence than it is about reporting; so it is a toolbox, and if you would look at our tool from the very beginning, the tool features that I like the most are actually blank; that would be the dashboard; that is the blank screen, and that is the custom reports, that is a blank table, and that is it. You are forced to think, and you are forced to think about what you want answered.
Eric Enge: Right, okay. And, the report-based approach leads to phenomena that I have heard you referred to as report surfing.
Dennis Mortensen: Yeah. I don't think I invented the term, but I think it's very common in analytics. And, we all do this; I do it myself, sometimes. But, you take your analytics code, and put it in the footer of your website, and it starts to collect some basic data. That is all fine, and that is what we did five years or six years ago, and that would give you some information about the number of page views and visits, and perhaps even unique visitors, or give us information about, what kind of operating systems, time of date, basic information, and they are all provide it to you in really sleek, good looking reports with grasping colors and everything.
Nevertheless, you end up doing, what most people end up doing; you start in the upper left corner, you look at the first reports that will tell you how many page views you have, you look at the next how many visits you have, and you end up surfing twenty reports. And, they are all really cool and nice; but you don't really drive any decisions. So, you ended up spending one hour looking at good information, nice looking reports, but you didn't drive any single decision; which is the whole purpose of actually deploying an analytics tool. And, if you don't drive those decisions, my recommendation would be, don't even deploy analytics tools; just use what comes right out of the package from your service provider.
Eric Enge: Right. So, basically the goal then with the analytics package is to drive real business decisions, so you look at metrics and data that could potentially drive an action, and calls on you to do something. And, if you are just drifting through reports, you are not really focused on that goal; is that a fair summary?
Dennis Mortensen: That is a fair summary; sometimes I actually have a harder way of saying it, when we talk to our clients. Because, my first recommendation is to say please, don't look at any of our canned reports; they don't mean anything, they are not really worth that much, they are there because all the competitors have them, and we kind of need to have them as templates, or an idea base for you to get started on. But, don't use any of these standard reports, they are s**te. Use two things, the dashboards, and the custom reports. Then, the client moves in to the dashboard and say, hey this is blank.
Indeed it is; and if it's blank, you are forced to ask yourself a question; what do I want to have represented on my dashboard; the number of visitors, and if so, why? Does that provide me anything meaningful, or that can drive action in my business; if not who has to think about another business question? And, again with the custom reports; what do I want to know, when you have a real question, either you can do that report yourself, or you can get help from professionals to actually set up that report for you. But, it forces you to think as a company; it also makes the deployment of the tool of course a bit more difficult. But, I think the output is so much better and the benefit of the tool is so much more.
Eric Enge: Right. So, let's talk a little bit about the role that a web analyst plays in this over all web analytics strategy.
Dennis Mortensen: Web analyst is say; kind of a title that the industry invented; I think if we just go back one step, and ask yourself as a company; what is that I want out of my analytics package? And, I think the two directions that we are going is do I want reporting, or do I want analysis. Because, it is two different roles, and it's definitely different people that you want to employ doing the job, and by reporting I mean; if you run a successful E-commerce site, what is it that you want out of your web analyst, or your web analytics product?
Is that somebody just to confirm that, you have this much revenue, this much cost; the conversion is this, and you want that reported to your middle management, or management in general, so they can take decisions from that. Or, the other direction which I think is the right direction is to actually employ a web analyst and doing analysis; that is actually somebody looking at your data, not reporting the data. Because, reporting the data should be pretty much out of the box, and part of the deployment when you set it up, that the right people get the right information in the right formats.
The web analyst and his task, and role in the organization should be to look at the information and data, in a different perspective. And, as he analyzes it, he may figure out that this specific segment, from this specific part of the country, on this specific pool of products; in these campaigns, actually convert very nicely, and nobody else does. I think we should actually do more in that market, or with those people or with those products. And, that is actually the role of the analyst that is to analyze the data, and come up with suggestions on how to decrease cost, or improve conversions and so forth.
Eric Enge: Right. Now, we have talked a bit about the click fraud before, and you shared that with me some thoughts as to, how much click fraud is really out there. Can you talk about that a little bit?
Dennis Mortensen: Sure. Somehow people end up thinking that I am commissioned by Goggle to go out and advocate that there is not really any such thing as click fraud. However, I strongly believe that it's really over-hyped. If you looked at News Week three months ago, it says that there is in between 20% to 30% click fraud. I think that is absolutely ridiculous, and I just don't believe it. If that was a fact, people wouldn't be running the campaigns that they are running. And me, myself; I wouldn't be paying $8 a click for the keyword web analytics on Goggle.
So, it's pretty much basically wrong. What is it that I am basing these assumptions on? First of all is the fact that we run analytics for some four thousand companies. Based on that, half of those are managed by agencies; agencies doing key web placements in the millions, and millions of pounds. These agencies use our tool to set-up specific click fraud reports. We don't provide a click fraud report out of the box, because I think, first of all we have to define what click fraud is. And, click fraud might mean something different to you and me; and something completely different to somebody else.
I think a lot of the debate is that, there might be a pool of invalid clicks; that's not necessarily click fraud. It could be something beyond, what you would see as fraud. And, I think some retailers define it in one way; another retailer defines it in another way, but we can see that the agencies that we work with, define this on a client to client basis. And, they actually come up with the metrics for how much of their paid placement is fraud. And, just based on that we can see what kind of reports they export as documentation for a refund with the search engines.
So, I am sitting here, looking at the refund exports that they are doing with our system, and I haven't even seen a report above 10%. Most of them are in the 2%, 3%, 4%; and that's all dependent on the specific retailer, on the specific industry. Some industries are more likely to have invalid clicks, and even click fraud, and others are not. So, this is definitely based on a high spend with the agencies that we work with, and the exports they do to the search engines. So, these are numbers that I look on again and again, and it's never in the 20% to 30% range.
Eric Enge: Right. But, you did see some individual situations, where the number does drift up as high as 10% per site.
Dennis Mortensen: We have seen specific industries that are affected more than other industries; and we see industries where they don't even work with the refunds, because it's not even worth it. There might even be some invalid clicks here and there; there might even be some click fraud here and there. But, it's on a scale where it doesn't really matter; so some industries, it just doesn't exist; there is no such thing as refunds. Other industries, we can see there is a known task for the agency to export information out of our system, and provide that as documentation to the engines for refunds. And, when I say up to 10% that is very high; it's typically as I said before 2%, 3%, 4% and that's it.
Eric Enge: Right. So, can you come out at some of the industries in which you do see the higher levels of click fraud?
Dennis Mortensen: Yes, this is again; that's not research; now we are getting into my attitude and what I am looking at here. Overall I stand by my 2%, 3%, 4%; when looking into the industries; my feeling is when I go in and look at these reports, the bigger problems are more in the finance industry, where the value per click is extremely high, that we have seen that people have paid more attention to invalid clicks, and defining what they see as invalid clicks, and what they see as click fraud, and thus catching more, and the percentage is a bit higher than in other industries.
Eric Enge: Right. So, keeping in mind that click fraud for the most part seems to be driven by publishers doing the things that cause clicks on AdSense, or similar program type ads, on their sites in order to make more revenue, it would make more sense for them to pick the terms and ads that, give you a higher dollar value per click, right?
Dennis Mortensen: Exactly. And, I agree with that; and that is the whole reason that you would see potentially more click fraud and invalid clicks in these types of industries. You will have all the way up to $50 per click in the finance industry. And, it's easier to detect click fraud on fewer clicks then it is on, a high volume of a low value clicks.
Eric Enge: Right. Now, you talked a little bit about invalid clicks. Define that for me? How does it differ from click fraud?
Dennis Mortensen: I think click fraud is, what we just talked about before. That is me setting up a website, placing AdSense on the website, showing ads, and then clicking on those ads for my own benefit to increase my publisher revenue; that is definitely click fraud. No doubt about that; then I think there's a lot of cases, where we could be talking about invalid clicks instead. That could be me doing a search on Goggle; seeing a specific sponsored search result, clicking that, going back, clicking on it one more time. Because, I use Goggle as a navigation tool.
Is that click fraud? It's probably just invalid clicks that Goggle as part of their system will sort out; or you might see them in your analytics tool, because we recognize and collect data in real time. But, Goggle only consolidates their data on a daily basis. So, you might see these clicks in your analytics tool and think ah, now we are talking about click fraud. But, we are actually just talking about invalid clicks; me using Goggle as a navigation tool that I would rather just type in the keywords, and so click the paid ads, because it's a quicker way for me to navigate.
I may do that five times within one minute, because I am investigating something right now. I see that as invalid clicks, not as click fraud; and a whole bunch of that is actually sorted out by the engines themselves. But, you instantly see them in your reporting tool of course, because we don't discount anything; we report on every single visit that you have.
Eric Enge: Right, great. Well, lastly let's talk a bit about the challenges facing the web analytics industry in 2007 and beyond.
Dennis Mortensen: I think the industry in itself which is, you know quite young. I started back in '96, and that gets of a ten years; but it's actually only in last four years, five years, six years, that the tools have become sophisticated enough to really drive efficient analysis. And, I don't necessarily think that 2007 is going to be a tipping point of the industry's changing. I think, there is a lot of low hanging fruit out there, for a lot of people to use analytics, to optimize their web properties. However, I think 2007 is going to be the year where, there would be a lot of debate about the all mighty page view metric. Everybody talks about the page view metric, because that is how we measure the value of publishers today.
That is the way most analytics companies measure the price for their tool. But, page views are going to change, because we will have a lot of Web 2.0 applications out there, that will start with one page, and you will spend twenty minutes on their website; and you haven't even changed that single page. That would be one page view, but it might also be a hundred and fifty events; and a good example of that is for example, Goggle Maps. If you look at Goggle Maps, you have one URL but a lot of interaction with the map itself. So, that's definitely going to be a challenge, how do we want to measure that?
Most of the vendors today can actually collect the data from AJAX applications, or Rich Internet applications as they call them. We can collect all the data, but it's not so much about collecting the data. It's more about the methodology; what is it that I am supposed to collect, and what is it that I am supposed to report on. And, that is going to be something that we have to decide on. And, I think we will see a lot of testing, and people figuring out what to do in 2007.
Eric Enge: Right. I guess, to some degree that will probably follow the money, right? If you have an AJAX application or something like Google Maps, and there is some sort of advertising tied to it, then, that's probably what you are going to want your measurement to line up with.
Dennis Mortensen: I agree, yeah. That definitely makes sense, because in the good old days, if I went on to Amazon; I might buy a new book, and that will be fantastic. I might have a seven step funnel that starts out by me choosing the book, adding it to the basket, and so forth, and ending up at the check out. That is pretty simple, and you can put that into any analytics application out there. However, if I turn that into a Rich Internet application; who is to say whether that's seven steps; of course it will follow the money all the way to check out.
But, I could actually choose to say that's only two steps; that's me dragging and dropping it into the basket, and that is me checking out. Or I could say, that's actually forty steps; that is me choosing to have it delivered at another address, that is me choosing to buy two copies, that is me choosing to have it wrapped, that is me changing my zip code three times, that is me doing a lot of events on my way there.
So, somebody has to choose what data to college, because I can track every single action that you take, every single drop down, every single click, every single input, and I can collect that data. All of a sudden, the seven actions may look like it's two or forty-five. And, somebody has to choose, whether it is two or forty-five, and I am quite sure some will under analyze and some people will over analyze.
Eric Enge: Right. And, I guess this is similar problem with mobile devices, right.
Dennis Mortensen: Indeed, yeah. What we've seen with the mobile devices so far is that not that many people are focusing on the analytics data yet, and taking action on it. There is simply too little volume to really justify having people going into detail with this. Six or seven years ago, in traditional web analytics, you had an expert company provide you a solution, and you trusted them to provide you the best possible solution, and that was it. Nowadays, you would correct, track, and report and everything you do yourself and not really trust your consultancy.
You would actually figure out, what's working best for you. But, now the solutions I've seen on the mobile market, is back to having the experts provide the data you need and people trusting that, and planning to look at it from a reporting point of view later on.
But, I think there's another challenge in 2007 as well, which is personalization, because there used to be one unique homepage and that was it. For example, if I go to stonetemple.com there is one homepage; even though it might change a bit. There might be news added to it, there might be articles and so forth. But, it's pretty much the homepage; if I want to go to Indextools.com and sign up for a trial, there is just one specific funnel; that's a fixed funnel, its three steps or four steps and that's it. But, with personalization, there is not really any single homepage anymore, because based on me, who visited stonetemple.com probably fifteen times, I would have looked at specific content, and had a specific behavior, and at some point of time you would start serving me content to my behavior. And, that's when the homepage is not really just the homepage anymore, it's actually Dennis's homepage.
Eric Enge: Right.
Dennis Mortensen: And, having that in mind, how do I optimize and report on the homepage, when I all of the sudden have a hundred thousand homepages. And, that's going to be quite exciting I think.
Eric Enge: Right. Sure makes the notion of A/B testing or multivariate testing, a lot harder to think about doesn't it.
Dennis Mortensen: Yes it does, because there are so many benefits in just running a simple A/B test. Even moving beyond that, and running a multivariate test, there are a lot of benefits. But, all of a sudden, if you are running a multivariate test on multiple personalized versions of this specific page, it becomes a bit harder to think about, what is it that I am supposed to do here? And, do I even have enough information about Dennis, who visits Stone Temple to figure out what he wants? So, I am not saying it can't be solved, but it's definitely making things a bit more advanced and exciting.
Eric Enge: Entertaining.
Dennis Mortensen: Yes, yeah. I think that's the right word entertaining, yes. But, that's the way the industry is, whenever we think we are there, a new challenge comes up.
Eric Enge: That's right. A new challenge is that, it's going to cause you to tear all your hair out.
Dennis Mortensen: Exactly.
Eric Enge: Oh great, thanks for taking the time to speak with us today, Dennis.
Dennis Mortensen: No worries, thanks a lot for having me on here.
About the Author
Eric Enge is the Founder and President of Stone Temple Consulting (STC). STC offers Internet marketing optimization services, including SEO, Social Media and PPC optimization, and its web site can be found at: http://www.stonetemple.com.