The following is the transcript of an interview of Mark Lucovsky, a Technical Director of Engineering at Mountain View, California-based Google, Inc. Mark has held this position at Google since November 2004.
Before joining Google, Mark was a Distinguished Engineer at Redmond, Washington-based Microsoft Corporation. While at Microsoft, Mark was a founding member and principal architect of the 32-bit Windows team, where he designed and coded major portions of the Windows operating system.
Prior to Microsoft, Mark designed and coded operating system internals for a variety of UNIX and proprietary OS-based minicomputers, minisupercomputers, mainframes, PCs and workstations.
Eric Enge: Could you give us a little bit of your background, and how you got into this at Google?
Mark Lucovsky: I have spent about twenty-five years working in Operating Systems. I worked at DEC, and I worked at a bunch of start-ups doing mini-supercomputers in the early eighties, and worked at Data General on some of their systems. Before coming to Google, I was with Microsoft for about sixteen years as the principle architect on Windows NT. So, I started the Windows NT project up and worked on it through Windows XP. I wrote most of the kernel executive, kernel32, and the Windows API. About two years ago I decided it was time for a switch, and I came to Google.
Eric Enge: A major life change.
Mark Lucovsky: Major life change, yeah. I thought it was time to do something different, and work on web programming. Now, my background has always been platforms and API design, and that sort of thing. So, first thing I did at Google was basically spec out and design GData. For the last couple months I have been working on the AJAX APIs, specifically the Search API.
Eric Enge: One of the things that struck me as I learned about the Ajax API is that I really haven’t heard a lot about the program, and it seems almost like a hidden type initiative. I think a great place to start our conversation here would be for you to just a give an overview of the AJAX Search API and the kind of things you can do with it, give a basic outline for the people who will be reading this interview.
So, you can do a search, get a handful of results, and then you can decide, do I want to draw them on my page, or do I want to use the data in a slightly different way. And, a great example of using it in a different way might be our map search module, where you do a search, but instead of listing the search results in text, you can put the search results on a map. Or another example might be our video search capability, where one way of using the system is to do a video search, and then draw a thumbnail, a title, or snippet, a date, together with a normal styled textual video search result, but anther way is rendering it as a set of thumbnails of a video, and then if you click on a video, it plays the video on the current page.
Eric Enge: Right, and then one of the unique things about all these is that it’s embedded within your site, so when you click on the video for example, instead of going to Google Video to see it, you see it right there.
Mark Lucovsky: Well, that’s turned out to be a very important feature for some of our customers. I have one customer, I can’t share you his details, or his name or anything, but his revenue is up 3500% since adding the AJAX Search API to his site. And what he attributes that to is people are staying on his site a lot longer, they are sucked in by the mapping, by the videos, and they either buy something from him, or they click on some of his AdSense ads. Because they might be looking up this specific, or that specific item, and while doing that, they find data through his text searches, they watch videos, they get intrigued by the video, and they buy something.
Eric Enge: Right, that old-fashioned web site phrase “stickiness” comes to mind.
Mark Lucovsky: Yeah, it’s kind of a stickiness thing, but it’s stickiness in terms of the data that you are looking for. The subject — that is the reason that you are on the page in the first place –can be used to draw in more dynamic content from all over the web.
In the old days we used to call those things portals, where you would build a portal and it would aggregate a bunch of data onto a page. What we have done with the search capability is mimic pieces of that portal where it makes sense. So, if your site is about entertainment, or teaching you this skill or that skill, you can pull in some multimedia from across the web that might be relevant, and use it on your page.
Another example might be you’re a small bed-and-breakfast in Seattle, and somebody stumbles on your page. One of the first things they are going to want to know before booking a room there, is what else can they do in the area. There are a lot of ways that people can do that kind of research, but if they leave your page before booking the room the chances of them finding that small bed-and-breakfast again are next to zero, right.
If you put a map on your page, and allow them to do searches around the proximity of your hotel right on the page, and if you go one step further, and seed that map with great restaurants, theaters, and Starbucks coffee places, they can kind of hit your page, do some quick map research, and decide, “this looks good, I am going to book the hotel”. And so, that’s a sale. We can call it stickiness, we can call it whatever you want, but really it’s about letting the user ask some questions, get the answers, and make up his mind on the spot without leaving your site.
Eric Enge: So, just to be clear about this. In the Google Maps case, you are providing a map interface and allowing that to trigger related searches in Google local?
Mark Lucovsky: Oh yeah. That’s what local search is all about.
Eric Enge: Right. Yes, but your triggering it off the map, and you are embedding the results in the site owner’s page.
Eric Enge: Yes.
Mark Lucovsky: Okay, so what happens is that’s just a regular use of Google Maps API, and then with AJAX Search we have a searcher that can take a center point from a map, or a city or whatever, and do searches for business listings relative to that center point.
So one of things that we have done to help people out here is, since we are pretty good programmers around here, we built little controls that say, look if you are not really a great programmer, and want this capability, we will generate the code for you, and what we generate is a map and a search control that are integrated together. And when you do a search, it shows the answers on the map with driving directions from the center point to the location you are looking for, and an ability to jump out to a detail page – all kinds of information. But you can be basically a non-programmer; and get this up and running in minutes.
Eric Enge: Yes, so I did embed the Matt Cutts SEO videos in a little article I wrote about the AJAX Search API this weekend, and it was pretty neat.
Mark Lucovsky: And I bet that you did not find that very hard.
Eric Enge: Yeah exactly, and I am definitely not a programmer.
Mark Lucovsky: Yeah, if you wanted to put a searchable map on your website, it’s no harder. In fact, it is a little bit easier to put a searchable map on your website than it is to put that video bar on your website.
We have a wizard that does the heavy lifting for you. (Editor: here is the link)
Mark Lucovsky: Well our target really is that full spectrum. And we deal with that, in a couple of different ways: first; the API has a lot of power built into it out of the gate. So if you are a programmer, and you want to do custom positioning, process callouts, add hidden query terms, do restricted searches, or do funny things with tabs and that sort of thing; all of that is there and we have, a basic class reference manual that’s oriented towards the professional programmer.
For example, we say here is a method, here is a class, here are the arguments, the whole nine yards. Professional programmers are served by our documentation and the richness of our samples, and they can get up and running very quickly. The bloggers and some of the guys that can dabble in HTML — what we have done for them, is we’ve built targeted samples. So, when we started talking about things you could do on a blog, we went out and created two sample blogs; one on TypePad and one on Blogger that included videos and center column search results. And then we published the templates for those. So we showed people, hey, this is what you can put on a blog with just a few lines of code.
And if you want to use one of our sample templates to get started, here is the set of templates for TypePad, here is a set of templates for Blogger. And so, we made it very easy for people to do that. For example, I saw a blog this morning that’s in India, the guy is an Indian blogger, and he basically took my template and didn’t even change a thing; it’s got a map in there, a searchable map, where the center point of the map is Santa Barbara, California. Why he has that on his blog is beyond me. I think, he just liked the idea of putting a map on his blog.
Another guy has a restaurant review blog. He took the same solution and the tips that we showed on how to put it into a blog, how to add hyperlinks in your post to drive the map, and he has built himself a restaurant review blog where he writes an article about a restaurant, and when you click on the name of the restaurant, it updates the map on the side of the blog, showing how to get to the restaurant.
You know, even reading sample code and changing it is stress for some people. So, we have a set of wizards, and when you put the video bar in your site, did you use the video bar wizard?
Eric Enge: I just did it straight through the wizard, I was very lazy.
Mark Lucovsky: So, we did those wizards for people that just want to configure it. There are two or three options that we expose through the wizard. And then, we give you this lump of code that you just put in your page.
So, we have a lot of people that are doing clever things like that. For the last group of users we built a series of gadgets that you can put on your personalized home page. And, that at least gets you exposure to things that you can do with search that are a little bit different from the Google.com search page.
Eric Enge: So, in terms of the business model, can you talk about why Google is pursuing this?
Mark Lucovsky: Yeah, I think that it is a stretch to use the words “business model” with everything that Google does. Remember our mission is all about organizing the world’s information and delivering it to anybody that needs it.
So, that’s Google’s core business model. That’s what we are all about. What we have done with AJAX Search is taking that same mission statement, and say not only are we going to deliver that information through web pages, we are going to deliver that information to applications. So, that’s the mild twist on Google’s mission statement – instead of just giving that information to users, we are going to give it to users and their applications.
Eric Enge: Yes, it’s a distributed distribution model.
Mark Lucovsky: Yeah, it’s a distributed distribution model, but it also lets applications start using that data in a variety of ways. I think the video bar is one example of that use case, where we said I am going to give you a thumbnail of a video. I am going to say what the title of the video is. I am going to tell you how long it is. I am going to tell you that if you want to play that video, here is the swf file for the flash player that will play that video.
If you decide to take all of that information, and make it so that when you click on a thumbnail, it plays a video on your page, great. But you don’t have to do that, if you don’t want to. If you want to show that video differently you can show it differently, if you want to write an email client, that mails a video to your friend, you can do that with the API. We just try to jumpstart some of these scenarios to get people thinking about things that they can do with search results other than print them linearly down a page.
Eric Enge: Right. One other thing you can do as I understand it is you can receive the data, and then process it yourself in some fashion.
Mark Lucovsky: Absolutely, that’s what the video bar and the map search control, the map search wizard are doing. They are going off and executing search, and then instead of taking the HTML content of the search result and drawing it on the page, they are doing different things with that data.
Eric Enge: Right, so then presumably your user could take regular web search results and�.
Mark Lucovsky: Do whatever they want.
Eric Enge: Right, reorder them even right?
We are not doing that for any selfish reason; we are doing that to watch out for the user that is visiting your site. And here is the typical example that we are afraid of, imagine the case where a user comes to your site and does a search for iPod; and what he gets as a first search result is something titled The Apple iPod and the URL is www.Apple.com/store. And then, there is a little snippet of text.
Then what the user is expecting is that, if he clicks on the title, he is going to go to the URL associated with that title, right?
Eric Enge: Yes.
Mark Lucovsky: That’s what you expect, that’s what I would expect. If a programmer decided that oh, no, no, no, that’s not how my page works; what my page does is, when I see www.Apple.com/iPod, I substitute in a different URL, I substitute an URL that says www.MP3world.com/Promo_code=Zune. And now, what that site is doing is, when the user clicks on Apple iPod instead of going to where the search results would direct them, this page is vectoring the guy off to some other place, and we call that a modification of the data that we returned, and we don’t think that that’s the right thing to do for a user. What we will allow you to do — we say this clearly in the terms — is you can easily append your own data on to our search results as long as you represent that the additional data comes from you, not from Google, right?
Somebody searches for Apple iPod, they expect Google quality search results. If underneath that search result, you want to put in a little statement that says, here is something else that you might want to consider and hyperlink to another site, then go ahead, this API will let you do that. We give you everything you need to do that successfully, but don’t make it look like that came from Google. Make sure that your users understand that it came from your website. It’s the same principle that we use when we mark sponsored links. We are not interspersing advertisements in with the search results. When we have an advertisement, we clearly mark this is an advertisement.
This isn’t a search result. So, we are asking people that use this API, to use it responsibly, and use it the same way that you would expect the site to use it if you were a visitor.
Eric Enge: Sure. So if users want to modify the results, they can do this by using the Ajax Search API together with the Google Custom Search Engines program.
Mark Lucovsky: Oh, absolutely. We do a lot of things to support finer grain searching. Let’s go to the Salesforce.com implementation. Type Google in that little search box off to the side. Notice how the search results all have a URL that is the Salesforce.com domain.
Eric Enge: Right, this is a custom search engine.
Mark Lucovsky: That is the custom search engine right there and what they have done is they have said I want to have a web search found on that tab, and I am site restricting it to a custom search engine.
If you click on the guide’s tab, then that’s a different facet of the custom search engine.
On the community tab is a regular web restrict, not a custom search engine. You can see how they have done this in a very classy way. To each one of those tabs they are delivering eight search results. So when you do a search for Google, they are getting back sixty-four search results, and doing the tabbing. That’s why there is very little lag as you go from tab to tab. Because as soon as you hit enter on the search box, they have gone out and done the search, and they are just doing some Ajax DHTML stuff to switch what you see as you switched tabs.
Eric Enge: Right, so that’s the tab interface.
Mark Lucovsky: Take a look at VisualDxHealth.com
They have done a lot of clever things here, you can see how there is a set of sponsored links off to the right hand side, I am surprised they did that. That’s pretty cool of them.
John Biundo: So they integrated AdSense on their own basically.
Mark Lucovsky: Oh no, that’s not AdSense, it’s ads coming from the search API. Like I said, we give people the ability to shape the return values anyway they like and all they did is absolute position the ads, so that’s pretty straight forward. If you look at the Cybersmusic implementation that shows a Pearl Jam search result.
You can see on this guy’s page, this one loads a little bit slower, where you can see about a third of the way down the page, there is a powered by Google logo and then a set a tabs.
And he’s got a first tab of blogs that represents all blogs, so it’s not a custom search engine, it just says when you go to that page, he is going to do an implicit search for Pearl Jam across all blogs. In the next tab, Cybersmusic has an implicit search for Pearl Jam restricted to that website.
The next tab is the entire web, the next tab is news articles on Pearl Jam and the last tab is the custom search engine. So that’s how this site works. And then to the left, is also a video bar, and so when you do a search about Pearl Jam, he is going to give you five tabs of search results and a strip of eight videos on Pearl Jam.
So, those are some other things that you can do with the API pretty easily. That guy at Cybersmusic is absolutely not a programmer, but he knows how to put together a website.
Eric Enge: Right. Very cool, so what’s the methodology available for someone to get help if they need it?
Mark Lucovsky: We have two communication mechanisms, and this is common across all groups of developer products. We have an outward facing blog, where we make announcements to the community of the users, and we announce new features, changes, and things that are on deck. And then we also have a very active discussion board which is where users can ask questions, users can help each other, and Google engineers are actively engaged. I am on that board every night along with others from our team, and you wouldn’t believe some of the questions that we help people with. Like, in the early days there was this one guy who couldn’t get his code to work at all, and he couldn’t figure it out. Well, it turned out he had spelled Google wrong. I looked at his code and, I said hey you have to spell Google.com correctly, and once we fixed this everything was fine.
If we get a customer out there that’s actively answering questions you know, I will send them a note, and then send them some T-Shirts and some Google pens and that sort of thing. I found that once you do that, their productivity goes up and that’s when they start answering even more questions. So we do that, for the guys that are really pitching in and helping out. But in general, the simple questions get answered quickly, and feature requests get posted there. The video bar was actually a feature request that we turned around in like a day. Four hours after I launched it, we had five thousand people using it; and the next day, it was five times that; now, it’s basically off the charts.
Eric Enge: So, one of the things that we have been involved in for a while is the Custom Search Engine program, and as we look at that, one of the things that we note is, that when someone uses the custom search engine, they may not even know it, right? They may not be able to easily distinguish it from regular web search engines.
Mark Lucovsky: Yup, that’s right.
Eric Enge: So one of the things that is intriguing about the Ajax Search API is that you can do more things and integrate other elements. It becomes a way to create something which is far more distinct and can create a situation where the two programs really enforce one another.
Mark Lucovsky: I think that VisualDxHealth is a great example of exactly what you are talking about. This is a use of the AJAX Search API that has a five tab system going. So, anytime you type a search term, you are getting forty results right there. And, it is being very upfront about that this is not coming from the entire web, and that these are results grouped by category – there is disease overviews, symptoms, diagnosis, treatment options. I don’t think anybody is confused that this isn’t just a straight search across the web.
So, he’s created a great experience. There are equivalent sites like this for security vulnerabilities, where the guys have got it down to threats, alerts, the whole range of security issues. We have done things like this for computer science curriculum search, where we have broken down the search engine into lectures, courseware, slides, presentations, that sort of thing.
John Biundo: On this VisualDxHealth page, they are showing the sponsored links on the right.
Mark Lucovsky: Yeah.
Eric Enge: Are those, the ads coming back from Google Co-op?
Mark Lucovsky: I believe these are coming from web search with a site restriction to a custom search engine; if that makes any sense. So if you want to think of that, as those coming back from Co-op, great. Those are coming back from Co-op delivered through the AJAX Search API.
Eric Enge: So does that link into the revenue sharing capabilities built in to Co-op then, when done that way?
Mark Lucovsky: No. It’s not like when you take a Custom Search Engine to put the form on your page, that’s a slightly different model.
Ads in the AJAX Search API are basically an experiment still. We are trying to figure out if ads are useful in this model, and how do our customers want to use the ads and benefit from the ads? We don’t know the answers yet.
Yeah, Google is not benefiting from these ads, because advertisers are not being charged for them. The only guys that actually benefit, to be honest with you here, are the advertisers. They are basically getting free impressions.
And that’s a good thing — we are basically giving you (the advertisers) free impressions on these sites.
Eric Enge: Right. So, have you run into any issues with browser compatibility, such as with Safari and Opera?
Mark Lucovsky: Well, we run on IE, Safari, Firefox, and Opera. That’s my basic test environment. I am pretty conservative on how we are using AJAX in this system; we are very conservative on how we use the things that typically burn you.
So, there’s not a good uniform simple-to-use XML stack available for the browser. So, it’s very cumbersome to use, and very hard for novice programmers to use correctly. So I do all of my XML processing with a Java based stack running on Google servers, and what I produce out of that is JSON serialized search results. It’s basically the right protocol for this kind of application. Now I am the biggest fan of XML in the world. GData is an XML protocol I did, and I did very large systems at Microsoft based on XML. So, I understand what it’s good for and where it is easily processed, and I also know where it’s not easily processed. And it’s not easily processed in a browser environment.
If we had used XML, we would have 1/10th as many innovative solutions based on this API. With XML you are catering only to the absolute high end programmers and most of those guys would get it wrong too.
John Biundo: Right. So, one of the questions I had was whether or not you envision target platforms besides the web browsers, or if you envision that at all? You mentioned early on maybe somebody writing an email client to process search results; is that a model that you think is possible, one that you want to encourage?
Mark Lucovsky: When I think of an e-mail client these days, I think of a web based e-mail client. I don’t know why anybody would want to run a native application for something as critical as email, since it’s critical to your communications infrastructure. The best way to think about that is next time you go into an Apple store, no matter where it is in the world, walk into the next Apple store and count the number of computers that are up and running and have internet access in the store, and it’s usually fifty to sixty, something like that.
And then do a quick count, look at every single table, look at what everybody is doing, and you will find that ninety plus percent of the people in that store are using those Macintoshes to access their email. They have this expectation that, with email you can walk up to any computer and get access to your mail, without having to download any kind of client setup, configure anything. You know, it’s just baked into our expectations, especially if you are young. You know, nobody these days would walk into an Apple store and say, I want to check my email, the first thing I need to do is, pull out my USB flash drive and install Outlook, or crack open Eudora and create a new profile, and configure my POP3 server address; you know, those days are gone.
John Biundo: Right. Do you imagine other kinds of platforms besides web browsers?
So if you take that down to you know, a piece of software that costs, you know a $1,000; let’s say Photoshop wants to integrate this in, we will look at it by saying okay, Photoshop wants to use this. But, it only benefits people that spend the $1,000 on the software; is that any different than a pay only website? So that’s one of the factors, that goes into our thinking on, who should this be enabled for, what platforms, what environments. But, this is all work in progress; if we wanted to open this up to desktop applications or server applications, we could do it in a matter of hours. There are a lot of factors that go into it though, what’s the potential for abuse, what’s the login mechanism, what’s the accounting mechanism, how do we know that somebody isn’t using this as a high speed scraper, and how can we detect it and deal with it?
So, we just have to answer those questions, and then we can open it up, if it’s the right thing to do.
Eric Enge: I would like to ask just one final question. And that is, future plans and what you are thinking of doing here?
Mark Lucovsky: You tell me, what would you want to see in this?
Google has a lot of search back ends. What we have been trying to do with the API is address the ones that make the most sense, in the order that makes the most sense.
Eric Enge: Right, so some of the things that struck us just in our initial review. It would be nice to get more than eight results per category; I am sure you have heard that before.
Mark Lucovsky: That’s probably one of the top feature requests.
We have taken some steps to resolving that. We don’t have all the answers yet, we don’t have a UI model that we are happy with, to add pagination yet. So, if it makes sense to do that, if we can design it in a way that we feel that our customers can take advantage of, we will definitely consider that sort of thing.
Eric Enge: Right, and then another idea is that the whole notion of enabling the AdSense revenue sharing, much like the Custom Search Engine program does.
Mark Lucovsky: Yeah, we have some pretty cool prototypes for that.
Eric Enge: Yeah, that I think would be a very nice thing to add to the things that you have.
Mark Lucovsky: You have to understand that opening a new revenue stream and tying it to an API is the easy part. The hard part is, what’s the message, what’s the program, what do the advertisers need to see in the program? There are a whole slew of things that are related to that. Such as I gave you that example of somebody doing a search for Apple iPod and getting vectored off to some random site that isn’t related to what they were really searching for. We need to work through the issues and come up with the right program.
This API is only a few months old, so we need to figure out what the hot things are, how people are really using it. People are not using this the way that we designed it, the way that I thought people would use it. We have this really cool thing called the search result clipping, where you can do a search and copy a search result, and stick it on a webpage. And you see a very, very low adoption of that, and we thought that would be the killer feature.
Eric Enge: And, we think it’s a great program, John and I were both very excited about it as we first even heard about it, and then began to dig into this. We quickly thought “wow, there is a lot of stuff here that you can really do. It’s very, very cool.” It’s almost like an opportunity to create new content essentially out of a search mash up, if you will.
Mark Lucovsky: That’s right.
Eric Enge: And, that’s a very neat concept. In any event, thanks for your time.
Mark Lucovsky: Thank you.