While perhaps best known as a major social networking site in the Web 2.0 vein, Facebook is becoming a popular place to interface with potential customers, similar to what companies such as Scribe and iLike have done. Facebook also is being viewed as a place for enterprise collaboration, with Serena Software leveraging the site for this purpose. To get perspective on Facebook, InfoWorld Editor at Large Paul Krill recently spoke with Aditya Agarwal, Facebook director of engineering. Agarwal formerly worked in the Oracle server technologies division.
Agarwal oversees product design and development at Facebook and also is responsible for Facebook's search strategy. Agarwal discussed the company's intentions, search efforts, developer strategy, and infrastructure. He also commented on why the company has been gaining traction in the enterprise.
InfoWorld: It’s interesting that you worked at Oracle right before Facebook. And you went to Facebook three years ago when the company was not as in the news as it is now. Why would you leave an established software giant like Oracle to go to a startup like Facebook?
Agarwal: I think that at that stage it was clear that Facebook was a young startup with a lot of energy. It was something that had tremendous potential. And there is so much information out there. People put a lot of information online. They made it accessible, but over time really the [intent] is going to be to try to help people make the Web more social. And even then, about three years ago, it was apparent that that was the direction, the trend that the Web was moving in.
InfoWorld: You wanted to talk about search efforts. What exactly is happening there with Facebook? When I think of search, I think of Yahoo or Google or some of the others. I don’t necessarily think of Facebook.
Agarwal: The reason you use search in Facebook is that there is a lot of information that people are generating within the context of social networks. It becomes quite imperative that you have a way of being able to search that information. And I think that Facebook is taking the first crack at that. And the first crack that we really tackled was -- how do we let people find other people? And you are right in that the scope of this is different than from other Web search engines, but there is a value-add that you can provide when you think of search -- returning users' relevant results that are not based on traditional mechanisms. [For example], when you search for somebody named John, chances are that you are going to get your friends who are named John, your friends of friends who are named John. So [in the case of someone in Silicon Valley, the search might include people] from San Francisco, San Jose, Palo Alto who are named John. I think that we try to essentially provide value based on the unique position that we have. That’s the way that we approach search.
InfoWorld: So where are you headed with search?
Agarwal: Well, I think we are continuously innovating in the search space. One of the things that we have learned is that search is obviously something that requires a tremendous amount of infrastructure, and we’re in the process of scaling our systems out, building up new services and in the future obviously exposing more functionality to the user.
So to give you an example of something that was released very recently, we now allow users the ability to search through their entire inbox, the people inbox, the people’s messaging system. The important thing to note there is [that] that’s a pretty difficult problem in the sheer size, the scale of the data that we have. But the way that we view search today is that as Facebook becomes more and more [of a public] communications utility and people start using Facebook messages almost exclusively instead of something like e-mail, then we need to provide them with search capability there because it’s an important part of having a complete holistic experience.
InfoWorld: You also wanted to talk about Thrift, which is your software framework for cross-services development. Could you tell me what the intent of that is and what’s happening with that?
Agarwal: Thrift is a lightweight RPC framework that allows really high-performance, intra-datacenter communication. Another aspect of Thrift is [it] allows you to have cross-language communication. So you could be writing some functionality in PHP and by phone, and it seamlessly makes calls out to services that might be running in Java or C++. The key thing I think, or the key motivation behind building out Thrift and not using any of the existing frameworks, for example, something like CORBA or something like a REST services framework, is that Thrift is designed to be extremely high-performance. So when you’re looking at providing really a quick user experience, you’re thinking of response times, latencies in terms of 100 milliseconds, in terms of 150 milliseconds, and at that latency level, you have to have something that is much more [higher-performing] than anything out there that’s in the market right now. So we decided to develop this in-house and then basically release it as open source to the world. A lot of the functionality that allowed us to innovate quickly, we also wanted to expose to everybody else because part of our vision is to provide a platform but also provide the tools that people can use to help scale their services.
InfoWorld: What type of services are people building for Facebook using Thrift?
Agarwal: That's a good question, and a lot of our core services like search, you’ve seen chat, our advertising systems, some of our mobile systems. There are a lot of services that are built on top of Thrift, and they’re built in a variety of different languages. We have services running in C++, we have services running in Java, Python, Erlang.
The best part about Thrift is that you can choose to use the tool that you want, the language that you want, the framework that you want to best solve your problem. You’re not constrained by a standard three-tier architecture, you’re not constrained by only being able to use one application language. We really believe in giving our engineering organization the best tools to solve problems, and Thrift is a key part of that because it allows you to concentrate on creating the product that you want, solving the problem that you’re trying to tackle and not have to worry about -- how do I go in between languages? How do I efficiently transmit data between one project and another?
InfoWorld: Facebook is generally viewed as a platform for social networking in a nonbusiness sense, but I know that’s changing. What exactly does Facebook have to offer enterprises?
Agarwal: Without getting into the actual specifics of that question, I think that the way I think about that is that we strive to provide a platform that can enable many different types of applications. And our focus is really on having a platform that [is] fully feature-complete enough to allow people to build different kinds of things on top of it.
InfoWorld: And what type of enterprise capabilities are people building on top of Facebook?
Agarwal: I don’t have that data off the top of my head, which is about the enterprise capability that applications are providing, but I’d be happy to put you in touch with somebody from our platform product marketing team, who could maybe provide a more accurate answer to that.
InfoWorld: Slide, a personal media network, has used Facebook as a Web vehicle. iLike discovery service has used Facebook for a music discovery service. Is Facebook surprised by businesses that are using Facebook to grow their businesses?
Agarwal: I’m not entirely surprised because I think that Facebook has -- it’s such an important and useful communications utility -- but more than that, it also has an incredible amount of distribution. Like the distribution engine in the form of [Facebook] News Feed and all of the different other platform integration points that we offer. It’s pretty compelling in terms of having a vehicle to kick-start the growth of a particular service or an application, which I guess which is just simply not provided in Web 1.0.
InfoWorld: You’ve sort of mentioned this with Thrift, but what type of developer services does Facebook offer?
Agarwal: If you’re a Facebook platform application developer, then obviously we have a number of client libraries. And the approach that we have taken is that we support some of the client libraries internally, but by and large, we let the developer community contribute client libraries in different languages because we feel that they are in a better position to maintain and optimize those particular client libraries. We have a suite of open source software, also, that we are happy to share with developers, Thrift being the major one there. We are going to be releasing a new open source software package called Scribe, which is a distributed logging framework that we use extensively in-house. We thought that we would give that back to the community as well. And a lot of the other open source projects that we work on that are not -- that were not created by us but we have contributed heavily to, such as memcached and obviously Apache, MySQL, and so on.
InfoWorld: Does Facebook have any plans to participate in Google’s OpenSocial Initiative? I know you’ve been a hold-out on that.
Agarwal: Actually, once again, I think that somebody from the platform product marketing team would probably be the best person to answer that.
InfoWorld: I heard a presentation last week about Enterprise 2.0 and enterprise social networking. During the presentation, a Serena Software representative said the company is using Facebook as its collaboration platform. Is that something that’s pretty widespread? Do you expect it to grow? And once again, is that something that Facebook would have expected three years ago?
Agarwal: It doesn’t surprise me because, once again, a lot of our internal world view and how we approach the products that we build is that we want to be enablers for external applications, but also we are [a] communications utility. We are facilitating interactions between people. And for me, when you talk about enterprise collaboration software, that is one aspect of that. And what we build should allow developers and users alike to benefit in terms of building applications that provide that functionality and for users to actually be able to utilize that particular functionality.
InfoWorld: How do you respond to criticism that the Facebook platform is a closed platform?
Agarwal: I will take an initial stab at answering the question, but I’m going to have to defer to the platform product marketing team. I think that I don’t view it as a closed system. I actually view it as an enabler in the sense that we are enabling the ability for external Web sites and the application that you earlier mentioned to provide a compelling user experience in the form of new functionality. So I’m not quite sure I would view it as a closed platform.
InfoWorld: What can you say about the infrastructure at Facebook? What powers Facebook?
Agarwal: I’d be happy to talk about that. We have two different kinds of code stacks, and I think I described one earlier, which is primarily running on Thrift and some other kind of core component that we have. The other part of our stack is something you mentioned briefly in your question, which is a kind of a LAMP stack, obviously running a Linux kernel. We modify that to be more optimized for our purposes, and then after that we are running the Apache Web server, we’re running PHP, and we’re running memcached, which is a distributed in-memory hash table. And we have 25TB of in-memory cache, which is kind of where more than 95 percent of our data access comes from. And we have MySQL, which is a persistence store on the back end, and we obviously dabble heavily in each of the underlying technologies to make them more efficient, make them more [higher performing] and basically help in scaling
InfoWorld: Are you using mostly open source software to power Facebook?
Agarwal: Most of our systems are running open source.
InfoWorld: Have you found Linux adequate for what you’re doing?
Agarwal: Yes, absolutely. In fact, Linux is more than adequate for the kind of things that we’ve been doing, and as we roll out new features and new products and we see certain parts of Linux that don’t work as well, then our first [reaction will be], OK, how can we fix it and give this back to the Linux community such that everybody might benefit from it?
InfoWorld: How many servers do you use?
Agarwal: We have more than 10,000 servers.
InfoWorld: All Intel servers?
Agarwal: I’m not aware of the hardware specifications, but we have more than 10,000 servers.
InfoWorld: I have nephews that are using MySpace. I don’t know if they’re using Facebook. What would you say the main difference between MySpace and Facebook would be, for somebody like myself who doesn’t get the time to hang out in some of these social network sites?