Princeton SNS Group

For the past few years, we’ve been working on problems related to geo-replicated storage. We want to store data “in the cloud,” but that data should reside within multiple datacenters, not just in a single one.  When data is geographically replicated in such a fashion:

• Users can experience lower latency by accessing a datacenter near to them, rather than one halfway around the world.
• Network or system failures at a single datacenter doesn’t make the service unavailable  (even for data stored at that site).

This is common practice today.  Google runs multiple datacenters around the world, … Continue Reading

Early last fall, I started working on a project called js.js with two other graduate students, Naga Katta and Stephen Beard. We started using a public Github repository from the start, and at the beginning of January, the author of three.js found and tweeted a link to our repository to his many followers. Soon after, a post wound up on Hacker News. Unfortunately, we weren’t very far along in the project yet, and weren’t really ready to show anyone our work, so there was a lot of confusion and negative criticism.

We’ve reached the point … Continue Reading

This year saw some great faculty additions to the CS department:

• Mark Braverman in theoretical computer science,
• Zeev Dvir in TCS and Math,
• Rebecca Fiebrink in computational music and HCI (hired in 2010), and
• David Wentzlaff in the EE department, although his work includes the very CS-like topics of computer architecture and operating systems.

This coming spring, we’re happy to be interviewing again at the Assistant Professor level. Rather than limit the search to any particular areas of focus, we’re looking for top applicants across all areas of computer science.

[EXPAND Announcement]
Tenure-Track Positions.

The Department of Computer Science at Princeton University invites applications … Continue Reading

The CS community recently discussed extending the Q&A session that occurs after each talk at a conference into a more formal written Q&A.  More specifically, this was raised during the business meeting at SOSP and the proposal was to publish the results in SIGOPS OSR.  The idea was this written extension to Q&A could really get to the bottom of the issues raised, and it wouldn’t let speakers avoid questions by saying, “Let’s take that offline.”  There was some push back against this with arguments like “most questions are just misunderstandings” and “that will add a lot of pointless work … Continue Reading

ACM recently released the ACM Authorizer service. It allows authors of ACM papers to link to ACM’s digital library from their website. Users who click the link will receive access to the author’s paper free of charge.

The main concerns I had about changing my website to link to this new service are:

1. If ACM’s website is down, my paper is no longer accessible
2. An independent copy of my paper’s PDF would not longer be linked in Google’s search index

There is an easy workaround to this problem. You can keep your markup linking to a local PDF of your paper and use … Continue Reading

I recently attended OSDI 2010 where I sat through about 30 presentations on systems-related topics. I was surprised that there were so many occurrences of things that I think should be avoided when giving a presentation.

In this post, I’m going to outline several things that, in my opinion, you should avoid when giving a talk at a conference. Keep in mind that this is just my opinion and you’re welcome to disagree, but I think you’re wrong and I’ll explain why.

Note that the examples below are all taken from the OSDI slides that were posted after the conference. If … Continue Reading

Introduction
I’m currently working on a project called Sirikata, an open, programmable, scalable, secure, and extensible virtual world.

One aspect of this virtual environment is application-level messaging. Users can create objects in our world, and these objects can communicate with each other. To get an idea of what application-level messaging looks like, we wanted to take a look at one of the world’s biggest virtual worlds: World of Warcraft (WoW).

In WoW, users can create an “Addon”: a user-interface modification component that can add enhancements to the game. To communicate, these addons (written in Lua) have access to a function called … Continue Reading

Today, I am going to talk about a system called SSDAlloc, a hybrid RAM/SSD memory manager, that I have been working on over the past year and a half or so, jointly with my adviser Vivek Pai. The post is more of a research report briefing. I’ll first talk about what motivated us to explore this area and then delve into the technical jargon.

I will start with describing what motivated us to use SSD as program memory and then I will describe why using SSD as a swap space is a bad idea. After that, I will describe … Continue Reading

I’m back in Princeton after spending the week in Indianapolis, Indiana for the first ACM Symposium on Cloud Computing (SOCC 2010). I’m posting here with a brief summary of each talk from Day 2 of the conference as well as some of my thoughts. These are my reactions to the presentations only, as I haven’t read most of the papers.

[See my previous post for Day 1]

Keynote 2

Building Facebook: Performance at Massive Scale
Jason Sobel (Facebook)

Jason gave a great presentation about the architecture at Facebook and some of the lessons their engineering team has learned in the last several … Continue Reading

I’m currently in Indianapolis, Indiana for the first ACM Symposium on Cloud Computing (SOCC 2010). I’m posting here with a brief summary of each talk at the conference as well as some of my thoughts. These are my reactions to the presentations only, as I haven’t read most of the papers.

[See my next post for Day 2

Keynote 1

[Note: My SIGMOD paper was being presented opposite the first keynote, so Steve Ko who is also at the conference wrote this first summary.]

Evolution and Future Directions of Large-Scale Storage and Computation Systems at Google
Jeffrey Dean (Google)

Jeff … Continue Reading

[Given my continued use of Ed's Freedom-To-Tinker blog, I'm reposting this article from there.]

A few weeks ago, I wrote about a deluge of DMCA notices and pre-settlement letters that CoralCDN experienced in late August. This article actually received a bit of press, including MediaPost, ArsTechnica, TechDirt, and, very recently, Slashdot. I’m glad that my own experience was able to shed some light on the more insidious practices that are still going on under the umbrella of copyright enforcement. More transparency is especially important at this time, given the current debate … Continue Reading

I’m happy to be able to advertise that, after a two-year hiatus from hiring, the Princeton Computer Science Department will be interviewing for tenure-track faculty positions this year.  Because of Perry Cook‘s recently announced retirement—and he will be sorely missed!—there will be a focus on music, sound, and HCI.  Top applicants from all areas of computer science will be considered, however.

Computer Science Department
Princeton University

Tenure-Track Position

The Department of Computer Science at Princeton University invites applications for Assistant Professor. We are accepting applications in all areas of Computer Science, with particular emphasis on music, sound, and HCI.

Applications … Continue Reading

I recently posted an article on Ed Felten’s group blog, Freedom to Tinker, which covers both technical and policy-related I.T. issues. It describes some technical issues and my recent experience with DMCA enforcement agencies, BitTorrent, and CoralCDN. I think our readers would be interested in it as well, as well as join in the lively conversation among commenters.
In the past few weeks, Ed has been writing about targeted and inaccurate copyright enforcement. While it may be difficult to quantify the actual extent of inaccurate claims, we can at least try to understand whether … Continue Reading

Oct 2009 issue of the SIGCOMM CCR has an editorial by Kentaro Toyama and me where we ask the question if technologies for developing regions be considered a core area of computer science research? It is relatively easy to argue that technology can help improve the lives of the poorest billion people on the planet. But, is it research? More specifically, is it computer science research? This editorial stems out of our discussions at the CCC Workshop on Global Development. Keshav asked us to merge our, somewhat opposing, views into an editorial. You can read it here. … Continue Reading

Together with Arvind Krishnamurthy, I’ll be chairing this year’s International Workshop on Peer-to-Peer Systems (IPTPS).  The workshop was started in 2002, which coincided both with the popularization of P2P file sharing (Napster, KaZaA) and the introduction of distributed hash tables (DHTs) from several different research groups.

Eight years later, P2P file sharing is still going strong (now through BitTorrent), while the previously-academic DHTs have found their way into real use.  DHTs now form the decentralized lookup structures for file sharing services—in the form of so-called “trackerless” BitTorrent—with the DHT in the Vuze service comprising more than a million concurrent users.  (As an aside, I’m proud to note that Vuze’s DHT is based on Kademlia, which was proposed by one of my officemates in grad school, Petar Maymounkov.)

These self-organizing systems have also found their way into the datacenter.  One notable example is the storage system, Dynamo, that forms the basis for Amazon’s shopping cart and other back-end  applications.  Or Facebook’s Cassandra, used for its Inbox search.  Or the rest of the key-value stores that do automated partitioning.  And we are starting to see these techniques being proposed for scaling enterprise networks as well.  With that in mind, we wanted to broaden the scope of this year’s IPTPS to include topics relating to self-organizing and self-managing distributed systems, even those running in single administrative domains.

We also plan to have a demo session at this year’s IPTPS to highlight developed and deployed systems.  The workshop will be collocated with NSDI in San Jose, so will be especially convenient for those in the Bay Area.  We welcome submissions (both paper and demos) from researchers, developers, and hackers.  If you don’t want to write a paper, come show off your running P2P system.

Paper submissions are due Friday, December 18, 2009.  More information can be found at http://www.usenix.org/event/iptps10/cfp/.

Continue Reading

Jen Rexford and I are jointly seeking to hire a post-doc to join us at Princeton.   We are looking for somebody to start sometime between February and June 2010 (ideally, as soon as possible) and stay for a duration of 18-24 months.

This research opportunity is part of the SCAFFOLD project.  SCAFFOLD is a new network architecture that focuses on better supporting distributed and wide-area services.  These services, run over multiple hosts distributed across many locations, need to respond quickly to server and network churn: both unexpected changes (due to equipment failures and physical mobility) and intentional changes (during planned … Continue Reading

This year’s HotNets workshop was held over the past two days in the faculty club at NYU; it was nice being on old turf.   The HotNets workshop has authors write 6-page “position” or “work-in-progress” papers on current “hot topics in networking” (surprise!).  Tucked into a cosy downstairs room, the workshop was nicely intimate and it saw lots of interesting questions and discussion.

One topic that was of particular interest to me were new ideas about datacenter networking; HotNets included two papers in each of two different research areas.

The first thematic area was addressing the problem of bisectional … Continue Reading

The last post argued how CoralCDN’s API through domain manipulation provided a simple yet surprisingly powerful content delivery mechanism.  Unfortunately, its technique flies in the face of the web’s use of domain names.

Conflating naming, location, and authorization, browsers use domains for three purposes:

1. Domains provide a human-readable name for what administrative entity a client is interacting with (e.g., the “common name” identified in SSL server certificates).
2. Domains specify where to retrieve content after they are resolved to IP addresses (through DNS).
3. Domains specify what security policies to enforce on web objects and their interactions, especially as it relates to browser Same Origin … Continue Reading
   

While my previous post argued that CoralCDN’s architecture design might not be ideal given its deployment, it has proven successful from the simple perspective of real-world use. Rather than any technical argument, we believe that the central reason for its adoption has been its simple user interface: Any URL can be requested through CoralCDN by appending nyud.net to its hostname.

Interface design

While superficially obvious, this interface design achieves several important deployment goals:

• Transparency: Work with unmodified, unconfigured, and unaware web clients and web servers.
• Deep caching: Support the automatic retrieval of embedded images or links also through CoralCDN when appropriate.
• Server control: Not … Continue Reading
  

Most of my posts about CoralCDN to date have discussed techniques to make the system more robust; now I discuss what it got wrong.  While nice, many of these optimizations were in fact moot: CoralCDN’s design is ill-suited for its current deployment and usage.

Let us frame this argument by first considering some usage statistics from CoralCDN’s deployment.  The available aggregate data from 167 of the ~250 operating CoralCDN nodes during one recent, randomly-chosen day (January 20, 2009) shows that these nodes received a total of 9.74M requests … Continue Reading

[Today we'll be having a guest post by Anirudh Badam, a PhD student in the larger Network Systems Group at Princeton, related to systems research for developing regions.  The work he'll be talking about was recently named one of Technology Review's Top 10 Emerging Technologies for 2009.  -- Mike]

To provide Internet connectivity in the developing world is a daunting task, with problems pertaining to a high cost of bandwidth, ill-provisioned equipment and power, scarcity of on-site expertise, and adverse environmental conditions. Most common way to offset bandwidth cost/consumption is to deploy high-performance web proxy … Continue Reading

So after a few months of (occasional) posts, we finally decided to give the blog a name.

Dirty Slate Design reflects a design philosophy for systems and networking research, where deployability is a central goal.  While it’s often tempting to try to solve problems by wiping the slate clean and starting afresh, expecting a complete redesign and redeployment of systems as important, complex, and far-flung as the Web or the Internet is rather optimistic at best.

So, rather than implying something that is just “quick and dirty,” this philosophy tries to push new functionality or designs in a way that can be … Continue Reading

In the previous post, I discussed how CoralCDN implemented bandwidth restrictions that were fair-shared between “customer” domains. There was another major twist to this problem, however, that I didn’t talk about: the challenge of performing such a technique on a virtualized and shared platform such as PlanetLab.  While my discussion is certainly PlanetLab-centric, its questions are also applicable to other P2P deployments where users run peers within resource containers, or to commercial hosting environments using billing models such as 95th percentile usage.

Interacting with hosting platforms

CoralCDN’s self-regulation works well in trusted environments, and this approach is used similarly in other peer-to-peer … Continue Reading

For commercial CDNs and other computing services, the typical answer to resource limits is simply to acquire more capacity.  As CoralCDN’s deployment on PlanetLab does not have that luxury, we instead apply admission control to manage its bandwidth resources.  This post describes some of these mechanisms, while we’ll take a step back in the next post to describe some of the challenges in doing resource accounting and management on a virtualized and shared platform such as PlanetLab.

Following the Asian tsunami of December … Continue Reading

At its heart, CoralCDN provides a caching serving, not a persistent data store.  Thus, it ultimately requires that a URL’s origin server is initially available, so that it can pull in content to some CoralCDN proxy and make it available across the network.   While traditional web proxies normally interact with sufficiently-provisioned or otherwise well-behaved origin webservers, CoralCDN experiences a different norm.  Given its very design goals, its proxies typically interact with overloaded or poorly-behaving servers; it therefore needs to react to (non-crash) failures as the rule, not the exception.  Thus, one design philosophy that has come to govern CoralCDN … Continue Reading

Some of you might have heard about the Computing Innovation Fellows program, which is a new funding opportunity for recent PhDs interested in pursuing a 1-2 year postdoc.  Realistically, this program was a response to the terrible job market (both in academia and at research labs) that graduates are facing this year.  It’s pretty impressive that the CCC and CRA were able to put together the plan, funding, and organizing so quickly!  For those not aware of the program and interested in a postdoc next year, check it out.  Applications are due June 9, 2009.

I also just … Continue Reading

So let’s start with the first of seven lessons from CoralCDN’s deployment:

• How all published distributed hash table (DHT) algorithms are susceptible to race conditions and routing errors for non-transitive network connectivity, and what can be done to mitigate these problems.

Some challenges with deploying DHTs

CoralCDN’s primary goal was to enable websites to survive spikes in traffic.  We can see examples of such so-called flash crowds through CoralCDN: The figure on the left shows a spike to Coralized slashdot.org URLs that occurred in mid-2005.  Requests grew from nothing to … Continue Reading

Before finally getting to some experiences, I wanted to touch on some of the security mechanisms that CoralCDN proxies incorporate to curtail misuse, especially important given their deployment at PlanetLab-affiliated universities.

Limited functionality

CoralCDN proxies only support GET and HEAD requests.  Many of the attacks for which “open” proxies are infamous are simply not feasible.  For example, clients cannot use CoralCDN to POST passwords for brute-force cracking.  It does not support SSL and thus risk carry more confidential data.  CoralCDN proxies do not support CONNECT requests, and thus they cannot be used to send spam as SMTP relays or forge From: addresses … Continue Reading

In this post, I describe the architecture and mechanisms of CoralCDN at a high-level. This is meant to provide some of the background necessary for some of our experiences and lessons with operating the system.

System Overview

CoralCDN is composed of three main parts: (1) a network of cooperative HTTP proxies that handle users’ requests, (2) a network of DNS nameservers for .nyud.net that map clients to nearby CoralCDN HTTP proxies, and (3) the underlying Coral indexing infrastructure and clustering machinery on which the first two applications are built.  You’ll find that I refer to the entire system as “CoralCDN”, but the … Continue Reading

We’ve recently been working hard on Firecoral – a browser-based, peer-to-peer content distribution network for web caching. I’ll be presenting a short talk on Firecoral at the 8th International Workshop on Peer-to-Peer Systems (IPTPS) on April 21st in Boston, MA.

Peer-to-peer content distribution has been inarguably successful for large file distribution (e.g. BitTorrent), but P2P services have been restricted to stand-alone applications, not transparently incorporated into Web browsing and seamlessly running over HTTP. CoralCDN has served as a web content distribution network for the past five years, but its deployment has been limited to PlanetLab and demand quickly … Continue Reading

Emil Sit, a friend from MIT with whom I published one of my first papers, has been blogging a series of “interviews” from colleagues about their experience building distributed systems. He started this series after describing some implementation issues with building Chord, which was one of the first and remains the canonical distributed hash table (DHT).   CoralCDN uses a DHT for its indexing, and its first implementation actually used MIT Chord as a software layer.  (I later implemented my own DHT layer, although instead based on Kademlia — which was proposed by an officemate … Continue Reading

Having spent the last 9 years in industry, it’s been a refresher diving back into academic waters, not unlike taking a cold shower on a blistering hot day. It shocks you a bit at first, but once acclimated you feel so much better. One of the primary reasons for my return to the ivory internet tower is to expand the horizon of knowledge and understanding, both my own and of the systems/networking field, in large part motivated by my experience in industry. Innovation and expansive thinking are often scant luxuries, indulged in only when the rare respite between development cycles … Continue Reading

Over the next few weeks, I’ll be posting a number of my “experiences” from the design and deployment of CoralCDN.  For those who aren’t familiar with CoralCDN, it’s a semi-open, self-organizing content distribution network (CDN) that I’ve been operating on PlanetLab for the past five years.

Our goal with CoralCDN was to democratize content distribution:  to make desired content available to everybody, regardless of the publisher’s own resources or dedicated hosting services.  It provides an open infrastructure that any publisher is free to use, without any prior registration. Publishing through CoralCDN is as simple as appending a suffix to a URL’s … Continue Reading

The call for papers for the 3rd Workshop on Networked Systems for Developing Regions (NSDR) was announced today. NSDR 2009 will be held with ACM SOSP this year at Big Sky, Montana. Direct all whining about the location to the SOSP organizers please!

I thought I’d share a little history of NSDR on this blog. Research in technologies for developing regions has been going on for a while. For example, the TIER group at Berkeley started in 2003. However, this area (often dubbed as ICTD) lacked a sense of community with no specialized workshops/conferences.

In 2006, I was attending SenSys at … Continue Reading

Architecting distributed systems can be very difficult. Arguably the hardest part of programming a distributed application is getting node coordination correct. I’ll define a node in this context as a service running on a single server which communicates with other nodes and together make up your distributed application.

What I mean by coordination here is some act that multiple nodes must perform together. Some examples of coordination:

• Group membership
• Locking
• Publisher/Subscriber
• Ownership
• Synchronization

One or more of these primitives show up in all distributed systems, so implementing them correctly is extremely important. While developing CRAQ, I originally implemented a very simple group membership service, but … Continue Reading