Overlay Networks
From Blindside
Contents |
[edit] Overlay Networks
[edit] What is it
Short definition
An Overlay is a set of servers deployed across the Internet that a) provide some sort of infrastructure to one (or ideally several) applications, b) in some way take responsibility for the forwarding and handling of application data in ways that are different from or in competition with what is part of the basic Internet, c) are operated in an organized and coherent way by third parties (which may include collections of end-users) to provide a wellunderstood service that is infrastructure-like, but, d) are not thought of as part of the basic Internet. (The Growth of Internet Overlay Networks: Implications for Architecture, Industry Structure and Policy, http://atlas.csail.mit.edu/papers/TPRC_Overlays_9_8_05.pdf)
Longer Definition
Computer networks are often conceptualised as a stack of layers, an idea extensively developed in Europe during the late 1980s and early 1990s under the name of OSI (for Open System Interconnection). The OSI model had seven layers: physical (the actual electrical connections and bit stream); data link (encodes and decodes data packets into bits and handles physical layer errors and supplies the transmission protocol); network (switching, routing, and addressing); transport (transfers data between hosts); session (creates and tears down connections between applications); presentation (translates between application and network formats for data); application (what the end user sees and uses). A well-known spoof T-shirt adds two more layers: money and politics. In discussing today's networks it's common to use a simplified version of four layers that map loosely onto the original OSI 7-layer model.
An overlay network, therefore, is a virtual network at the application layer that directly interfaces to users and runs on top of the physical network itself. Overlay networks may use their own protocols and communication environment.
Examples include content delivery and caching such as Akamai, multicast overlays, peer-to-peer networks such as Gnutella, file-downloading overlays such as BitTorrent and eDonkey, routing overlays such as Skype's VOIP service, anonymity and anti-censorship overlays such as Onion routing or Freenet, and grid computational overlays such as SETI@Home or Stanford's Protein Folding project. In fact, given that in the early days of the Internet many users access the network via dial-up – voice telephone calls converted to and from data – the early stage of the Internet could be thought of as an overlay on the telephone network.
The usefulness of such networks ought to be obvious: far greater computer resources can be assembled from the millions of clients or peers available on the Net than from the comparatively limited number of centralised servers. Even servers belonging to governments or large organisations will buckle in the first hours after the official release of popular downloads such as the US's Starr Report or the latest service pack for Microsoft Windows – or, possibly eventually, during the World Cup final streamed over the Internet from the BBC's iPlayer. Overlay networks can remove the strain from such servers by allowing peers to connect to each other flexibly and share the already downloaded portions among themselves, adapting routing paths to avoid network congestion. These abilities give overlay networks the appealing characteristics of scalability and robustness.
Overlay networks have a number of advantages. They can be built out as needed and require no change to the underlying network infrastructure. They can be optimised for the particular application they are intended for. Increased control and flexibility mean the overlay network may even be more robust than the underlying infrastructure. However, they also have a number of disadvantages. They are less efficient than the underlying network, they are more complicated to manage, and they may not be accessible to every node on the underlying network (for example, accessing P2P applications often poses problems for users behind firewalls).
[edit] Impact & Maturity assessment
Impact: 3 and Maturity: 3 (varies according to the individual overlay network
[edit] Information Assurance issues
Since overlay networks are open to all kinds of Internet users, security and privacy issues may be quite serious. P2P users, for example, may find their hard drives and personal data exposed to others and also to computer malware. In addition, these networks are highly decentralised and therefore may be weak at coordinating resources. Finally, the issues of fairness with respect to resource sharing and collaboration among end-nodes have not yet been well addressed. Risks associated with overlay networks: The issue of copyright infringement with respect to P2P file-sharing networks is well known; however, these user activities are not an intrinsic quality of this technology.
There are concerns that certain overlay networks can degrade overall performance of the Internet at the margins, and that the growing number and diversity of overlay networks constitutes a certain level of threat to the Internet as a whole.
[edit] Implications for UK Government
[edit] Timescale
Overlay networks have recently attracted considerable attention and are expected to be a promising technology. The impact of these networks has already been felt in areas such as content distribution and VOIP, and will continue to grow over the next two to five years.
[edit] Examples
VYYO helps Bentley implement ultra band spectrum
Microsoft Maps shift to in-the-cloud software
[edit] Comments (attributed)
“By adding an overlay network of computational services to the internet - made up of computing and storage resources -- the industry could bring greater intelligence into and across the network core.” – Pat Gelsinger, Senior VP and CTO, Intel.
[edit] Organisations
[edit] Documents & research papers
The growth of internet overlay networks: Implications for architecture, industry structure and policy (PDF), by Dave Clark, Bill Lehr, Steve Bauer, Peyman Faratin, Rahul Sami (MIT), and John Wrocklawski (USC Information Sciences Institute), 2005.
Overlay Networks and the Future of The Internet (PDF) by Dave Clark, Bill Lehr, Steve Bauer, Peyman Faratin, Rahul Sami (MIT), and John Wrocklawski (USC Information Sciences Institute), 2006.
SkipNet: A Scalable Overlay Network with Practical Locality Properties (PDF), by Nicholas J. A. Harvey, John Dunagan, Michael B. Jones, Stefan Saroiu, Marvin Theimer, and Alec Wolman, Microsoft Research, 2002.
P2P Overlay Network to Support E-Commerce (PDF), by Gurleen K. Arora, Martin Hanneghan, Madjid Merabti, Liverpool John Moores University, 2006.
On Failure Detection Algorithms in Overlay Networks (PDF), by Shelley Q. Zhuang, Dennis Geels, Ion Stoica, and Randy H. Katz, UC Berkeley, 2005.
A Virtual Internet Architecture (PDF), by Joseph D. Touch, Yu-Shun Wang, Lars Eggert, Gregory G. Finn, USC Information Sciences Institute, 2003.
Overcast: Reliable Multicasting with an Overlay Network (PDF), by John Janotti, David K. Gifford, Kirk L. Johnson, M. Frans Kasshoek, and James W. O'Toole, Jr (Cisco Systems),
The End of Internet Architecture (PDF), by Timothy Roscoe (National ICT, Sydney, Intel Research, ETH Zurich), 2006.
Resilient Overlay Networks (PDF), by Doug G. Andersen, Hari Balakrishnan, M. Frans Kaashoek, and Robert Morris, MIT Laboratory for Computer Science, May 2001.
Overlay Networks: A Scalable Alternative for P2P (PDF, by Digao Doval and Donal O'Mahony, Trinity College Dublin, July/August 2003.
A Framework for Programmable Overlay Multimedia Networks, by N. R. Manohar, A. Mehra, M. H. Willebeek-LeMair, and M. Naghshineh, IBM Research, 1999.
[http://infolab.stanford.edu/~crespo/publications/op2p.pdf Semantic overlay networks for P2P systems (PDF), by Arturo Crespo and Hector Garcia-Molina (Google, Stanford University).
[http://www.eecs.harvard.edu/~mema/courses/cs264/papers/eclipse-infocom06.pdf Eclipse Attacks on Overlay Networks: Threats And Defenses (PDF), by Atrul Singh, Tsuen-Wan "Johnny" Ngan, Peter Druschel, and Dan S. Wallach (Rice Unieversity, Max Planck Institute for Software Systems), 2006.
[edit] Experts (academic, practitioner)
Hilmi Ozguc is co-founder and CEO of Maven Networks
Milan Vojnovic is a researcher with systems and networks group at Microsoft Research, UK
Ayalvadi Ganesh works as a researcher in the Networking group with Microsoft Research
