Big Data and the Evolving Internet

Big Data is exposing glaring weaknesses in today’s Internet. And this, in turn, is driving the next stage of network evolution, much as the introduction of Ethernet and the Internet Protocol did back in the 1970s when IBM’s System Network Architecture was the only game in town.

This according to a story in Computerworld written by Stephen Lawson, based on a presentation by David Lambert, president and CEO of Internet2. Lambert spoke at the Open Networking Summit conference held this week in Santa Clara, CA.

Lambert pointed out that the current Internet began as a tool to help researchers from many different locations share data and insights.  It still fulfills that function, but in this new era of Big Data brought on by rapid advances in computing and storage, current Internet technology can’t support the massive amounts of data that engineers and scientists work with on a daily basis. Specifically, Lambert said that the technology used on the Internet today isn’t flexible enough to support new requirements, such as large file transfers, massive data sets, and content caching and distribution.

Software-defined networking (SDN) is one of the technologies underpinning the next steps in the evolution of the Internet. Lambert commented that SDN in universities today resembles the early Internet decades ago; practitioners in such data intensive fields as genomics need a new, flexible, open networking approach to advance their research.

Internet2 runs a nationwide network linking research institutions, and it’s already using elements of SDN on its production infrastructure,” reports Lawson. “SDN, a closely watched set of technologies at various stages of development, is intended to shift the control of networks from specialized devices such as switches and routers to software that can run on standard computing platforms and be virtualized. It promises a range of benefits that could include lower costs, faster service deployment and more network innovation.”

Internet2 is running a production pilot for SDN and a new high-speed backbone to provide users with the bandwidth needed to handle Big Data.  The pilot includes OpenFlow-enabled routers on a 100 Gigabit Ethernet network.  29 major universities have committed to deploying the 100 GbE network and using Internet2’s OpenFlow-based services.

The thing that excites me most about the development of OpenFlow and SDN … is the opportunity to have a network stack that’s open again, that people can actually get their hands on, and use it and do disruptive things,” Lambert said.

Read the Full Story.