Networking doesn’t need VMWARE but it does need better abstractions

Posted on January 19, 2012 by gaberger

Lately there has been a lot of talk around the network and the corresponding conflation of terms and hyperbole around “Network Virtualization including Nypervisor, Software Defined Networking, Network Abstraction Layer, SDN, OpenFlow, etc.

Recently a blog entry entitled “Networking Needs a VMWare (Part 1: Address Virtualization)” appeared on Martin Cassado’s blog which tries to make a case for comparing the memory virtualization capability in today’s modern hypervisors to network virtualization.

This sort of left an uneasy feeling in fully describing why we are seeing this activity in the network domain specifically to deal with the broken address architecture. This post is to try and bring some clarity to this and to maybe dig deeper into the root causes or problems in networking which have led us to this point.

The synopsis in the blog goes like:

One of the key strengths of a hypervisor lies in its insertion of a completely new address space below the guest OS’s view of what it believes to be the physical address space. And while there are several possible ways to interpose on network address space to achieve some form of virtualization, encapsulation provides the closest analog to the hierarchical memory virtualization used in compute. It does so by taking advantage of the hierarchy inherent in the physical topology, and allowing both the virtual and physical address spaces to support complete forwarding and addressing models. However, like memory virtualization’s page table, encapsulation requires maintenance of the address mappings (rules to tunnel mappings). The interface for doing so should be open, and a good candidate for that interface is OpenFlow.

The author of the blog post is trying to describe a well-known aphorism by David Wheeler, which states: “All problems in computer science can be solved by another level of indirection”. This statement is at the heart of “virtualization” as well as other references in communications layering, computer architecture and programming models.

Sidebar OSI Model

Lots of networking professionals like to refer to the 7-layer OSI model when talking about network abstractions. The problem is the OSI model was never adopted; in addition most OSI engineers agree that the top 3-layers of the OSI (Application, Presentation and Session) belongs in “one” application layer. We utilize a derivative of that model which is essentially the four-layers representative in the TCP/IP model.

Lets first try and define what an address is and then what is meant by encapsulation being careful not to conflate these two important yet independent terms.

Addressing and Naming

The first thing to recognize is that the Internet is comprised of two name spaces, what we call the Domain Name System and the Internet Address Space. These turn out to be just synonyms for each other in the context of addressing with different scope. Generally we can describe an address space as consisting of a name space with a set of identifiers within a given scope.

An address-space in a modern computer system is location-dependent but hardware-independent thanks to the virtual memory manager and “memory virtualization”. The objective of course is to present a logical address space which is larger than the physical memory space in order to give the illusion to each process that it owns the entire physical address space. This is a very important indirection mechanism, if we didn’t have this, applications would have to share a much smaller set of available memory. Does anyone remember DOS?

“Another problem with TCP/IP is that the real name of an application is not the text form that humans type; it’s an IP address and its well-known port number. As if an application name were a macro for a jump point through a well-known low memory address.”

Binding a service, which needs to be re-locatable to a location-dependent address, is why we have such problems with mobility today (in fact we may even conclude that we are missing a layer).  Given the size and failure rates of today’s modern data-centers this problem also impacts the reliability of the services and applications consumers are so dependent on in todays web-scale companies.

So while this is a very important part of OS design, its completely different from how the Internet works because the address system we use today has no such indirection without breaking the architecture (i.e. NATS, Load Balancers, etc).

If this is true, is the IP address system currently used on the Internet “location-dependent”?  Well actually IP addresses were distributed as a “location-independent” name, not an address.  There are current attempts to correct this such as LISP, HIP as well as “BeyondIP” solutions such as RINA.

 So it turns out the root of the problem in relation to addressing is that we don’t have the right level of indirection because according to Saltzer and Day, we need a “location-independent” name to identify the application or service but all we have is a location-dependent address which is just a symbolic name!.

What is encapsulation?

Object Oriented Programming refers to encapsulation as a pattern by which [“the object's data is contained and hidden in the object and access to it restricted to members of that class”]. In networking we use encapsulation to define the different layers of the protocol stack, which, as we know “hides” the data from members not in the Layer, in this way the protocol model forms the “hour-glass” shape minimizing the interface and encapsulating the implementation.

Sidebar Leaky Abstractions

Of course this isn’t completely true as the current protocol model of TCP/IP is subject to a “leaky-abstraction”. For instance there is no reason for the TCP logic to dive into the IP frame to read the TOS data structure, doing so would be a “Layer Violation” but we know that TCP reaches into IP to compute the pseudo header checksum. This rule can be dismissed if we think of TCP/IP as actually one layer as it was before 1978. But the reality of the broken address architecture leads to the “middle boxes” which must violate the layers in order to rewrite the appropriate structures to stitch back together the connection.

So how does encapsulation help?

In networking we use encapsulations all the time..

 We essentially encapsulate the data structures which need to be isolated (the invariants) with some other tag, header, etc. in order to hide the implementation. So in 802.1Q we use the C-TAG to denote a broadcast domain or VLAN, in VXLAN we encapsulate the host within a completely new IP shell in order to “bridge” it across without leaking the protocol primitives necessary for the host stack to process within a hypervisors stack.

From the blog.. “encapsulation provides the closest analog to the hierarchical memory virtualization in compute”

So in the context of a “hierarchy” yes we encapsulate to hide but not for the same reasons we have memory hierarchies (i.e. SRAM(cache) and DRAM). This generalization is where the blog post goes south.

So really what is the root of the problem and how is SDN an approach to solve it?

From an earlier statement we need a “location-independent” name to identify the application or service but all we have is a location-dependent address which is just a symbolic name!. If we go back to Saltzer we see that’s only part of the problem as we need a few more address/names and the binding services to accomplish that.

 One interesting example to this is the implementation of Serval from Mike Freedman at Princeton University. Serval actually breaks the binding between the application/service name and the inter-networking address..(Although there are deeper problems then this since we seem to be missing a network layer somewhere). Serval accomplishes this through the manipulation of forwarding tables via OpenFlow although it can be adapted to use any programmable interface if one exists. Another example is the NDN Project led by Van Jacobson

In summary

Yes it is unfair to conflate “Network Virtualization” with “OS Virtualization” as they deal with a different level of abstraction, state and purpose. Just as hypervisors were invented to “simulate” a hardware platform there is the need to “simulate” or abstract the network in order to build higher-level services and simplify the interface (not necessarily the implementation). In fact a case can be made that “OS Virtualization” may eventually diminish in importance as we find better mechanisms for dealing with isolation and protection of the host stack while network virtualization will extend beyond the existing solutions and even existing protocols allowing us to take on a new set of challenges. This is what makes SDN so important; not the implementation but the interface. Once we have this interface, which is protocol independent, we can start to look at fixing the really hard problems in networking in a large scale way..

Posted in Future Internet, OpenFlow, Software Defined Network | Leave a comment

NodeFlow: An OpenFlow Controller Node Style

Posted on January 8, 2012 by gaberger

In less you’ve been under a rock lately, you might have heard something about Software Defined Networks, OpenFlow, Network Virtualization and Control Plane/Data Plane separation.

Some of the reasons for the interest might be:

  • Evolution of the system architecture as a whole (Network, NIC, PCIE, QPI, CPU, Memory) along with X86_64 instructions, OS, drivers, software and applications have allowed for many services to run on a single host including network services. Extending the network domain into the host allows for customizable tagging, classification, load balancing and routing, with the utopia being ubiquitous control of logical and physical by a combination if in-protocol state, forwarding tables and a distributed control system.
  • Non-experimental network pathologies, which are causing havoc with large-scale systems. Turns out there are some very “real” problems, which were never part of Ethernet and TCP/IP design space and software allows us to experiment with different ideas on how to solve these problems.
  • Leveraging a possibly untapped design space in order to be differential,  leap frog competition or disrupt the marketplace

So what is OpenFlow? Well according to the Open Networking Foundation:

“OpenFlow is an open standard that enables researchers to run experimental protocols in the campus networks we use every day”

This paradigm shift into the guts of the network might be better explained by a surgical assessment of the network core, its protocol structure, the devices, which deal with enrollment, classification, multiplexing/demultiplexing, flow control and routing but this will be a post for another day.

In the meantime the “network” has evolved into a first class citizen amongst infrastructure architects, software developers and consumers alike. No, I am not talking about the Social Network by big boy Zuck, but the fact that networks are finding them selves ingrained in almost anything not nailed down. This so called “Internet of Things” tells us that soon the network will be stitched into our lives through the air and into our clothes.

There are many arguments about the value of OpenFlow and SDN, but to find the benefits and use-cases the network domain experts may find the current toolsets and platforms a bit impenetrable. The current controller implementations are written in a combination of C, Python and Java and because of the “asynchronous” nature of the OF protocol, additional libraries have to be leveraged including Twisted and NIO which make it more difficult to understand exactly what is going on.

To that end I introduce NodeFlow, an OpenFlow controller written in pure JavaScript for Node.JS.  Node.JS provides an asynchronous library over JavaScript for server side programming which is perfect for writing network based applications (ones that don’t require an excessive amount of CPU).

NodeFlow is actually a very simple program and relies heavily on a protocol interpreter called OFLIB-NODE written by Zoltan LaJos Kis. I have a forked version of this library (see below) which have been tested with OpenFlow version 1.0.

Sidebar: A note on OpenFlow

Even though the Open Networking Forum has ratified the 1.2 protocol specification, we have yet to see a reference design which allows developers to experiment. In order to get a grasp of the programming model and data structures to this end I have concentrated on the most common implementation of OpenFlow 1.0. in OpenVSwitch.

Sidebar: Why Node.JS

Node.JS has become one of the most watched repos in GitHub and is headed up by the brilliant guys at Joyent. Anyone interested should check out Bryan Cantrill’s presentation  Building a Real-Time Cloud Analytics Service with Node.js

Setting up the development environment

Leveraging OpenVSwitch and tools such as MiniNet, anyone can create a simulated network environment within their own local machine. Instructions on how to setup the development environment can be seen here Download and Get Started with Mininet

Code review

We first setup the network server with a simple call to net.createServer, which we provide the port and address to listen on. The address and port are configured through a separate start script.

NodeFlowServer.prototype.start = function(address, port) {
var self = this

var socket = []
var server = net.createServer()

server.listen(port, address, function(err, result) {
util.log("NodeFlow Controller listening on " + address + ':' + port)
self.emit('started', { "Config": server.address() })
})

The next step provides the event listeners for socket maintenance, creates a unique sessionID from which we can keep track of each of the different switch connections and our main event process loop which is called every time we receive data on our socket channel. We use a stream library to buffer the data and return us the OpenFlow decoded message in the msgs object. We make a simple check on the message structure and then pass it on for further processing.


server.on('connection', function(socket) {
    socket.setNoDelay(noDelay = true)
    var sessionID = socket.remoteAddress + ":" + socket.remotePort
    sessions[sessionID] = new sessionKeeper(socket)
    util.log("Connection from : " + sessionID)

socket.on('data', function(data) {
    var msgs = switchStream.process(data);
    msgs.forEach(function(msg) {
    if (msg.hasOwnProperty('message')) {
         self._processMessage(msg, sessionID)
    } else {
         util.log('Error: Message is unparseable')
         console.dir(data)
   }
})

In the last section we leverage Node.JS EventEmitters to trigger our logic using anonymous callbacks. These event handlers wait for the specific event to happen and then trigger processing. We handle three specific events just for this initial release: ‘OFPT_PACKET_IN which is the main event to listen on for PACKET_IN events, and ‘SENDPACKET’ which simply encodes and sends our OF message on the wire.


self.on('OFPT_PACKET_IN', function(obj) {
 var packet = decode.decodeethernet(obj.message.body.data, 0)
 nfutils.do_l2_learning(obj, packet)
 self._forward_l2_packet(obj, packet)

})
 self.on('SENDPACKET', function(obj) {
 nfutils.sendPacket(obj.type, obj.packet.outmessage, obj.packet.sessionID)
 })

The “Hello World” of OpenFlow controllers simply provide a learning bridge function. Here below is the implementation, which is fundamentally a Python port of NOX Pyswitch.


do_l2_learning: function(obj, packet) {
 self = this

var dl_src = packet.shost
 var dl_dst = packet.dhost
 var in_port = obj.message.body.in_port
 var dpid = obj.dpid

if (dl_src == 'ff:ff:ff:ff:ff:ff') {
 return
 }

if (!l2table.hasOwnProperty(dpid)) {
 l2table[dpid] = new Object() //create object
 }
if (l2table[dpid].hasOwnProperty(dl_src)) {
 var dst = l2table[dpid][dl_src]
     if (dst != in_port) {
       util.log("MAC has moved from " + dst + " to " + in_port)
     } else {
          return
     }
} else {
     util.log("learned mac " + dl_src + " port : " + in_port)
     l2table[dpid][dl_src] = in_port
}
 if (debug) {
     console.dir(l2table)
 }

}

Alright, so seriously why the big deal.. There are other implementations which do the same thing, so why is NodeFlow so interesting. Well if we look at setting up a Flow Modification, which is what gets instantiated in the switch-forwarding table, you see we can see every element in JSON notation thanks to the OFLIB-NODE Library. This is very important as deciphering the TLV based protocol from a normative reference can be dizzying at best.


setFlowModPacket: function(obj, packet, in_port, out_port) {

var dl_dst = packet.dhost
var dl_src = packet.shost
var flow = self.extractFlow(packet)

flow.in_port = in_port

return {
 message: {
   version: 0x01,
     header: {
       type: 'OFPT_FLOW_MOD',
       xid: obj.message.header.xid
     },
     body: {
       command: 'OFPFC_ADD',
       hard_timeout: 0,
       idle_timeout: 100,
       priority: 0x8000,
       buffer_id: obj.message.body.buffer_id,
       out_port: 'OFPP_NONE',
       flags: ['OFPFF_SEND_FLOW_REM'],
       match: {
         header: {
         type: 'OFPMT_STANDARD'
         },
         body: {
           'wildcards': 0,
           'in_port': flow.in_port,
           'dl_src': flow.dl_src,
           'dl_dst': flow.dl_dst,
           'dl_vlan': flow.dl_vlan,
           'dl_vlan_pcp': flow.dl_vlan_pcp,
           'dl_type': flow.dl_type,
           'nw_proto': flow.nw_proto,
           'nw_src': flow.nw_src,
           'nw_dst': flow.nw_dst,
           'tp_src': flow.tp_src,
           'tp_dst': flow.tp_dst,
         },
       },
       actions: {
         header: {
           type: 'OFPAT_OUTPUT'
         },
         body: {
           port: out_port
         }
       }

    }
 }

Performance and Benchmarking

So I used Cbench to compare NOX vs. NodeFlow and here are the results.

NOX [./nox_core -i ptcp: pytutorial]

NOX c++ [./nox_core -i ptcp: switch]:

NodeFlow [running with Debug: False]:

C based Controller:

As you can see from the numbers NodeFlow can handle almost 2X what NOX can do and is much more deterministic. Maxing out at 4600 rsp/sec is not shabby on a VirtualBox VM on my Mac Air!

Summary

At just under 500 LOC this prototype implementation of an OF controller is orders of magnitude less than comparable systems. Leveraging JavaScript and the high performance V8 engine allows for network architects to experiment with various SDN features without the need to deal with all of the boilerplate code required for setting up event driven programming. Hope someone gets inspired by this and takes a closer look at Node.JS for network programming.

So how do I get NodeFlow?

NodeFlow is an experimental system available at GitHub: git://github.com/gaberger/NodeFLow.git along with my fork of the OFLIB-NODE libraries here: git://github.com/gaberger/oflib-node.git. If you would like to contribute or have any questions please contact me via Twitter @gbatcisco

Special thanks to Zoltan LaJos Kis for his great OFLIB-NODE library for which this work couldn’t have been done and Matthew Ranney for his network decoder library node-pcap.

Posted in Data Center, Future Internet, OpenFlow, Software Defined Network | Comments Off

Cisco UCS “Cloud In A Box”: Terabyte Processing In RealTime

Posted on January 5, 2012 by gaberger

Now I hate using the term “Cloud” for anything these days but in the latest blog entry from Shay Hassidim, Deputy CTO of Gigaspaces Terabyte Elastic Cache clusters on Cisco UCS and Amazon EC2 the Cisco UCS 260 took the place of 16 Amazon High-Memory Quadruple Extra Large Instance. With 16:1 scaling imagine what you can do with a rack of these, in other words forget about Hadoop, lets go real-time data grid enabled networking!

With 40 Intel cores and 1TB of memory available to Gigaspaces XAP high performance In Memory Data Grid the system achieved an astounding 500,000 Ops/sec on 1024B POJO, the system could load 1 Billion objects in just under 36 minutes.

Now this might not sound extraordinary, but when you consider how to build an application where the bottleneck on a 40 core, 1TB system is CPU and Memory bound, properly deal with failures and have automation and instrumentation, you can’t beat this kind of system. Gigaspaces is also integrated into Cisco UCS XML-API for dynamic scaling of hardware resources.

Eventually people will catch on that memory is critical for dealing with “Big Data” and it’s no longer an issue of reliability or cost. Without disk rotational latency in the way and poor random access we can push the limits of our compute assets while leveraging the network for scale. Eventually we might see a fusion of in-memory data grids with network in a way, which allows us to deal with permutation traffic patterns by changing the dynamics of networking, processing and storage.

Posted in Data Center, Enterprise Cloud, MultiCore-ManyCore, Platform as a Service | Comments Off

A View from the Developer Nation

Posted on January 3, 2012 by gaberger

QCONSF 2011

By: Gary Berger

Every year the folks at Trifork and InfoQ put on a conference in San Francisco (expanding to other geographies) for enterprise software developers. The conference is geared towards team leads, architects, product managers and developers of all skill sets. This conference is different than most because the constituents are all practitioners (i.e. software developers) not academics, and the agenda spans from Architecture to Agile and Cloud. Many of the leading players in the industry attend, including Twitter, NetFlix, Amazon and Facebook. The discussions are usually very intriguing and provide glimpse into the complex ecosystems of these successful companies.

After being involved in this community for several years I can say that the level of maturity continues to accelerate as developers become experienced with new toolkits, new languages and are encouraged to fail fast through agile techniques like Lean Development, Kanban and Agile. Open Source continues to cross pollinate software developers allowing them to iterate through solutions find what works and keep producing minimal viable products on schedule. The fast innovation in this industry is the culmination of many geniuses over time, paying forward with their hard work and dedication so others can benefit and you can see the influences across architectures starting to emerge.

This paper does not intend to be a comprehensive text on software development practices, tools and techniques. I will however try and distill common themes and highlight specifics, which I find and where others might want to dig deeper to investigate on their own behalf.

JAVA as a Platform

It’s been several years now since I have been advocating the use of JAVA as the abstraction layer for modern applications. I have been exposed to JAVA for many years including designing an infrastructure for one of the world’s first equity matching kernels built in Java. At the time we leveraged Solid Data (DRAM) based storage and one of the first Object Databases to store our transactions, We leveraged CORBA for inter-process messaging and Netscape HTTP server for our front-end. It was a complicated set of interacting components that needed to work together flawlessly to be successful. The system itself was completely over engineered, a standard design philosophy in those days, and performed perfectly for several years before the business could not be sustained any longer. Turned out the “Mom and Pop” investor wasn’t interested in trading after-hours and there just wasn’t enough liquidity to sustain a marketplace.

Needless to say JAVA is everywhere and has recently become poster child for the nascent Platform as a Service model.  Even companies like Yammer or falling back to JAVA from Scala because of the inconsistent build tools and developer education. To be fair, its also the complexities that come from PolyGlot programming so take it or leave it. A great view on the practicality of JAVA from Erik Onnen here: http://bit.ly/t2OJe2. Don’t forget to check out Oracle’s Public Cloud offering here: http://bit.ly/ubHvN0

Why JAVA? Why Now?

About a year ago now, Rod Johnson proclaimed, “Spring is the new cloud operating system”.  Even though the term “cloud” has been overloaded to the extent it’s almost superfluous, it was a profound statement when centered on the JVM and the JAVA language. For more information see Alex Buckley’s presentation on JAVA futures here: http://bit.ly/rQjlsX.

Sidebar: Rod’s Key Note

Rod Johnson is the General Manager of SpringSource, a company acquired by VMWare to expand their footprint into the core of application development. Rod usually speaks about the evolution of the Spring Framework his gadgets such as Spring Roo and Aspect Oriented programming, but this time he was somewhat reflective on his career and his life. From the beginning, his industry acclaimed book on Java development practices turned into a massively successful Open Source project called the Spring Framework from which the first real threat to the JEE ecosystem was felt. Spring, and its philosophy around dependency injection are mainstream and his teams have a catalog of products under the SpringSource banner. Rod has had an astonishing career and talks enthusiastically about the potential for entrepreneurs to change the world through software. See Rod’s presentation here: http://bit.ly/rCh7vz

Oracle acquired Sun Microsystems in early 2009 and the Java community was rocked. Hints of legal battles over Java with Google left people to assess the impact on the Java ecosystem. Even the OpenJDK project the open source incarnation of the JAVA platforms survival was put in question.

Luckily, Oracle has caught on that JAVA is the way forward, and is now marketing the JEE as a “Platform as a Service”. Clearly it is the breadth and depth of the JAVA platform, which makes it successful, with a large set of toolkits, frameworks and libraries a highly educated developer community, the JAVA platform is everywhere. From Google to NetFlix and Twitter, JAVA can be found running some of the worlds most successful systems.

JAVA is now Oracles future. From Fusion to Exalogic and the JEE platform itself taking on a full service model, JAVA continues to have a profound impact on the industry.

Sidebar: The “Application Memory Wall” and its impact on JAVA

When trying to understand the scalability limits of a system we sometimes have to look under the covers at the intent of the application and how the design leverages the core resources of a system. Gil Tene from Azul Systems boils down the problem this way. We all know that the memory wall exists and is determined by the memory bandwidth and latency which evolves at a much slower pace than CPU’s. However we have yet to breach this wall thanks to the phenomenal engineering of modern chips such as Intel Nehalem instead we have been constrained by how applications are designed. This actually creates several issues, as infrastructure designers may “architect” the right solution to the wrong problem.

In the Java world the most restrictive function is Garbage Collection, the process of removing dead objects, recouping memory and compacting the fragmented heap. This process impacts all major applications and teams spend man years tweaking the available parameters on their collectors to push the problem as far into the future as they can, but alas it is unavoidable, the dreaded Stop-the-world-pause. Developers are must be very conscious of their code, how they organize their object models in order to limit the overhead, which every object has on the heap.  (See Attila Szegedi from Twitters presentation http://bit.ly/v8OWmP).

Azul Systems has some of the brightest minds in systems design, memory management and garbage collection and have built a business to solve this specific problem. Currently the most widely used Garbage Collector (GC) is the Concurrent Mark Sweep (CMS) collector in the HotSpot JVM. CMS is classified as a Monolithic Stop-the-World collector, which must provide a full-lock to catch up with mutations (i.e. changes started during GC) and compaction.

The C4 collector from Zing has recently been upgraded to version 5, which no longer requires a hypervisor for its page-flipping techniques and instead has boiled down the necessary Linux kernel changes to a kernel loadable module (trying to be pushed into upstream kernel through Managed Runtime Initiative). This frees up wasted memory and removes the overhead of a hypervisor while still providing pause-less GC for applications. Zing has many more benefits, which can be investigated offline. See Gil’s presentation here: http://bit.ly/vN5xQ0.

The JAVA ecosystem is thriving, both the improvements in the runtime and the language are becoming better at dealing with concurrency, expressiveness and performance. Despite the fact that the current incarnation of Objects has not lived up to their promise (see below), developers seem to always find away around that continuing the evolution of this platform.

Languages

For several years now the developer community has been entrenched in debate over Object Oriented Programming (OOP) vs. Functional Programming, but more specifically over the implementation of OOP methodology in Object Oriented Languages (OOL) such as JAVA.  In part, code reuse, componentization and frameworks have not lived up to their promises. Readability has for the most part been sacrificed, as encapsulation and inheritance have been somewhat abused thanks to the creativity of developers and high-pressure timelines. QCON’s “Objects on Trial” was an interesting debate over the use of Objects. Sorry to say the jury voted “guilty”. Regardless of the verdict Poly Paradigm Computing (i.e. mix of object oriented, procedural, functional, etc..) will play a big role in application development for the same reason why different languages are better for different purposes. This means that app developers need to transparently work across these styles, picking up on the strengths such as reusable software patterns while minimizing the weaknesses such as concurrency support.

Sidebar: Ruby, Groovy, Scala, Erlang, Javascript, DART

Anyone who says language doesn’t matter when designing a system either doesn’t have a system, which cares about flexibility, performance and agility or simply doesn’t have a large enough problem to care.  All things being equal, developers will gravitate to what they are most familiar with regardless of readability, agility or the complexity it adds. Language development continues to evolve, from adding simple annotations replacing lines of boiler plate code, aspect oriented programming to syntactic sugar which makes the readability of code much better. Languages are the face of the development process, the more expressive and easier to use the faster applications can be created and the easier to maintain and extend.

Over the past several years new languages have taken hold such as Ruby, Groovy, Scala, older languages have been given a second life with growing adoption of Erlang and JavaScript and experimental languages have popped up to challenge the others such as Go and DART.

The need for “Finding the right tool for the job” gives rise to the idea of PloyGlot programming, where many different languages are utilized based on their suit for purpose. Mostly you will find JAVA+ in every shop playing a critical role of the system with a sprinkle of JavaScript for client-side coding, some Erlang for middleware maybe, Python, Groovy, C, C++, PERL, etc.. Typically you have a high-performance core written in a statically linked language like C++ responsible for all of the critically performing code, you have a plugin or module architecture to extend the core and a scripting or higher-level language which provides familiar storage primitives, collections, etc..

Frameworks such as Ruby on Rails have been a blessing to those who could leverage its rich dynamic language and templating tools for building quick web-based applications but cursed when trying to do something at a level of scale beyond the Ruby runtime. (Twitter got burned by this and moved to Scala)

For more complex scenarios such as running distributed gossip protocols with 99.999999% reliability, Erlang might be the right choice at the sacrifice of a more complex coding (See Steve Vinoski’s talk on WebMachine, an HTTP server written in Erlang).

There were a number of conversations throughout the conference regarding the importance of a type system. “A type system associates a type with each computed value[1]“. Basically a type system ensures that the value provided or returned from a function is of the expected type. This has several benefits: It allows for code readers to explicitly see what variable types are being passed around (if they don’t obscure the reference), By providing a static type the compiler can allocate the proper size memory and lastly it allows the compiler to check the validity of the variable before getting a runtime error.

Unfortunately this isn’t a panacea, the type system may get in the way especially when the primitive types are hidden making it very difficult to read the code. More on this when we talk about DART.

One of the most popular languages in software development today is JavaScript. JavaScript has gotten a bad wrap and there are many things to be weary of when it comes to writing good JavaScript. Readers should consult Douglas Crockford books on JavaScript for more information http://amzn.to/176RA4.

This tiny little language is a pure Object Oriented Language (everything is an object). It provides for anonymous functions, dynamic typing, and functional reactive programming and has a host of examples from which developers can learn. Oh, and its embedded in every browser running on almost every device you can imagine. Originally JavaScript was designed as a programmatic interface for manipulating the DOM in web browsers but has found its way into server-side implementations through Node.JS and Rhino, which detach the language from the incredibly complex Document Object Model.

Sidebar: JavaScript and Node.JS

The popularity of Node cannot be understated. There is a rich set of new libraries created daily, from LDAP to Riak and all kinds of middleware to solve interesting problems. Node is merely a shim on top of JavaScript and the V8 VM which insulates some of the bad parts of the JavaScript language and confines the developer to a single-threaded model with local scoping, simplifying the design by only allowing sharing state through an IPC barrier similar to the Erlang. Node is exceptional at dealing with network-based applications supporting a variety of services such as HTTP, REST and WebSockets. Node.JS has become one of the most watched projects on GitHub and is gaining in popularity due to its rich but sometimes dangerous base language JavaScript.

As an example, here is how simple it is to create a Web server in Node.


var http = require('http');

http.createServer(function (req,rsp){
rsp.writeHead(200, {‘Content-Type’ : ‘text/plain’});
rsp.end(‘Hello World\n’);
}).listen(8080, “127.0.0.1″);
console.log(‘Server running at http://127.0.0.1:8080′);

I will reiterate the mantra of  “having the right tool for the job” and certainly Node, V8 and JavaScript have their place centered on I/O demanding applications. The eventing model and single-threaded model is not for everyone, but it does provide a powerful tool kit and expressive language to be useful for a number of different use-cases.

Structured Web Programming, Intro to DART

While JavaScript is gaining in popularity, primarily because of the dominance of web based applications and the now future success of HTML5, CSS3 thanks to Adobe dumping flash for mobile programming, but JavaScript may not have all the answers which is why Google is investing in DART.

Google may have realized that the divergence between native applications (i.e. ones that have to be coded for iOS and Android) and web based applications, were not going to be solved by JavaScript or the ECMA community the stewards of the specification so Google created DART.

DART is being created by a number of people within Google most notably are Gilad Bracha who gave the keynote and session on DART and Lars Bak lead developer of Google V8 virtual machine.

Gilad has a long history in language development including his work on StrongTalk and co-authored of the Java Language Specification.

DART is being built as a pure object oriented language with familiar syntax such as Classes and Interfaces with a type system that doesn’t interfere with developer productivity. In its current pre-release DART script is compiled into JavaScript to be run on the V8 virtual machine. Eventually there will be a standalone DartVM (maybe part of Chromium), which will allow DART scripts to avoid this code generation step.

New languages take time to be nurtured and accepted into mainstream, but what makes DART so powerful is the rampant change in client-side development and device refreshes which might give it a great leap forward if the OSS community adopts it.

Data Management

In the field of application science (if there was such a thing), data management would be the foundational pillar to writing applications, which are reliable, flexible and engaging. In the post client/server era we are bombarded with new ways of storing, retrieving, protecting and manipulating data. The 451 Group has put together a very comprehensive look at this space, which incorporates the traditional relational database stores, the emerging NoSQL stores and the second incarnation relational stores called NewSQL. Accompanying them is also the growing and important Datagrid and Caching tier.

Lets look a moment at NoSQL since it is the hottest field in data management today.

In 1998 Carlo Strozzi uses the term NoSQL to describe his new relational database system. In 2004 at USENIX OSDI conference, Jeff Dean and Sanjay Ghemawat published their paper “MapReduce: Simplified Data Processing on Large Clusters”. It was the start of the industries changing viewpoint on data persistence and query processing. By 2007, database guru Michael Stonebraker published his paper entitled “The End of an Architectural Era (It’s Time for a Complete Rewrite). In this paper he exposes the issues around relational database models, the SQL language and challenges the industry to build highly engineered systems that will provide a 50X improvement over current products. He does however ignore the impact of commodity scale-out distributed systems. And in 2009, Eric Evans uses the term “NoSQL” in a conference in describing the emerging “non-relational, distributed data stores”.

These systems have grown from bespoke highly guarded proprietary implementations to now general use and commercially backed products such as Hadoop. While the impact of implementations such as Hadoop, Cassandra, HBase, Voldemort, etc. has yet to be fully understood, every player from EMC to Arista Networks have banked on participating in this growing field and “Big Data” has become the new buzzword in the industry.

SQL and Not Only SQL

Many large-scale shops are using and/or experimenting with a multitude of DME’s including traditional databases such as MySQL, PostGres, BerkleyDB and are starting to experiment with NoSQL implementations. MySQL, PostGres and BDB are still very much in active use and are being used in new ways, i.e. MySQL as a graph database and BDB as a Key/Value store.

Sidebar: Facebook and MySQL

Facebook uses MySQL as a graph database. The data model represents the graph by associating each row with either a vertex or an edge. This data model is abstracted into their API so that the application developer’s just deal with data in those terms, there are no indexes or joins in the data layer. In order to deal with disk bottlenecks they use a block device cache on the database servers to improve I/O especially for random reads.

Hadoop, Cassandra, MongoDB, Riak

Each of these systems has benefits and tradeoffs depending on the use-case. Application designers need to understand how their application behaves, how write oriented they are, how read oriented, how dependent on transactions or sensitivity to bandwidth and latency there application is. The reality there is “no one solution to all problems” and there is a lot of experimentation going on out there.

Hadoop has been a very successful project despite its downfalls. The issue I will highlight with Hadoop relates to the memory management and impact of garbage collection discussed a priori.

Instead I would like to focus on Cassandra. Cassandra combines the eventually consistent model of Dynamo with the data model of Google BigTable. It was created by FaceBook and transferred to the Apache Foundation where it continues to thrive with active participation and new customer adoption. At its core Cassandra is a four to five dimensional key-value store with eventual consistency. You can choose to model it as:

  • Keyspace − > Column Family − > Row − > Column − > Value
  • Keyspace − > Super Column Family − > Row − > Super Column − > Column − > Value

Cassandra provides a number of benefits including its flexible schema, multi-datacenter awareness, its ability to retrieve ordered ranges and high performance distributed writes. Several years ago sites like Twitter were discussing Cassandra it appears that there is plenty of adoption including a major push by NetFlix. See Sid Anand presentation from NetFlix here http://bit.ly/rtSnPR)

Sidebar: Netflix

Most people know that NetFlix discontinued building in datacenters instead chose to adopt full production support on Amazon EC2. Through their partnership, NetFlix has been instrumental in shaping Amazon services and improving their design to support high performance multi-tenant infrastructure services. There is however a downside. Given the high I/O demand for some of NetFlix processes (i..e video codecs) they have to very diligent about understanding the performance envelope of their system. Because Amazon is allowed to choose where VM reside, there is a statistically significant chance that two high-demand users will wind up on the same physical gear. This has lead some to do a burn-in process to indirectly determine if there is another high-demand customer sharing the system, which could cause instability, or breach service levels. See Adrian Cockcroft presentation for more information on NetFlix and Amazon http://bit.ly/ryjft8

Sidebar: A note on Riak

Riak is a distributed Key/Value store somewhat based on Amazon Dynamo. Just as Cassandra, Dynamo uses an eventual consistency model but one which allows the developer to make more discreet choices in the consistency/availably tradeoff i.e. you can choose to accept writes even under partitioning events where you may not have quorum [AP], or you can enforce consistency by making quorum mandatory [CP]). Riak handles complex processes such as, Vector clocks, Merkle trees, consistent hashing, read repair, hinted handoff, gossiping, etc., these infrastructure services are highly complex and are hidden from the application developer through a flexible API.

Riak is written in Erlang, which provides a highly reliable system for building distributed services. Erlang is designed to be highly robust in the face of failure and leverages a number of techniques to maintain reliability. But there is a price for writing in Erlang’s functional event driven style. It is either loved or hated, but the functional model and actor patterns that it embraces has also become a reference point for how new languages (see DART Isolates). Riak’s presentation can be read here: http://bit.ly/vcoX5r.

NewSql, A blast from the past

So what in the world is NewSQL? NewSQL is the second incarnation of relational datastores, which may have more in common with the original use of the term NoSQL by Strozzi. Essentially these database engines provide the same DB semantics such as indexes, foreign keys, stored procedures and are geared towards OLTP workloads that require ACID based consistency. They borrow from a number of techniques found in NoSQL datastores such as sharding and reconciliation but are designed to be compatible with traditional DDL and SQL based language constructs. NewSQL implementations such as VoltDB and Drizzle and will play an increasing role as these types of datastores will continue to live on.  See Ryan Betts presentation here: http://bit.ly/vPHval.


Reliability at Scale

Complex systems can trigger bottlenecks and exhibit near linear falloff as side effects dominate interactions. . Facebook’s Robert Johnson gave a great talk on how they use their fail-fast development process to reduce failures at scale. See Roberts presentation here: http://goo.gl/xAqmz .

FaceBook has a unique position on dealing with scale with over 800M subscribers they process more requests in an hour than most Fortune 500 do in a month. For FaceBook scalability is intrinsically linked to reliability (i.e. dealing with failures). “The goal isn’t to avoid mistakes, it’s to make the costs of mistakes low”. With an aggressive approach to moving things fast, they reduce the convergence time for failure allowing them to be more consistent (sort of a high-pass filter).  By making frequent small changes, they reduce the surface area allowing them to quickly skirt around problems without having a large impact. They have honed this methodology to a craft allowing them to not only fail-fast but also encourage developers to try new things. Their goal is to run a new experiment every day, which allows them to continue to make the site engaging.

Measurement and Monitoring

Critical to all well run services is the proper use of instrumentation and monitoring. In the Internet and more specifically with Cloud, it is even more important as systems typically consist of hundreds if not thousands of independent services tied together to deliver a user facing service.  A theme emerged throughout most of the talks which center around application architectures, which was to instrument the heck out of everything. printf like logging (where the developer inserts log statements into the codebase), GC logging, application logs (i.e. webserver) are all part of a necessity to traceback the system at the time of the failure. In these systems coordinated failures (failures which coincide across many services) become more prominent and latency issues can be hidden in the inter-service calls requiring a mass amount of information. Some have hundreds of machines dedicated to instrumentation, logging and monitoring.

Sidebar: Disaster Porn and Joyent

One of the most interesting talks was by Bryan Cantrill from Joyent. Bryan is one of the original developers of DTrace at Sun Microsystems and now runs one of the largest competitors to Amazon EC2. DTrace is an expressive debugging tool built in Solaris it is also incorporated in the Open Source version of Solaris called Illumos and is a core component in MacOS. DTrace allows you to do some very sophisticated things including filtering the call stack running across every core and embedding references in dynamic languages such as JavaScript to reconcile address pointers to libraries and even to source code. Bryan demonstrated some wizardry of analyzing a core dump of a crashed Node.js application, which was simply unbelievable. It means that the pains of troubleshooting stack traces of managed languages such as Python, Ruby and JavaScript are within reach (as long as you use Illumos). See Bryan’s presentation here: http://bit.ly/rGfKTP

The Simian Army

So how do you deal with a business that is tied together with hundreds if not thousands of services? Each having some combinatorial effect on the other but distributed which makes understanding what is happening at any point in time almost impossible? Call in the monkeys!!

Chaos Monkey

 “The Chaos Monkey’s job is to randomly kill instances and services within our architecture. If we aren’t constantly testing our ability to succeed despite failure, then it isn’t likely to work when it matters most – in the event of an unexpected outage.”[2]

What started as a methodology to literally create chaos or the worst-case situation for operations has turned into a methodology for coping with the unknown? Keeping customers happy requires a range of understanding complex interactions from the impact of a simple server failure to more exotic memory corruption errors and race conditions.

As noted earlier, modern applications are designed to be both modularized and resilient in the face of catastrophe. In a similar vane to the way packet networks were designed to continue operation on the loss of a packet, modern application services are isolated from one another so as to avoid a mass cascaded failure which would impact the entire service. In a technique similar to the way security researches Fuzz applications in search of memory corruption vulnerabilities, the Simian Army is put to work to challenge the best developers to keep their systems running in the face of utter chaos.

Latency Monkey

Hard failures are one thing but soft failures such as service degradation are harder to spot before users start to feel something is wrong. These monkeys inject random delays in client-side and server-side processing to test the ability for applications to detect and recover in a graceful way. These tests might expose more complex issues such as thundering herd and timeouts.


The Role of the Network

 The network is a critical component especially as designers gravitate towards distributed services. Whether or not you are dealing with two-phase commit or quorum based consistency models the network plays a major role in application design. Application architects must make clear tradeoffs when it comes to consistency and availability under network partitioning events and must pay clear attention to how the network flow and error control are impacting the application.

During Facebook’s presentation they were clear to post the slide to the right to the community when talking about dealing with scalability.

Incast[3][4] is a well-documented phenomenon but solutions vary from engineering larger buffers to high-resolution TCP timers. The impact for applications with coordinated “synchronization” is a complete falloff in network throughput for 100s of milliseconds. There is also another phenomenon called “port blackout” or “outcast”[5] where switches with tail-drop queues exhibit unfairness due to RTT bias. The effect is that short-flows with small RTT can be starved by longer-flows with larger RTT. Given the sacrifices that have to be made under these conditions and others. Application developers struggle with dealing with prioritizing communications. For critical writes the network must have capacity to commit in a consistent way, non crucial services such as bookkeeping writes should be relegated to a best-effort service. Today application architects solve these problems a number of ways, from moving to UDP and building error and flow control in the application layer to providing rate limiters within the application API itself.

Network designers try and solve the problems with larger buffers (which can cause instabilities itself because the error control channel is caught up in data transfer PDUs), leverage active queue management such as RED (although algorithmically flawed) and even stochastic based drop queues (i.e. random instead of tail-drop).

Conclusion

Application development continues to go threw a massive amount of change. We have not discusses some of the practices around Agile, Lean software development, technical debt and all of the work going into the client-side such as HTML5 and CSS. We are seeing an increasing role in dealing with infrastructures at scale and redefining the proper models and patterns for continuously reengineering to grow the business. We all know that Moore’s Law does not track the improvements in performance and our view of time shared systems doesn’t reflect the combinatorial effects if O(N^2) application interactions. Hypervisor based clouds are also becoming problematic for some. A great talk by Erik Onnen from Urban Airship talked about the difficulties in maintaining their business on EC2 including:

  • Erratic network latency
  • Strange network behavior
  • Kernel hypervisor issue
  • Undocumented limitations
  • Database scaling

This should not be a big surprised, after all Hypervisor based systems are based on multi-tiered time sharing which can be highly volatile especially for data intensive applications. Erik’s presentation can be seen here: http://bit.ly/sHZFpY.

As application developers learn to improve their use of memory, and are given better interfaces to manage communications and quality of service it may be possible to solve some of these challenges. Certainly there is no turning back, these problems need to be fully understood and plans need to be in place so businesses can continue to use the Internet to grow their businesses, put people to work to strengthen world economies and allow innovation to transcend individual expectations to make a better world for tomorrow.

[1] http://en.wikipedia.org/wiki/Type_system
[2] http://www.codinghorror.com/blog/2011/04/working-with-the-chaos-monkey.html
[3] http://dl.acm.org/citation.cfm?id=1049998
[4] http://www.pdl.cmu.edu/PDL-FTP/Storage/CMU-PDL-09-101.pdf
[5]http://www.cs.purdue.edu/research/technical_reports/…/TR%2011-019.pdf

Posted in Software Development | Comments Off

Forrester Views Cloud/Web is Outmoded and App-Internet is the new model

Posted on December 20, 2011 by gaberger

LeWeb 2011 George Colony, Forrester Research “Three Social Thunderstorms”

Over the past several years the word ‘Cloud’ has been used and to some extent abused  almost to the point of being superfluous. Every technology company, provider and enterprise is immersed in some sort of “cloud” project although the exact descriptions of these projects may fall short of the NIST formal definitions.  I think as technologists we tend to rebel against the status quo in attempt not just to redefine the marketplace but also to claim for our own a new path as we iterate over the current challenges for delivering new applications and services.

Just as we have overused and bludgeoned the hell out of terms like internet, virtualization and web (the prior name cloud), we are bound to move into a new set of vernacular definitions such as intercloud, interweb, fog computing  or in the case of Forrester CEO George Colony APP-Internet.

“Web and cloud are .. outmoded” concludes Mr. Colony as he goes on to explain the App-Internet as the next model offering a “faster, simpler, more immersive and a better experience”.

The thesis for this conclusion is based on the figure above where the y-axis is defined as “utilities per dollar” and the x-axis is time. P is representative of “Moores Law” and speaks to the scalability of processing power. In reality the beauty behind Moores law is lost in translation. What Moore really said was “transistors on a chip would double every year” and subsequently David House, an Intel executive at the time, noted that the changes would cause computer performance to double every 18 months [1].

If you plot transistors per chip against actual computer performance you might see a different picture due to the thermodynamic properties and manufacturing complexity of CMOS based technology not to mention the complexity in actually utilizing that hardware with todays languages, application methodologies, libraries and compilers.

S is for the growth in storage which Colony calls the “Hitachi’s Law”. This predicts that storage will double approximately every 12 months. This also is somewhat contrived as the limits of scaling magnetic medium on disk are becoming extremely difficult as we approach the limits of perpendicular recording although maybe there is some promise with the discovery of adding NaCl to the recoding process[2]. Yes we can build bigger houses with disks packed to the ceiling, but the logistics in managing such a facility is increasingly hitting the upper limits. (imagine shuffling through a facility over 100,000sqft and replacing all those failed hard drives)

N is related to the network where Colony goes on to describe the adoption rates of 3G vs 4G. First and foremost nailing down exactly what 4G is and means is an exercise in itself, as most vendors are implementing various technologies under this umbrella[3]. With an estimated 655Million people adopting 4G in its various forms by 2010[4] and the quick adoption of new mobile devices, I think this is a bit short sighted..

But there is another aspect to this which is missing which is all of the towers that collect those 3G and 4G signals need to be back-hauled into the Internet backbone. With 40GE/100GE ratified in the IEEE, I suspect the first wave of 100GE deployments to be put into production in 2012 [5]

Colony goes on to say “If your architecture was based on network you are wasting all of these improvements in processing and storage.. the center (meaning the warehouse scale datacenters such as Google, Amazon and Microsoft) is becoming more powerful and the periphery is becoming ever more powerful…

His point is valid to an extent but not because of the P, S, N curves but because now that the devices are so powerful AND we have such a robust network infrastructure we can take advantage of all of this processing power and storage available to us. Afterall if transport pricing had continue to rise as the late great Jim Gray predicted in his paper on Distributed Computing Economics [7] we would not even be having this discussion because without the distribution of data capability in the network, all we would have were some very smart expensive devices that would essentially be a fancy calculator.

To that point Colony compares todays devices with their predecessors but as stated earlier its not a fair comparison. “In 1993 the iPad 2 would have been considered one of the 30 fastest computers in the world”. Unfortunately the problem space has changed from 1993 and if we follow Parkinsons Corollary called “Jevons Paradox” or the proposition that technological progress that increases the efficiency with which a resource is used, tends to increase (rather than decrease) the rate of consumption of that resource[6] it would be hard to compare these two accurately.

So the reality is that all of these iterations, from the early ARPANET viewpoint of access to expensive time-sharing computer centers to the highly distributed and interconnected services we have today are just a succession of changes necessary to keep up with the demand for more information. Who knows what interesting changes will happen in the future but time and time again we have seen amazing strides taken to build communities and share our lives through technology.

 So lets take a closer look at the App-Internet model.

Hmm. So how is this different from todays “Web-Centric” application architecture? After all isn’t a web browser like Chrome and Safari an “application”?.

Jim Gray defined the ideal mobile task to be stateless (no database or data access), has a tiny network input and output and has a huge computational demand[7]. To be clear, his assumptions of course were that transport pricing would be rising to make the economics infeasible, but as we know the opposite effect happened as transport pricing has fallen

[8]

“Most web and data processing applications are network or state intensive and are not economically viable as mobile applications” Again the assumptions he had about telecom pricing made this prediction incorrect. He also contended that “Data loading and data scanning are cpu-intensive; but they are also data intensive and therefore are not economically viable as mobile applications. The root of is conjecture was that “the break-even point is 10,000 instructions per byte of network traffic or about a minute of computation per MB of network traffic”.

Clearly the economics and computing power has changed significantly in only a few short years. No wonder we see such paradigm shifts and restructuring of architectures and philosophies.

The fundamental characteristic which supports a “better experience” is defined as latency. We perceive latency as the responsiveness of an application to our interactions. So is he talking about the ability to process more information on intelligent edge devices? Does he not realize that a good portion of applications written for web are built with JavaScript, and that the advances in Virtual Machine technology like Google V8 is what enables all of that highly immersive and fast responding interactions? Even data loading and data scanning has improved through advances in AJAX programming and the emerging WebSockets protocol allowing for full duplex communications between the browser and the server in a common serialization format such as JSON.

There will always be a tradeoff however especially as the data we consume is not our own but other peoples. For instance, the beloved photo app in Facebook would never be possible utilizing an edge centric approach as the data actually being consumed is from someone else. There is no way to store n^2 information with all your friends from an edge device it must be centralized to an extent.

For some applications like gaming we have a high-sensitivity to latency as the interactions are very time-dependent both for the actions necessary to play the game but also how we take input for those actions through visual queues in the game itself. But if we look at examples such as OnLive which allows for lightweight endpoints to be used in highly immersive first-person gaming, clearly there is a huge dependency on the network. This is also the prescriptive approach behind Silk, although Colony talks about this in his context of App-Internet. The reality is that the Silk browser is merely a renderer. All of the heavy lifting is done on the Amazon servers and delivered over a lightweight communications framework called SPDY.

Apple has clearly dominated pushing all focus today on mobile device development. The App-Internet model is nothing more than the realization that “Applications” must be in the context of the model something which the prior “cloud” and “web” didn’t clearly articulate.


The Flash wars are over.. or are they?


 So what is the point of all of this App-Internet anyway? Well, the adoption of HTML5, CSS3, JavaScript and advanced libraries, code generations, etc.. have clearly unified web development and propelled the interface into a close to native environment. There are however some inconsistencies in the model which allows Apple to stay just one-step ahead with the look and feel of native applications. The reality is we have already been in this App-Internet model for sometime now, ever since the first XHR (XMLHttpRequest) was embedded in a page with access to a high performance JavaScript engine like V8.

So don’t be fooled, without the network we would have no ability to distribute work and handle the massive amount of data being created and shared around the world. Locality is important until its not.. at least until someone build a quantum computer network.

over and out…

  1. http://news.cnet.com/2100-1001-984051.html
  2. http://www.techspot.com/news/45887-researchers-using-salt-to-increase-hard-drive-capacity.html
  3. http://en.wikipedia.org/wiki/4g
  4. http://www.fiercewireless.com/story/real-world-comparing-3g-4g-speeds/2010-05-25
  5. http://www.businesswire.com/news/home/20110923005103/en/Xelerated-Begins-Volume-Production-100G-Network-Processor
  6. http://en.wikipedia.org/wiki/Jevons_paradox
  7. http://research.microsoft.com/apps/pubs/default.aspx?id=70001
  8. http://drpeering.net/white-papers/Internet-Transit-Pricing-Historical-And-Projected.php (Note: This is more representative as a trend rather than wholly accurate assessment of pricing)
Posted in Data Center, Enterprise Cloud, Future Internet, Software Development | Comments Off

Cloud Networking Hyper or Reality?

Posted on December 14, 2011 by gaberger

A colleague of mine pointed out a new post by Jayshree Ullal from Arista Networks on Cloud Networking Reflections. I can’t help to comment on a few things for my own sanity.

Prediction #1: The rise in dense virtualization is pushing the scale of cloud networking.

Evaluation #1: True

IT is very “trend” oriented, meaning sometimes the complexity of operating a distributed system are people are too busy look deep into the problem for themselves and instead lean on the communities of marketing wizards to make a decision for them. Despite VMWare’s success, hardware virtualization makes up a very small part of the worldwide server base, which is estimated at around 32M servers [1]. I predict within a few short years a reversal in this trend, which peaked around 2008 for several reasons.

  • One is the realization that the “hardware virtualization” tax grows increasingly with I/O, a very significant problem as we move into the era of “Big Data”. The reality is as we move to more interactive and social driven applications the OS container is not as crucial as it is in a generalized client/server model. Application developers need to continuously deal with higher degrees of scalability, application flexibility, improved reliability, and faster development cycles. Using techniques like Lean software development and Continuous Delivery, application developers can get a Minimal Viable Product out the door in weeks sometimes days.
  • Two, the age of  ”Many Task Computing” is upon us and will eventually sweep away the brain-dead apps and the entire overhead that comes with supporting multiple thick-containers. I say lets get down with LXC or better yet Illumos Zones, which gives us the namespace isolation without the SYSCALL overhead.
  • Three, heterogeneous computing is crucial for interactive and engaging applications. Virtualization hides this at the wrong level; we need the programming abstractions such as OpenCL/WebCL for dealing with specialization in vector programming and floating-point support via GPU’s. Even micro-servers will have a role to play here allowing a much finer grain of control while still improving power efficiency.

Its not “dense virtualization” pushing the scale of cloud networking, it is the changing patterns of the way applications are built and used. This will unfortunately continue to change the landscape of both systems design as well as network.

My Advice: Designers will finally wake up and stop being forced into this “hyper-virtualized” compute arbitrage soup and engineer application services to exploit heterogeneous computing instead of being constrained by a primitive and unnecessary abstraction layer. In the mean time, ask your developers to spend the time to build scalable platform services with proper interfaces to durable and volatile storage, memory and compute. In this way you isolate yourself from specific implementations removing the burden of supporting these runaway applications.

Prediction #2: “Fabric” has become the marketing buzzword in switching architectures from vendors trying to distinguish themselves.

Evaluation #2: Half-True.

I think the point of having “specialized” fabrics is a side effect of the scalability limits of 1990′s based network design, protocols and interconnect strategies. Specialized and proprietary fabrics have been around for years, Think Machines, Cray, SGI and Alpha all needed to deal with scalability limits connecting memory and compute together. Today’s data centers are an extension to this and have become modern super-computers connected together (i.e. a fabric)

Generally the current constraints and capabilities of technology today have forced a “rethink” on how to optimize network design for a different set of problems. There is nothing terribly shocking here unless you believe that current approaches are satisfactory. If the current architectures are satisfactory, why do we have so much confusion on whether to use L2 multi-pathing or L3 ECMP? Why is there not ONE methodology for scaling networks? Well I’ll tell you if you haven’t figured it out. Its because the current set of technologies ARE constrained and lack the capabilities necessary for truly building properly designed networks for future workloads.

“The beauty of Arista’s approach is we can scale and manage two to three times better with standards. I fail to understand the need for vendor-specific proprietary tags for active multipathing when standards-based MLAG at Layer 2 or ECMP at Layer 3 (and future TRILL) resolves the challenges of scale in cloud networks. “

Scale 2x to 3x better with standards? How about 10x or better yet 50x? Really 2-3x improvement in anything is statistically insignificant and you are still left with corner cases, which absolutely grind your business to a halt. Pointing out MLAG is better than TRILL or SPB or ECMP is better than whatever is not the point. I mean really, how many tags do we need in a frame anyway and what the hell with VXLAN and NVGRE? Additional data-plane bits are not the answer, we need to rethink the layering model, address architecture and error and flow control mechanisms.

There is no solution unless you break down the problem, layer by layer until you remove all of the elements down to just the invariants. Its possible that is the direction of OpenFlow/SDN, the only problem maybe that completely destroys the layers entirely but maybe that’s the only way to build them back up the right way.

BTW. There is nothing really special about saying “standards”, after all TCP/IP itself was a rogue entry in the standards work (INWG 96) so its another accidental architecture that happened to work.. for a time!

My Advice: For those who have complete and utter autonomy, treat the DC as a giant computer which should be designed to meet the goals of your business within the capabilities and constraints of todays technology. Once you figure it out, you can use the same techniques in software to OpenSource your innovation making it generally feasible for others to enter the market (if you care about supply chain). For those who don’t, ask your vendors and standards bodies why they can’t deliver a single architecture which doesn’t continuously violate the invariances by adding tags, encaps, bits, etc..

Prediction #4: Commercially available silicon offers significant power, performance and scale benefits over traditional ASIC designs.

Evaluation #4: Very true.

Yea no surprise here, but its not as simple as just picking a chip off the shelf. When designing something as complex as an ASIC, you have to make certain tradeoffs. Feature sets build up over time, and it takes time to move back to a leaner model of primitive services with exceptional performance. There is no difference between an ASIC designer working for a fabless semiconductor company spinning out wafers from TSMC and a home grown approach, it is in the details of the design and implementation with all of the sacrifices one makes when choosing how to allocate resources.

My Advice: Don’t make decisions based on who makes the ASIC but what can be leveraged to build a balanced and flexible system. The reality is there is more to uncover than just building ASIC’s, for instance how about a simpler data plane model which would allow us to create cheaper and higher performance ASIC’s?

Prediction # 5: FCoE adoption has been slow and not lived up to its marketing hype.

Evaluation # 5: True.

“A key criterion for using 10GbE in storage applications is having switches with adequate packet buffering that can cope with speed mismatches to avoid packet loss and balance performance, “

This is also misleading as it compares FCOE with FC with 10GE sales as a way of dismissing a viable technology. But the reality is that the workload pattern changed moving the focus from interconnect to interface.

From an application development point of view, interfacing with storage at a LUN or “block” level is incredibly limited. It’s simply just not the right level of abstraction, which is why we started to move to NAS, or “file” based approaches and even converging the reemergence of content based and distributed object stores.

Believe me, developers don’t give a care if there is an FC backend or FCOE, it is irrelevant, the issue is performance. When you have a SAN based system you are dealing with a system balanced for dealing with different patterns of data access, reliability and coherency. This might be exactly what you don’t want, you may be very write intensive or read intensive and require a different set of properties than current SAN arrays provide.

The point about adding buffering to the equation not only makes things worse, but also increases the cost of the network substantially. Firstly the queues can build up very quickly especially at higher clock speeds and the impact on TCP flow-control is a serious issue. I am sure the story is not over and we will see different ways of dealing with this problem in the future. You might want to look a little closer at FC protocols and see if you can see any familiarity with TRILL.

My Advice: Forget the hype of Hadoop and concentrate on isolating the workload patterns that impact your traffic matrix. Concentrate on what the expectations of the protocols are, how to handle error and flow control, mobility, isolation, security and addressing. Develop a fundamental understanding of how to impart fair scheduling in your system to deal with demand floods, partitioning events and chaotic events. Turns out a proper “load shedding” capability can go along way in sustaining system integrity.

…

Yes I know, thats a lot of opaque nonsense, and while many advantages exist for businesses which choose to utilize the classical models, there are still many problems in dealing with the accidental architecture of todays networks. The future is not about what we know today, but what we can discover and learn from our mistakes once you realize we made them.

While I do work at Cisco Systems as a Technical Leader in the DC Group, these thoughts are my own and don’t necessarily represent those of my employer.

[1] http://www.mediafire.com/file/zzqna34282frr2f/koomeydatacenterelectuse2011finalversion.pdf

 

Posted in Data Center, Enterprise Cloud, Future Internet | Comments Off

Join the New York SDN Users Group

Posted on October 31, 2011 by gaberger

http://www.meetup.com/The-New-York-SDN-User-Group

Explore the world of Software Defined Networking with your colleagues and meet some of the most influential people in the SDN movement today.

Posted in Uncategorized | Comments Off

A look back at Internet History

Posted on September 20, 2011 by gaberger

While doing research on the evolution of protocols I came across an interesting documentary film about the history of the ARPANET and birth of the Internet[1]. Definitely a must watch if you haven’t seen it to see how our industry was first envisioned by the early ARPAnet engineers.

Some reflection on the ideas which even today have not fully lived up with the pre-Internet expectations.

Applications moves across computers transparently, while the network provides automatic load leveling to avoid congestion. Users login to the network without care of where the computation is taking place and distributed operating systems decide which computer can best perform the job.

Files can be backed up on more than one site so files are always accessible and retrievable. People, data, computers, circuits, protocols and facilities are all resources which allow people to work together and collaborate.

Computing centers can be built with specialized form and function, data communication evolve allowing every terminal, computer to interconnect. Connect at messages not at circuits.

Some day you can store 100,000 books for $1MM

Many realize a big problem in the Internet (really not an Internet but a bunch of concatenated IP networks or catenet) has been around the adoption of TCP/IP by DARPA in the early days. Alex McKenzie from BBN has a great history on the discussions that progressed from ARPANet NCP to INWG-96[2]

Even after the international body selected INWG-96 for testing amongst the emerging networks such as ARPANET and Cyclades. DARPA adopted TCP/IP, the somewhat defective version of INWG 39 that lacked a proper addressing model, incorporated an ineffective congestion control system, and failed to clearly distinguish the end-to-end TCP layer from the IP layer (the infamous TCP “pseudo-header” ties TCP sessions to specific network interfaces and sub-networks.) The effort to correct the TCP/IP deficiencies, ISO’s Open System Interconnect project, became mired in politics and missed deadline after deadline, ultimately missing the window for an Internet-wide upgrade, and now we’ve landed in the IPv6 conversion.[3]

There are many ideas how to solve the problems in TCP/IP some are under the Clean-Slate programs such as GENI, FIND, etc.. Some see hacking the datalink layer, others RINA and yet others see OpenFlow as the framework to fix the structural problems with TCP/IP. A good overview of the problems and the possible solutions was just published by Time Warner Cable Research, Remaking the Internet: Taking Network Architecture to the Next Level[4]

All of the current protocol work masks the problems of the network and internetwork layer (figure 1). This creates complexity, adds to the cost of the network and ultimately diminishes the use of the network I.e. Such as the move to cloud computing.

Clearly this problem exists, its not going away and in fact is getting worse every day as the amount of smart connected devices grows. Eventually someone will need to push this industry into proper theory and engineering after 40yrs of craft. This is not to say it is easy, it won’t be. Many still have a hard time seeing beyond TCP/IP as its the only protocol they have known.

Clean-Slate opportunity is all around for those willing and able to take on this endeavor. We have learned a alot in the past 4 decades, TCP was under development for almost a decade and implemented in less than 1 year from specification to implementation . I think its time for a change.

Figure 1

  1. http://video.google.com/videoplay?docid=4989933629762859961
  2. http://alexmckenzie.weebly.com/inwg-and-the-conception-of-the-internet-an-eyewitness-account.html
  3. http://www.hightechforum.org/the-internets-first-turning-point/
  4. http://www.twcresearchprogram.com/pdf/TWC_Bennett_v3.pdf
Posted in Future Internet | Comments Off

Facebook announces HipHop for PHP

Posted on February 5, 2010 by gaberger

Companies like Facebook not only see their core values centered around delivering products and services which “impact” people but also technology. Besides the OSS contributions to Memcached, Hive, Cassandra and Thrift, facebook team announced HipHop for PHP.

Mark Zuckerberg adopted OSS early on utilizing Memcached, Apache HTTP, MySql and PHP in his Harvard dorm to build the first Facebook site. Facebook is now comprised of millions of lines of PHP code. PHP provides Facebook with an expressive well understood programming environment for quickly generating new web content.

But of course due to their size and scale ( 400 Billion page views per month servicing 350 million people) they were faced with the constraints of an interpreted scripting language and its impact on CPU and memory.

PHP and ZEND runtime are highly optimize but at large scales the inefficiencies are quite costly. HipHop for PHP is a code generation technology with static analysis and transformation capabilities to turn PHP source code into c++ source code able to be compiled by g++. HipHop allows Facebook to run 50% less CPU with equal traffic on their web tier and 30% less CPU with 2x traffic on their application tier. Thats a significant number in terms of 10s of thousands of servers.

By allowing developers to continue to use PHP with its common syntax, tools and debugging, they don’t have to learn another language and deal with changing millions of lines of code (although who knows if that will be inevitable). Developers can code and debug with their natural toolsets and dynamic runtime while in development and than promote their code into a compiled set pushed out to the production systems.

HipHop is a great example of the ingenuity of the developer community to think outside the box and concentrate on delivering value to the business. I am sure their efforts will benefit the community of PHP developers at large in a profound way.

Bravo Facebook team..

Posted in Uncategorized | Comments Off

Distributed Computing

Posted on February 4, 2010 by gaberger

We can all agree that we are in the midst of a shift in the practice of information technology delivery, fueled by economization, global interconnection and changes in both computer and social sciences.

Although this can be considered revolutionary change in some circumstances, it is rooted in problems known almost 20 years ago. For those of you interested in the history and a very clairvoyant look at this current shift read “A Note on Distributed Computing, “. This paper concentrates on integration of the current language model to address the issues of latency, concurrency and partitioning in distributed systems.

“They [Programmers[ look at the programming interfaces and decide that the problem is that the programming model is not close enough to whatever programming model is currently in vogue..A furious bout of language and protocol design takes place and a new distributed computing paradigm is announced that is compliant with the latest programming model. After several years, the percentage of distributed applications is discovered not to have increased significantly, and the cycle begins anew."

This paper concludes with very specific advice:

"Differences in latency, memory access, partial failure, and concurrency make merging of the computational models of local and distributed computing both unwise to attempt and unable to succeed."

Now there are a few things not known back in 1994 including where exactly Moores Law would take us, language development, ubiquitous device access and the scale at which the Internet has grown but when you examine the issues discovered by the likes of Google, Amazon, Facebook, etc... you recognize that the cycle has indeed begun anew.

The interesting part is the velocity of innovation to solve these problems along with the cooperative nature of open source software has fueled an even broader manifestation of change and companies of all sizes can help contribute to the greater good of open software, enabling communities of interest to develop and share information in an open, yet secure way.

[1] Waldo, Wyant, Wollrath, Kendall, Sun Microsystems, 1994, SMLI TR-94-29

Posted in Enterprise Cloud | Comments Off