I spent the Week at EclipseCon, the Open Source Software tools conference. EclipseCon is a conference like no other, it is where the industry gets together to discuss how it is building the tools that are used to build the applications that we are all using. Since tools precede applications it tends to see into the future. Eclipse is the dominant non-Microsoft software tools platform, so unless it can be built using things that are currently being built for Eclipse or by Microsoft Visual Studio 2010, it’s very unlikely that it will be built in the next few years. Conversely, tools are only there to sell runtimes, and if there are developments in runtimes there will be investment in corresponding features of tools.
The perplexing feature of EclipseCon is that there are almost no users present, except in the sense that everyone is eating their own dog food, using Eclipse to build things in and/or for Eclipse. This means there is no hype, just a hard-bitten technical cynicism about how the marketing guys are spinning the latest technology. And yet you can see the cloud creeping across the hallways and in through the doors of the conference sessions, and onto the presentations and panel sessions.
The first conclusion of this Open Source community, slightly self-congratulatory, is that their tools and their runtimes are building the cloud. There is one exception – Microsoft Azure, but that isn’t viewed as a serious threat, the view of the community is that, far from being a commoditizer, the community model of Open Source is also the best and cheapest way to innovate, and can out-innovate Microsoft, bankrupting its R&D budget just as president Reagan’s star wars initiatives are thought by many to have out-invested and out-innovated the Soviet Union and put an end to the Cold War.
Microsoft were at EclipseCon both directly and through partners and we will be filing another post on Azure and Open Source – strange bedfellows, but bedfellows they are.
One of the key issues we’ve been tracking is the runtime platform for the cloud. In this context there was an illuminating panel session on the future of Application Servers, and the way that they need to change to support the cloud. Although different people are starting from different places, there was largely a consensus on what needed to happen.
The big mundane but crucial issue is management. You need to be able to securely deploy applications to large-scale application servers running across hundreds of machines, and the universal expectation is these machines will be virtual. The management tools need to be able to deal with the scale, and (as far as most vendors believe) need to handle secure multi-tenanting and to provide billing/charge-back mechanisms.
The application server needs to scale out horizontally, so that when the management layer adds resources they can be effectively used by the application. Also, when services are provided via the application server, these too need to scale out. Scaling in the java stack is best provided by intelligent distributed caching, and there are many ways to do this. There will be some consolidation of caching solutions as the stack vendors (Oracle IBM, Red Hat, Novell) choose their strategic caching technologies.
The feeling at the application server level at least is that we are finally moving to a post-relational world, driven by the persistent distributed object cache, rather than the relational database. Now we have been hearing this for twenty years (indeed since Reagan was president), but the current argument is as follows, the tabular nature of the relational database is ideally suited to linear structures like disk drives where disk heads have to move around and so random access is much slower than block access, but for some time it has been more cost-effective to throw lots of machines at the problem and have as much as possible in RAM, rather than have everything pass through a table on a disk. Once you are distributed and in RAM and managing an object cache, the tabular artefact associated with disk drives can be dispensed with and you can work with whatever data-structures exist within your application, preferably some kind of tree that can be partitioned across machines and used to drive data locality via the load-balancer.
If you are Oracle, you are fighting a rearguard action to defend your revenue base while you advance to conquer new territories. If you are Red Hat you are charging on into the future.
Co-existence of OSGi and JEE platforms
We’ve mentioned OSGi in the context of VMware/SpringSource, the reality is that there are two completely different platforms with strongly overlapping feature sets. Compatibility is being achieved by gluing two servers together and pretending they are one. OSGi runs in places where JEE Application Servers don’t go (like mobile phones) so it stands a chance of ultimately dominating JEE. OSGi is much more modular, but if the intention was through modularity to simplify the life of the application programmer, there is currently no sense in the tools community that the goal has been achieved.
Non-Java Languages on Java Virtual Machines and Application Servers
Application Servers on bare-metal Hypervisors
And so we come to the elephant in the room. If you can deploy and manage applications written in any language to run on a JVM using application servers where they can access all the services they require, where exactly is the operating system in the mix? Obviously there’s no consensus on this one,with Red Hat protesting that with WebLogic Server Virtual Edition, the approach has been tried with limited market uptake, but hey, so have persistent object caches and indeed the Cloud itself is a re-hash of the Application Service Provider of 1999.
Something Else Entirely
Yet at EclipseCon there is a nagging doubt that even with all this stuff (which is way ahead of simply putting VMs in the cloud) we are simply doing a variation on what we did before, it isn’t radical enough. With the exception of Hadoop, there’s nothing yet that was “born in the cloud”, and we should expect someone somewhere in the ultra-innovative Open Source tools community to seed the cloud so that it ultimately delivers a rainstorm that washes away the present.