IT Forecast for 2010

I started making forecasts for the IT sector (and the XML community within that sector) in 2005, though the precise venue for those forecasts has changed depending on who I'm writing for at any given time. In the past I've reviewed my forecasts from the previous year just to get a gauge on my own accuracy, but as my predictions last year were scattered across about six different articles, I think it may be better here to step back a bit and condense things down. I'll try to cover the relevant predictions from the past year here, when appropriate (I actually nailed all my predictions this last year). As per usual, I'll start with big macro-trends and work my way down to the IT, then XML markets in general, keeping these focused on how they will affect the IT and XML communities specifically.

1. Economic Whiplash

2009 will not live in most people's books as being one of the more pleasant of years - the market, already in free fall after the Lehman crash in November, continued slide until it hit a low (S&P wise) of 667 in March. After that, it rebounded, dramatically, so that it is now sitting at 1127 as I write this today. This is still well below the high point for that index, of course, but on paper it looks fantastic - if you had invested heavily in March, you would have seen returns well north of 60% in just nine months.

Some of that was dead cat bounce - when the markets fall dramatically, they hit a bottom that may or may not represent the final lows of a cycle, then usually rise again as investors pile back in, thinking to snatch up stocks at generational low prices. Some of it (by some accounts much of it) may have been due to the US Government turning the normal cash flow into the Fed into a firehose, in an effort to make the banks solvent enough to lend once again, and hence get the engines of growth restoked. Whether this was all legal is a question that I suspect will likely be answered in subsequent years - once the dust finally settles - and the verdict will almost certainly depend upon what happens in the next eighteen months.

Unfortunately, there's now a rather awkward situation at play - the banks are, on paper, more or less solvent. Unfortunately, that paper is also dependent upon a few convenient fictions, not least of which being that the banks are able, for the time being at least, to put any valuation at all on their assets no matter how absurd in order to determine their capitalization ability. This means that, so long as "Mark to Model" remains in effect with FASB, there's no real way for the market to set corrective values on the assets, which in turn means that mortgages will continue to be overpriced, commercial real estate (the NEXT wave of defaults) will continue sitting in a zombie state, and as a consequence the real "correction" that the economy needs in order to become healthy once again remains in abeyance.

The second problem with this situation is that the decision has been made to sit on interest rates at 0%. In effect, it's more profitable for the banks to sit on their assets than it is to loan the money out. If the profit in question is going back into making the banks whole, this is probably not a bad thing - depending upon the particular economic philosophy that you follow. In the Keynesian model, this is the equivalent of filling up the gas tank, and you're not likely to leave the gas station if the tank is only half full. The concern is that there may very well be people siphoning off that gas even as it fills, and similarly that there are leaks in the gas tank (due to interest accruel on all that debt) that may in fact be draining off the incoming gas faster than its being filled.

Unfortunately, at this stage, few banks are becoming forthcoming about their own debt situation, and increasingly people are becoming ... concerned ... that the gas tank has in fact been ripped out entirely and the gas is being poured into a whole in the ground that may be enriching a few people but at the expense of everyone else. Ironically, the Internet is becoming increasing central to these questions being raised. The last time this was tried (in the late 1990s when Long Term Capital Management imploded and very nearly brought down Wall Street) there were few awkward questions being raised, most people were essentially unaware of how close to a Depression we came in 1999, and the only real side effect of it was the collapse of the Internet Bubble by 2000. Today, whether the banks and/or government desire it or not, the financial system has become considerably more transparent to people with tools to be able to analyse the problems and cut through the soothing words, and the effect of that is that this solution - diverting funding into the banks and giving them time to heal - isn't going to happen.

What makes matters worse is that this is now a global crisis. A couple of weeks ago Dubai came perilously close to defaulting on its sovereign debt, saved only at the last minute by both its brother state Abu Dhabi and by concerted effort by European central bankers who were exposed to the debt. Unfortunately, this hit a Europe already weakened by the US collapse, and now the second tier of the European Union - Greece, the Balkans, the Baltic States, Ireland, Iceland, Spain, perhaps even France and (non-EU member) England - are even more vulnerable to shocks as their own financial situations unravel.

Similarly, in the US (and Canada), a number of states are either bankrupt or teetering on the edge, not least of which being the largest state of California. Gov. Arnold Schwarzenegger was able to broker a deal to keep the state out of the red and pay its IOUs from earlier in the year, but its finances are still precarious, especially as larger municipalities each are forced into bankruptcy as well. While there hasn't been significant capital flight out of California yet, it is meaning rising business and personal taxes even as incomes and gross revenue continues to fall.

All of this will likely come to a head in 2010, especially as many (most) of the problems that need to be resolved have essentially only been pushed forward a year. Credit availability remains limited and expensive, and it is very likely that even with the overnight rate at 0%, the actual cost of getting a loan, even for those organizations with perfect credit ratings, will increase dramatically. Yet if the Federal Reserve does increase the overnight funds rate to anything approaching its "real" market-driven value, the effect on the banks will be immediate and devastating. I personally suspect that it will become necessary at some point in the next twelve months, with the results that at least one of the big five megabanks (my bet is on Citigroup, though I'd say Bank of America is also vulnerable) will no longer exist by the end of 2010. (I also expect that political pressures on Goldman Sachs will reach such a fever pitch that they may very well be tied up in litigation from states and municipalities, if not the US Attorney General, by year's end).

My call (based largely upon financial analysts that have called it pretty much correctly from about 2008 onward) is that most of the equity market gains of 2009 will be wiped out in 2010, first as retail markets collapse after this disaster of a Christmas season, then as one after another government stimulus measures are forced to end due to the growing political outcry against the banks. This unfortunately will tend to have some impact upon IT, though I'm suspecting it won't be as dramatic as the last year - too many companies have already trimmed to the bone and have not been rehiring in significant numbers (though unemployment increases have slowed somewhat) and in many cases those companies are now in a position that if they don't start hiring some IT people, their existing IT infrastructure will likely decay beyond a point of usability.

Ironically, while not necessarily good for people invested in equity markets, a significant decline in the market (I'd say we'll test S&P 500 before we test S&P 1200) would actually be beneficial to the market longer term. It will end up forcing out a lot of malinvestment in the financial services, will likely spell the death knell of the c. 1950s era corporation and will instead replace these with smaller, tightly run companies that are far more agile and flexible in the marketplace of 2010. Yes, a lot of good companies will also be shaken out - but what's happening now is a refactoring of the economy in the face of broader demographic and energy consumption issues (see #2). Thus, while the broader unemployment will likely continue to climb through much of 2010, I think that towards the end of 2010 the survivors will be hiring , lots of new startups will be seeking out new economic niches (and there will be new niches as the rubble settles), and especially in technical arenas, employment will be climbing again.

Additionally, I think that you will start to see significant uptick of more important Federal work in the US. While approval for stimulus programs went through early in Obama's first year, it takes time for proposals to be developed, for programs to be chosen and for organizations to ramp up on production - this means that a lot of the science and technology funding in particular will end up going towards new jobs in 2010, and this will also tend to be a boon to IT, though it will have the unintended side effect of removing skilled IT people from the private work force for a while. This could prove to be a problem by 2011, but it should have the added effect of pushing up IT salaries, which have been declining in the last couple of years.

2. Peak Oil

The issues surrounding Peak Oil are now reaching the mainstream, even as there has been something of a temporary respite from the high oil prices of 2007. While there are several mediating issues, increasingly people are beginning to understand that the credit crisis had its ultimate origins with a peaking of oil production globally in 2006-7. To understand this, its necessary to understand the relationship between oil and credit, especially with regards to US credit markets. In general, while innovative ideas can push the development of technologies, in order to build the infrastructure to even foster those ideas in the first place you need to have investments in education, in research institutions (or corporate research wings), in production facilities, distribution and reclamation. All of these require energy.

During the early part of the the 20th century (and even much of the late 19th century) that energy came from internal energy resources - oil production, coal production, later natural gas and to a certain extent nuclear, especially in Europe. Producing it internally and then exporting it provided the relevant capital to fund the infrastructure, which in turn led to better mechanisms for extracting that energy. In the United States this held true until 1971, at which point the growing infrastructure ended up requiring more energy than the US could produce internally, and it had to start importing oil. Significantly, the 1970s also were the point of the first oil shocks, the transition point during which countries began to nationalize their own oil production rather than letting proprietary corporations handle those resources, and marked the high point in inflation adjusted dollars for salaries - it also meant that more money was leaving the US for energy purchase than was coming into the US for energy production, money that wasn't going towards infrastructure development, social programs, educational investment or similar arenas.

This has resulted in a generally falling standard of living for households and diminishing markets for businesses, only somewhat abated by the rise in technology that increased efficiencies in production and elsewhere. While these efficiency gains are real, they have served only to slow the decline somewhat, and there is a case of diminishing returns on technology investment that's actually having a somewhat corrosive effect on certain industries (such as publishing).

By 2007, the financial markets were using increasingly arcane (and risky) mechanisms in order to keep up the illusion that growth was occurring, largely to justify their own existence (and what are increasingly seen as outsized fees for services rendered). A credit is a promissory note that is drawn upon the revenues derived from a (future) growing economy. Yet the US economy has being growing slower and slower in real terms since the 1970s, and it had a significant advantage compared to most other countries in that it held the reserve currency that oil purchases were dominated in. This meant that, by proxy, the US was still able to make money by being the banker for oil producing countries, so that in effect its economy still acted as if it was a net exporter of oil, save that global oil production has also failed to keep up with demand.

Meanwhile, actual oil production globally has topped out at about 85 million barrels a day. This figure seems to be an upper limit, and while oil consumption dropped some after the market collapsed, we've now entered a window where prices for oil tend to rise dramatically the closer that limit is reached. This means that ultimately there is a limit to the actual amount of available "hard credit" available as an input, that despite the finagling of central banks, there is only a certain amount of money that can be created in the system before either destabilizing inflation or economic crisis sets in. Unfortunately, that amount was exceeded dramatically in the 2000s, and in essence the credit "fields" have become exhausted.

Additionally most of the large oil fields that were discovered in the last hundred years are now reaching critical thresholds where the oil that's available in them is becoming harder to get access to. A good analogy here is a sponge. If you saturate a sponge, it practically drips with water, and it takes very little energy to extract say the first 50% of that water. However, you need to expend roughly double the energy to extract 50% of the remainder, and double that to extract 50% of that. This means that the costs (and corresponding technological requirements) to extract more than maybe 75% of a given field become prohibitive over time, and there will be some oil that will simply not be extractable without it reaching a stage where the energy inputs required far exceed the energy output. Put another way, at some point, the energy expended to get to the oil exceeds the evergy value of the oil in the first place.

There is an additional factor here, and that is this is a purely extractive upper limit. There is also an economic upper limit that has to take into account the geopolitical costs of developing those oil resources in the first place. Countries that recognize that their oil resources are limited will begin to become more restrictive in terms of how much that oil sells for (and in some cases may seek not to sell that oil at all, recognizing the need to keep it for their own internal use).

A bit of a side note. While there were a lot of reasons for the Iraq war (many questionable) one of the main ones was to insure that Iraq remained a significant energy provider. Another was to provide a strategic Middle East base that could be used to defend Saudi Arabia (or possibly take it over if it fell during a coup). Still another was to insure that neither Russia nor China could easily access the Middle East oil regions. It's noteworthy that none of these really had any relevance to whether Saddam Hussein was truly a bad leader, save that Hussein was also increasingly pulling away from the US in terms of his own oil production and was beginning to make deals with other countries, including both Russia and China. This is why, demands from anti-war groups to the contrary, the US will not be leaving Iraq completely any time soon, precisely because from an energy security standpoint, Iraq was a key factor in any strategy.

Long term (and unfortunately this means over the next decade) oil resource depletion is going to accelerate. Canterall in Mexico is declining fast, and Mexico will cease being an oil provider altogether by 2012, Venezualen oil production is dropping, and even the relatively recent discoveries of Bakken shale and similar reserves in the US will only contribute a small amount overall (about 400,000 barrels of the 28 million used daily in the US) to the total production. In the immediate term oil prices have remained high. Indeed, if you factor in the evaporation of the $12-14 trillion that has occurred in the last year, the actual value of oil today is the equivalent of about $132 a barrel in 2007, well above the level at which oil prices become corrosive to business.

The immediate relevance to IT beyond the fact that credit resources are no longer going to be there is that alternate energy will continue to be a major focus for investment, and thus for IT jobs. These won't begin to make up for the likely continuing decline in energy availability, but it will begin to drive state and regional efforts in particular to become as energy independent as possible by the end of the decade. It also means that longer term you'll see a distinction appearing between energy haves and have nots as energy rich areas begin to wean themselves off of a totally petroleum based economy while other areas lack either the financial resources or the energy resources to do so. This is a long term trend, but from my own personal observations, a number of compatriots of mine in the IT sector having been moving into alt-energy, recognizing that this is likely to be a good time to be positioning oneself in what will likely be a long term growth industry in an otherwise shrinking economy.

One other area that bears watching is food production - globally it was a bad year for food production, and this will likely cause both an increase in prices and greater distress among those already hard hit by the economy. This unfortunately is likely to be a growing problem as well, as those same oil inputs factor heavily in agriculture.

3. Distributed (Cloud) Computing and Data As a Service

There is no question that cloud computing and distributed services have combined to become the primary story of 2009. Microsoft, IBM, even Oracle all flogged their cloud computing offerings. Google made the signficant step of producing the first "cloud friendly" operating system with Chrome. Creating stody desktop apps was out, online/offline web-centric applications were in.  Everyone and their immediate siblings were busy cobbling together web apps and most hosting companies have never rebranded themselves as cloud computing platform providers.

Of course, most people in IT (outside of marketing departments) readily acknowleged that the "cloud" itself is mostly vapor, the latest manifestation in the continued virtualization of IT resources. The first aspect of this was the ready creation of distributed storage - the cost, safety, accessibility and convenience of storing resources (documents and media resources mainly) online has begun to become more attractive than storing the same resources locally. This of course also translates directly into the notion of Software as a Service, since most such software ultimately involves working with those same resources through web applications. In some cases these applications are centralized content management systems such as Sharepoint or Drupal, in some cases the applications are mixed systems such as Salesforce (now a part of Oracle) which combine these with tracking and resource integration tools, and in some cases the applications are effectively display share systems such as Scribd.com or Slideshare.com (as well as the older generation Flickr or Youtube offerings).

While resource production can occasionally at least be originated on the client side, increasingly even that is migrating to the web - as a personal example, almost all of the documentation that I originate any more, up to and including manuscripts for books, is produced using Drupal. This is even occuring with non-obvious web applications, such as graphics, sound and movie editors, a process helped along by a growing wave of Netbooks that have at best lightweight resources for editing.

The second stage of this process, Hardware as a Service or HaaS, has also been underway for some time. Hardware virtualization has made it easier for large scale hosting companies to better allocate resources by setting up larger systems that can then be partitioned into virtualized systems. Such systems obviously have the advantage of being easier to set up then physical systems (because you can effectively create a snapshot of a configured system and replicate it) and they can scale both in terms of memory usage and CPU usage quickly and efficiently so that rapid and irregular spikes in Internet traffic in particular can be accomodated dynamically with new server instances being brought online just when they are needed and destroyed when demand drops.

Such virtualization is also making its way to the desktop. I run a couple of different virtual boxes within my Ubuntu Linux laptop, one running Windows 7, the other different flavors of Linux, for testing of software in those environments using the virtualbox application. In essence, because I can shift from one to the other and treat them as if they are simply different applications in their own windows (even to the extent of being able to do such operations as copy/paste and drag/drop between Linux and Windows platforms) the concept of being locked into a specific desktop environment becomes increasingly a non-issue. It's not perfect - some games get twitchy because graphic cards aren't properly recognized - but it is certainly a vast improvement over times past when you often had to maintain dual boots (and risk wiping out whole partitions) just to be able to work with a different operating system.

The next stage of virtualization is just now beginning to gain steam: Data as a Service, or DaaS. Again, to some extent DaaS has been around for some time, but the reality is that the need to be able to work with SQL RDBMs content as an unmediated serialized data stream has been fairly limited. Most web services and service oriented architecture have similarly been focused on application centric development. Recently, however, you're seeing the next stage in the syndication process as people begin using syndication feeds and other linked data "messages" as mechanisms for working with database content directly, via a  process called RESTful Services.

RESTful services treats the Internet as a database and treats individual resources as the equivalent of data records. By making this transition to the document as data, one consequence is that applications can reduce their overall API dependencies dramatically, which in turn makes for significantly reduced siloization and makes for simpler applications as a consequence.

For the XML community, DaaS has the potential to either make XML usage ubiquitous or reduce it to obscurity. The signs at this point to the former - as a messaging format, JSON is perhaps more compact and friendly with regard to client-side development, but XML is able to transmit richer semantics, is able to better encode relational links and has less potential for malicious side effects. Moreover, XQuery and XML Databases have both reached just about an ideal point of completion to fit into this mode, though DaaS is also subsuming a lot of the no-SQL type data repositories - LDAP, Name/Value pairs with CouchDB and Erlang, as well as the on-the-horizon technologies of RDF triple stores.

I don't think that RDF by itself is going to have a banner year next year. Even with concepts such as Linked Data, and the new OWL 2 classes, the Semantic Web technology space is still rather embryonic, it's use cases haven't fully developed all that much beyond specialized publishing applications, and the benefit that SW has to offer - especially the ability to create self-assembling schematic structures - is something that will still require some form of killer app to make use of this. That's not to say that I don't think SemWeb won't reach critical mass ... as DaaS becomes more ingrained, I think Linked Data is actually a natural extension of this ... but I think that SemWeb won't really hit it's inflection point until 2013-2014. For now, most discussions of SemWeb are focused around closed box Semantic applications such as document enrichment, which may use closed RDF triple stores to perform the encoding of relevant names, places and events but that don't explicitly require that the consumer of this information even be aware that SemWeb tech was behind the encoding.

One other prediction regarding clouds - the moniker itself is already getting stale, and I suspect that by the end of the year "Cloud Computing" will long since have been retired as marketers look for some other sexy new term to describe an ongoing process. As the ongoing recession continues, it's also likely to start impacting grid reliability as cash strapped municipalities begin to draw back on maintenance and repair of power grids and infrastructure. This in turn will mean that we'll likely start seeing holes in the grid both as ISPs (the backbone of the cloud) continue to fail and as outages periodically knock out access. This will impact the selling point of Cloud as ubiquitous computing and likel slow adoption in general somewhat.

4. Mobile and Gaming

Something rather interesting happened in the mobile space fairly recently - while NetBooks started out strong at the beginning of the year, they've lagged considerably toward the end of the year in favor of smartphones. Apple, having effectively defined the smartphone field, is now facing a number of competitors that are seeing such factors as price points and wider networks as being potential competitive, including the introduction of Google in this space with the release of Android phones. If Netbook sales are beginning to evaporate in the face of only slightly less (and in some cases slightly more) powerful handhelds on one end and general notebooks on the other, it would indicate that we're moving to a mode where the handset form factor is becoming more attractive than the traditional querty desktop, at least for basic web connectivity devices.

If this is in fact the case, this is a very important transition. Already the mobile gaming industry is exploding, even in the face of a severe recession, and larger companies are sitting up and taking notice. Gaming tends to be a harbinger of other technology adoption, and with the increasing dominance of services-centric applications the mobile space represents a near ideal platform - works well with the expectations of lightweight interfaces (and prefers web apps to traditional heavy desktop apps), can effectively be scaled to work with less resource intensive modes, can work well with vector graphics (with SVG now becoming pretty much the primary vector format on many such systems, compared to Flash's near stranglehold in the Desktop arena) and far more suited to the kind of resource-oriented development that characterizes DaaS-like applications.

I'm going to go out on a limb here and make a prediction that the next major piece of the mobile solution will be resolved this year with the creation of relatively low cost (under $500) HUD-type glasses that would allow both game play and mobile connectivity without the need of having the screen housed in the same unit as the CPU. Couple this with USB based mem-sticks in the 16 GB+ range that can plug into these units and we should see situations where gaming systems and mobile devices essentially merge. My gut feeling is this is probably two years out for wide-spread adoption, but you'll begin to see it begin to take off in 2010.

On the gaming front, I think you're also going to see continued growth in the number of people that "go virtual" - essentially turning their online gaming activity or remote programming into their primary revenue stream while dropping out of the contemporary workforce. There will likely be two such groups of people - those that effectively become physically isolated, seldom leaving their house because their access to work or game resources (which may be the same thing) is accessible from their home network, or those that become wandering nomads, living in their cars, moving from city to city to take advantage of open wi-fi hotspots. This latter group may actually start defying the typical "homeless" label in that, without the overhead of maintaining a house or apartment, they can actually save quite a bit of money from doing software projects, designing custom online skins and objects, managing or brokering projects and so forth. Expect to see more "mobile conferences" as well, where tech nomads converge on a particular city for a week to network and socialize.

The latest generation Fords are also now integrating wi-fi access directly into their models, either via satellite connections or working in conjunction with wireless providers. While the mainstream appeal will likely be a way to tap into the online music universe, the attraction that these cars have to tech nomads should be obvious as well. These features should start to migrate downward into the midsized models by 2012. With the ongoing collapse of GM, you're also seeing the rise of startup car companies, some of whom are well aware of this move toward car as Internet enabled home, and it would not surprise me in the least to see specialized "mobile apartments" being marketed by some enterprising entrepreneur.

5. Programming and XML Trends

While specialized computing languages will continue to be pushed out to handle increasingly esoteric programming needs, I believe we are past the era where a new language coming out of the blue will become the next big thing - there will be no more new Javas, new C#s, new Rubys. The reason for this is a little complicated, but has to do primarily with scope - we're reaching a stage where there are plenty of well developed languages and language frameworks for handling most current levels of abstraction, and the next major level of abstraction is at the distributed systems level where I believe that DaaS-type declarative applications such as XQuery or Erlang hold sway.

The one major exception I can see to this is server-side JavaScript (SSJS), which I believe is now beginning to see traction from a number of different directions. JavaScript has now matured into a sophisticated language that seems to have found the right balance between ease of use and flexibility, and given that many people are already using JavaScript on web clients, I cannot imagine how SSJS could not get traction. This will only become more pronounced as the move towards distributed data services also gains traction. Most SSJS implementations use either the Mozilla originated Rhino or Spidermonkey JavaScript engines as their core, running in Apache or Java servlet contexts.

A related phenomenon is the integration of web servers directly into browsers, something which Firefox, Chrome and Opera all support (and I believe that Safari also supports - will have to check on that). The primary purpose of such servers is to create localhost-based applications that mirror online apps, so that the same basic interfaces can be used for local data storage and access as are used for remote data storage and access through web applications. Similarly, more applications are moving to a model where they create a local server (usually on an unused port), then provide the UI layer within a web browser. Such services models have been common for database applications in the past, but the use of web servers in this regard, while they have been used to good effect in the past, are likely to become a much more dominant model for application development moving forward.

HTML 5 is likely to play a part in this as well (my own comments about the language not-withstanding). The central worry that I have with the language (beyond wanting to insure that an XHTML5 solution also gets implemented in a browser) is that it will become a balkanized standard, where vendors choose to cherry pick those features of the language that they think are either easiest or offer the greatest market appeal, while other parts of the standard, just as critical, remain unimplemented. Still, expect at least the video portions, canvas and the offline storage APIs to be fairly commonly implemented pretty much universally by the end of the year.

Another trend worth watching is the use of HTML/Javascript based APIs for browser extensions. Mozilla's JetPack provides a good example of this, replacing the somewhat cumbersome process of using XUL components with HTML components in the same context, a move that's also being espoused by Google Chrome. These apps are not quite trivial to write, but they are certainl easier than the older extension models, and as such should open things up to an explosion of new extensions in this space.

I think community extensibility is also the name of the game moving forward. I've been a big fan of Drupal in the past, and still feel strongly for the platform, in that it provides a model of what I see applications turning into. Already I'm seeing Drupal being used as a foundation for applications that once may have been stand-alone apps, simply because by participating in the community process people can take advantage of a large and growing component library. This will become even more of an issue when IT managers are given a choice between purchasing expensive proprietary solutions, writing their own solution (also becoming increasingly expensive) or piggy-backing on top of existing community platforms and simply developing extensions to accomplish those tasks that don't already have existing functionality.

One other factor that will likely increase the use of community software is the growing shift in thinking that the cost of marketing and maintaining most software products has risen above the market price for such software, to the extent that people are now recognizing that while software can be a differentiator in providing services, making money off of in-house developed software is usually more of a pipe dream than not. Without that profit motive, software development is increasingly shifting to the use of software to generate goodwill, thus reinforcing the community software development paradigm.

Ironically, I see PHP and Python being the primary beneficiaries of this movement, with Ruby in particular becoming more of a niche language. While Ruby and Rails tend to provide better languages for development than either Python or PHP, there are fewer large scale Ruby community projects out there, and it may very well be that Ruby is likely to be starved for available niches in the mid-to-long term.

On the XML front, I see continued adoption of XQuery as a server language, as both Mark Logic and eXist are now both effectively calling the shots in this regard as far as cutting edge development. I think the XRX model that Dan McCreary has championed for a while now will continue to make inroads in that regard, though I also think (hope, in fact) that this year will see the rise of larger scale open CMS type systems built around XQuery and XSLT as a core. XML development overall continues to be a fairly small, focused discipline, compared to the rest of the software development base, but especially for organizations dealing with large amounts of data that the need to make public (such as government agencies), XML pipelines and XQuery/REST based systems continue to gain adherents.

I expect to see both the National Information Exchange Model (NIEM) and eXtensible Business Reporting Language (XBRL) to migrate to such query/manipulation environments heavily this year, with other open (and proprietary) XML super-collections making their way over the next two to three years to an XQuery platform. I also would not be surprised to see Microsoft make the jump to XQuery 1 with the soon to be published XQUF update framework standard by year's end, at least at the development stage, and perhaps the creation (or purchase) of a stand-alone XML Database platform to complement SQL Server.

Some of this may come, unfortunately, at the expense of XSLT 2.0 adoption, which is a pity as I see the two being complementary rather than competitive technologies. While there are more XSLT 2.0 transformation engines out there than a couple of ears ago, Saxonica's Saxon 9.x processor is still pretty much the gold standard there, and I've seen very little in the works that indicate that this will change.

I do think that a methodology for developing map/reduce type applications for XQuery represents perhaps the biggest challenge to the XML application space in the next couple of years. The core tools exist to do this, but at the moment these are proprietary with a hodge podge of conflicting APIs. Standardization of distributed queries could go a long way towards causing the explosion in adotion of XQuery based systems.

I also suspect that you'll start to see XQuery on the client as extensions in the next year or so.  XQIB (for XQuer in the Browser) showcases some of the potential for XQuery in this space (though it is very unstable in my own tests). XQuery will likely never be native to browsers, but even getting it in as an extension opens up some interesting avenues for web development. Until recently it was for IE only, but the XQIB Firefox extension was made available with XQIB 0.6.

On the flip-side, I think that SVG in the browser is beginning to really happen now, especially with Google's SVGWeb project creating an SVG component written in Flash of all things. Ironically, this means that we may begin to see rising adoption in SVG that was effectively halted when Adobe and Macromedia merged in 2007 and killed the Adobe SVG Viewer. Moreover, with a broader base to begin with (most browsers, with the exception of Internet Explorer) now support an SVG layer) technolog adoption isn't kept hostage to a single product or company, which diminishes the likelihood of another mortal blow to the movement.

XForms, as predicted last year, continues to grow in adoption, though it's been the surprising appearance of XSLTForms that has revived hopes for a client-side XForms solution. Both Mark Logic and eXist now support XSLTForms as their primary engine (though eXist also continues to be distributed with the superb server-side Orbeon XForms engine as well) and the component is gaining the attention of both tech companies and big data users. Ubiquity, a joint project between IBM and Mark Birbeck's BackPlane, is also approaching a good point of solidity, and it is likely that it will soon make its way into larger scale enterprise IT projects.

6. Industry

This has been a hard year for the IT industry, both at the enterprise level and at the SME level. Google is effectively in a hiring lock-down (though this may have eased somewhat), and IBM and Microsoft both shed several thousand workers each, especially around March of last year.

Hardest hit, however, has been Sun Microsystems. Sun was originally in talks with IBM to purchase the ailing server company and home of Java, but those talks soured. Oracle came in as a second prospective buyer, but significant concerns about the fate of mySQL, especially in the Eurozone, have made regulators there very unhappy, and it's becoming increasingly likely that Oracle too will end up passing up on Sun. Meanwhile, the company has been cutting staff, including many of its biggest marquee names, trying to get costs under control, yet with each passing day the company is continuing to hemorhage money. Antitrust concerns make it unlikely that another big name will come forward to purchase them, and in the current economic climate there is little appetite (or even capability) for leveraged buyouts of ailing companies, especially when there are so many other potential takeover targets that offer strategic benefits.

Perhaps in the long run Sun will end up making the final step of becoming a completely virtual open source project - it's assets maintained by a  pool of existing Sun customers, Java and mySQL (among other properties) reverting back to pure FOSS plays. Indeed, the future of Sun should be watched carefully, as it may be indicative of the fates of other companies in the "New Normal" era.

Another company that's still reeling is Yahoo, though it was thrown a major lifeline by Microsoft earlier this year after Jerry Yang stepped aside as CEO. Yahoo's agreement to become a major Microsoft customer within their server farms and search properties has slowed the exodus of people, although it's search properties have slipped down to become a decent third to Google and Bing. It's likely that over time Yahoo will become ever more absorbed into Microsoft, but with the Microsoft support, it also likely won't be in for the market gyrations that hit it in 2009 and 2009.

AOL completed its divorce from Time Warner this year (gads, I feel like a gossip columnist writing this), which may actually prove to be a major boon to the once dominant online community. Under new management with more control over its own fate than its had since it was tethered to Time Warner in 2000, AOL may be in a position to continue shaping its advertising and media properties into a new platform. On the other hand, it may also just fade away, though without its fairly outsized revenues in recent years going to the Time Warner mothership, there may be more money to invest back into R&D. The challenge for AOL will be for the company to develop a core specialization again, rather than just chasing fads (travel, media, search, etc.) that has typified that last several years.

IBM and Oracle are both going quiescent at this point. After having shed a considerable percentage of their workforce in response to significantly declining demand for consulting services, both companies will likely pull back in at least in the US, though IBM came under fire recently for announcing that there were expanding operations significantly in China. This is a trend to watch as well - after decades of outsourcing to China, there's now a fairly burgeoning market there for IT expertise and IT infrastructure development while the US market for the same services is largely mature and somewhat stagnant. This is also being accelerated by increasingly stringent H1-B visa restrictions in the US, which is making importing Chinese and Indian workers into the US more expensive. Since the markets are growing and the talent pool in China, after years of servicing US contracts, is now getting quite large, it makes sense economically if not politically for companies to be expanding in Asian markets, though this comes with the potential for backlashes in the US.

Microsoft is getting ... interesting again. After the disaster of Vista, Microsoft is now getting good word of mouth for Windows 7, which is quietly becoming the next XP for the company. While still wrestling with a somewhat convoluted definition of open source, the company is beginning to "get it" in this space, and lately has been fairly instrumental in pushing out community and FOSS initiatives that have even die-hard anti-Microsoft types giving the company a look again. While Azure, their cloud platform, has not exactly taken off, it's also garnering some favorable reviews, and the recent patent suit troubles with i4i in Montreal, Canada may have actually helped to spotlight what they are in fact doing with XML in that space.

The lawsuit itself has recently resulted in an injunction by an appeals court to halt the sale of Microsoft Word, but given that this is still very much Microsoft's cash cow, the final result will be that Microsoft will likely choose to settle out of court with i4i rather than let its crown jewels be held hostage. The other likely upshot is that ODF will end up becoming the defacto Word XML standard, as this capability is not directly impacted by the i4i lawsuit and it will take time to relicense and prep the core OOXML code - which means that the liklihood of some unification of ODF and OOXML has gone up considerably.

Microsoft should have a good year next year - in terms of its innovations and the likely effects of ongoing changes in the company, if not necessarily to its bottom line. The economy is still going to keep sales depressed, but with Bing use increasing and with a fairly stable cash flow situation, Microsoft will likely whether the storm better than most.

And then there's Google. If Microsoft's future looks brighter, Google's is getting cloudy. The advertising market continues to drop, decreasing its core revenue stream, and with Bing Microsoft is facing a real challenge in the search space after having several years of pretty near total dominance. While the growth in the Chrome browser market has been dramatic (from it's release in late 2008 it has now jumped to 7% of the total browsers in use, well ahead of Safari and Opera) the Google Chrome OS has not exactly caught on fire yet (though its still early) and Google Wave has suffered from poor service, a slow roll out, and a still evolving API, not to mention disappointment by many that it's not quite the game changer that many people had hoped (though as a technology it's still likely to take off down the road).

Google's also attracting increasing antitrust notice from both the US and European governments, which has the potential to turn into one or more lawsuits at some stage. Already it had to enter into a royalty sharing agreement with authors and publishers with its Google Books service. It's also facing increasing contentions with its GoogleVoice application (formerly Grand Central), especially since it is also developing the open source Android phone. Relations between Google and Apple in particular have been frosty of late, expect them to become downright frigid by the end of 2010.

This doesn't meant that Google will face an especially bad year. Most of these problems are ones that have emerged in variant forms for most companies that end up become dominant players in a sector, and are indicative primarily that these companies are now generating significant resistance to further growth from others in their respective spaces - and likely diminishing returns on stock valuations that reflect that slowing growth.

On a similar note, Amazon, which has actually had a remarkably good year this year in comparison to others, if not necessarily in total profits, is likely to experience some headwinds next year. While online retail sales did better than in-store sales on a percentage basis this year from early Christmas sales indications, both were abysmal by standard reckoning. On the other hand, by at least some measures, Amazon may have managed to hit a critical breakthrough this year - more people purchased eBooks over Amazon than purchased physical books over Amazon. While this may not necessarily translate to the broader market, it is nonetheless a milestone achievement that indicates the eBooks have become maintream, especially given that Amazon typically discounts physical books fairly dramatically, even with shipping costs. A significant portion of these eBook sales went to the Kindle, although the figure also likely included PDF versions of books.

The Kindle also has had an advantage this year of effectively having the market to itself - a hyped Apple ebook reader has yet to materialize (though rumors are that the iSlate will likely be the entrant in 2010) and the Sony Reader has made at best indifferent penetration in the US market, though it does well in Japan. There's a delicate balance going on right now in this space - while the electronic paper component of such readers offers a lower price-point than passive LCD screens, such readers are still expensive enough to compete with netbooks, which are more general purpose devices. They are also competing with tablet computers, though this space tends to have relatively poor sales because they are more costly than laptops and are somewhat awkward to use, especially with a keyboard. Late 2010 may see the introduction of color e-Readers as well, though my gut feeling is that such eReaders probably won't see the light of day until mid-2011 at the earliest - the technology is known, but whether it can be made cost effective has not yet been established.

Amazon's other major foray - it's cloud computing platform - is also now under attack as competitors eager to get into this space establish clouds of their own, and organizations such as NASA create cloud platforms for dedicated IT purposes. In all likelihood, Amazon may be in a position to effectively establish cloud virtualization standards this year based upon its own EC2 API, but if it waits too long, it's more likely that the resulting standards will be more vendor neutral and consortium driven.

I don't normally follow Apple all that closely, so these comments should be taken with a grain of salt. However, from my limited vantage point, Apple is clearly searching now for a new category space with the iSlate and iGuide - eReader and tablet computer formats respectively. The iPods are now long in the tooth and are now facing pressure from commodity producers of electronics, the iPhone is facing an increasingly crowded market as LTTE and 3G mobile handset vendors strike back with their own offerings, and the aluminum clad MacBook, while capitalizing on the "green" market, is now overpriced in a cost-conscious market, although I find it interesting that a number of my colleagues (primarily those at the editorial or management levels) have made the switch to MacBooks from Windows systems. If the iSlate, rumored to be the target for a major buy of 10" screens by Apple, is in fact their answer to both eReaders and netBooks, then it may very well be that Apple's 2010 strategy is to round out its economy level computing platform.

Finally, it's worth looking at the four "Socials" of 2009 - Twitter, Facebook, Linked In and Google Wave. Twitter was the hot media of 2009, but it's very likely that this represents the high point for the service. It is a concept that is relatively easily replicated (and Twitter like features are becoming de rigour for other services), it has had a frequent track record of slowdowns and service interruptions, and it's reached a sufficient volume that it is becoming increasingly difficult to separate signal from noise. While I don't see Twitter disappearing in 2010, I do see it becoming less relevant to the broader fabric of the Internet.

Facebook continues to surprise me. At this stage, I think that the only real competitor that Facebook really needs to worry about is Linked-In, which likely will continue to act as a brake to Facebook getting broader adoption in the business sector, and Google Wave (more of which below). This means that in 2010 it has a certain latitude to experiment and fail without it being strategically costly. However, it's still going to face revenue challanges given the overall advertising dearth, and will likely have to continue to strike a careful balance between deploying revenue generators and keeping its audience happy.

Linked-In on the other hand is well positioned moving forward, if it can get past the first half of 2010. The economy will probably take another hit between now and then as the need to inject stimulus begins to run aground of political and budgetary pressures, which means that the slowing unemployment rate may very well jump again; this will throw more people out of work, which may in turn put added pressures on Linked-In from people looking for work but not finding it. My belief though is that we'll actually start seeing a true (albeit anemic) recovery begin after summer 2010, which means that dedicated networks will definitely beat out want ads as a mechanism for finding qualified people.

Google Wave is still in early alpha, and it will take much of 2010 to mature. My experience with it was that it was an intriguing approach to messaging, one that could, especially with search added in, provide a serious challenge on the social networking side to Facebook my mid-2011. The challenge will be in optimization - the more optimal the code, the more it will be used, which will in turn degrade performance back to suboptimal levels. It's not as "fun" as Facebook is, but that's not necessarily a detriment - Facebook's interfaces do not lend themselves well towards business communication and collaboration, while I can definitely see Wave doing so. Thus, overall I expect the social space to remain fairly quiet through much 2010 but to start heating up again in 2011.

6. Publishing

2009 will be remembered as the year that traditional publishing died. The start of the year saw the demise of one newspaper after another as advertising evaporated. Book publishers slashed their in-queue publishing lists, and the title "Editor" became synonymous with "Unemployed". Journalists, already battered by declining ad revenues and increasing news competition from both Google search and legions of "amateur" bloggers, watched as their careers went up in smoke. Through the year the damage only deepened - to the extent that A-List authors decided it was better to start working on e-books and self-published efforts rather than watching their publishers disappear. By year's end, as reported above, e-books were outselling print books on Amazon, and anyone involved in the writing arts was searching desperately for work.

This is very likely a permanent sea change. For newspaper publishers, the problems started decades ago, as deregulated media rules made it possible for companies to buy up newspapers and divert much of their incoming ad revenue stream to dividend payments, squeezing newsrooms and forcing an increased reliance upon canned syndicated services. The rise of the Internet provided a new publishing platform, but it also largely democratized the publishing process, such that a single person running CMS web software could effectively compete with news organizations with eight and nine figure revenues. The shift away from subscription based revenue streams to advertising meant that the newspapers and magazines made more money when the economy was strong, but made them vulnerable when the economy weakened. The final coup de grace came with the economic collapse and the subsequent credit crunch which shut down rolling lines of credit precisely when these weakened companies were at their most vulnerable. Magazines are similarly vulnerable, though a rise towards micro-marketing means that such publications are perhaps more relevant if less timely.

2010 will see the consolidation of this trend. Most news organizations are producing both for print and for Internet production, which in turn equates to more sophisticated CMS systems being in place to handle the dual output. Surviving newspapers will likely become more expensive and smaller, and will likely end up becoming more magazine like. Expect also to see more print articles in such publications come with tiny URLs that will in turn point to web versions of the article along with relevant links to related content.

On the flip side, 2010-2011 will see the rise of the blogger as a paid profession. Quality still sells, and in general bloggers who are willing to marry their own brand with a publisher and who bring in thousands of viewers will be rewarded appropriately. However, because there is a readily visible metric for determining popularity for given writers, it also means that the publishing field is going to shift to a popularity model - a journalist's compensation is directly related to his or her ability to sell to the masses, even for hard news stories.

Bloggers and writers also are beginning to develop their own newsletter subscriptions, either free or paid, depending upon the publisher's inclination. Subscribers would then get newsletter content that doesn't always appear on the web - story chapters, artwork, investing advice, analysis or other secured content. This works best for original content that typically has little opportunity to be relicated elsewhere (which is why, as a model, it doesn't work all that well for news organizations in and of themselves). Such publications are also likely at the forefront of a boom in analysis firms, organizations that provide in-depth research on specific topics, though they may, with some adaptation also be suitable for creative organizations.

None of it will likely pay well immediately, though the more exclusive the content and the more useful or entertaining it is, the higher the effective returns. Merit - quality, reliability, timeliness, entertainment value - will drive this adoption, and will define more clearly the next phase of journalism and publishing.

7. Summary

In many respects, 2010 will likely be a time of coming to terms with the New Normal. People's expectations that we can return to the glory days of 2005-6 (or even the late 1990s) will be dashed, repeatedly. Some large banks will fail, people will lose faith in financial investors and start scaling back their expectations for return on investment. Lives will simplify for many, and become desperate for all too many. However, people will also begin to start planning ahead based on these new expectations, will find ways of working with limited capital, will scale back their expenditures, and will become more conservative in their planning, but will plan. The results will typically be more solid and well run companies, will be greater expectations of performance and likely will be a shift in the way that people get paid for a living. You'll see more people that forsake permanent fixed homes altogether, far more people that will work remotely because the cost involved in moving is not worth the pay. Society will adapt, and will rebuild from that.

Kurt Cagle is an editor, industry analyst and software architect living in Victoria, BC. He's the managing editor for XMLToday.org.