As the Big Data cascade starts to drown out other topics in 2012, it seems there will be an especial focus on WANs. Indeed, "DC2DC" (Datacenter-to-Datacenter) connectivity has been recognized for many years as a potential bottleneck and headache.

With Big Data driving "Big Traffic" - a term mentioned by the Internet Research Group (IRG) in a recent report - around the horn of a cloudy datacenter complex, the issue of WAN Optimization will be more prominent on roadmaps in 2012 and beyond.

The IRG recently completed its new report for Infineta Systems, one of a few WAN Optimization specialists seeking to find its place in cloud discussions and IT budgets. The use of Hadoop to scale up (unstructured) Big Data applications has put the WAN bottleneck issue in clear relief.

Through the Hadoop Distributed File System (HDFS), "Hadoop offers as much as a 30-to-one reduction in the cost of storage due to its use of commodity disk drives rather than conventional enterprise storage systems," according to the IRG report. "The confluence of (this) low-cost storage coupled with an affordable scalable execution environment meant lift-off for Big Data," the report states.

As companies employ data analytics to get a grasp of the value in their new-found data treasure troves, they often find significant "data isolation," the report notes, cases in which a company needs to analyze data from several sources in a way that was not planned when the original applications were set up.

The Old Way, Sped Up
Hadoop jobs are batch jobs (à la recherche du temps perdu), which brings a non-realtime element to analyzing disparate data. But this is the 21^st century, and hundreds of Hadoop jobs can be scheduled daily on today's systems, lasting from a few seconds to several hours in length. WAN bottlenecks not only impede the ability to do this, but simply "may not have enough throughput capacity to support these tasks.," the report states.

Infineta joins a group of mostly Silicon Valley-based companies, including Riverbed Technology, Blue Coat Systems, Exinda, Silver Peak Systems, and of course Juniper Networks, Citrix Systems, Cisco Systems, and all of the big end-to-end enterprise players in being aware of and attempting to address this problem.

As the new year dawns, I'll be interviewing as many people from these companies as I can to get my own grasp of the situation, which is going to be a major theme in all Big Data and cloud computing discussions.

For now, I'm heading deep into the provinces of Southeast Asia for a few days to see in the New Year. Mangos and rambutan will be on the menu as I contemplate the tropical blue sky and its variety of real clouds.

Follow me on Twitter