In the traditional sense, “Big Data” has been used to describe massive amounts of data controlled and analyzed by huge organizations like Google.  Below that echelon of organizations, “Big Data” is a relative term, proportionate to the size of an organization.  Regardless of size, sector, or vertical, the exponential growth of data and focus on data analytics has prompted companies to adopt Hadoop – an open source implementation of MapReduce – at an equally rapid pace to meet the needs of the business.  The overarching theme of “Big Data” is how to uncover new and valuable information from unstructured data sets, and turning that into a competitive advantage for the business.

The parallel processing technology of Hadoop makes it possible for these massive data mining and analytics tasks to be broken up into smaller jobs, which are then run over a cluster of commodity servers – keeping costs low, while scaling economically.  However, despite the increase in raw processing power and I/O performance of today’s commodity servers, the performance of these clusters is limited by the bandwidth of their network. 10GbE networks featuring Emulex adapters improve the hardware utilization rates of Hadoop clusters and allow massive scalability without encountering the network bottlenecks of traditional 1GbE.

As detailed in the Emulex White Paper, benefits of migrating from a 1GbE to a 10GbE network using an Emulex OneConnect OCe11102 include 4x data transfer rates.