In the traditional sense, “Big Data” has been used to describe massive amounts of data controlled and analyzed by huge organizations like Google. Below that echelon of organizations, “Big Data” is a relative term, proportionate to the size of an organization. Regardless of size, sector, or vertical, the exponential growth of data and focus on data analytics has prompted companies to adopt Hadoop – an open source implementation of MapReduce – at an equally rapid pace to meet the needs of the business. The overarching theme of “Big Data” is how to uncover new and valuable information from unstructured data sets, and turning that into a competitive advantage for the business.
The parallel processing technology of Hadoop makes it possible for these massive data mining and analytics tasks to be broken up into smaller jobs, which are then run over a cluster of commodity servers – keeping costs low, while scaling economically. However, despite the increase in raw processing power and I/O performance of today’s commodity servers, the performance of these clusters is limited by the bandwidth of their network. 10GbE networks featuring Emulex adapters improve the hardware utilization rates of Hadoop clusters and allow massive scalability without encountering the network bottlenecks of traditional 1GbE.
As detailed in the Emulex White Paper, benefits of migrating from a 1GbE to a 10GbE network using an Emulex OneConnect OCe11102 include 4x data transfer rates.
The following table summarizes some of the findings found in the White Paper:
With increased network throughput and the Emulex 10GbE OCe11102, data ingestion and data replication take less time. As a result, more CPU cycles become available for data analysis, improving system response time and reducing the required number of Hadoop cluster nodes. Cluster size reduction and increased transaction response time result in significant cost reductions in implementing, operating, and maintaining a Hadoop cluster. Download the full White Paper for the detailed report on how to increase Hadoop performance.
Articles and Emulex Blogs
Latest Hadoop Blogs