A number of industries rely on high-performance computing (HPC) clusters to process massive amounts of data. As these same organizations explore the value of Big Data analytics based on Hadoop, they are realizing the value of converging Hadoop and HPC onto the same cluster rather than scaling out an entirely new Hadoop infrastructure.
Enabling Value for Converged Commercial HPC and Big Data Infrastructures through Lustre*
September 15, 2015 by
Podcast: New Xeons Power Cisco UCS Realtime Analytics
May 5, 2015 by
Jim McHugh from Cisco describes how the new Intel Xeon processor E7 v3 processor family will bring to Cisco UCS systems in the big data and analytics arena. He emphasizes how new insights driven by big-data can help businesses become intelligence-driven to create a perpetual and renewable competitive edge within their field.
The Analytics Frontier of the Hadoop Eco-System
October 15, 2014 by
“The Hadoop MapReduce framework grew out of an effort to make it easy to express and parallelize simple computations that were routinely performed at Google. It wasn’t long before libraries, like Apache Mahout, were developed to enable matrix factorization, clustering, regression, and other more complex analyses on Hadoop. Now, many of these libraries and their workloads are migrating to Apache Spark because it supports a wider class of applications than MapReduce and is more appropriate for iterative algorithms, interactive processing, and streaming applications.”
Interview: Replacing HDFS with Lustre for Maximum Performance
October 2, 2014 by
“When organizations operate both Lustre and Apache Hadoop within a shared HPC infrastructure, there is a compelling use case for using Lustre as the file system for Hadoop analytics, as well as HPC storage. Intel Enterprise Edition for Lustre includes an Intel-developed adapter which allows users to run MapReduce applications directly on Lustre. This optimizes the performance of MapReduce operations while delivering faster, more scalable, and easier to manage storage.”
Intel Steps up with Enterprise Edition for Lustre Software
October 1, 2014 by
In this video from the 2014 Lustre Administrators and Developers Conference, Brent Gorda from Intel describes how the company is adding enterprise features to the Lustre File System.
Performance Comparison of Intel Enterprise Edition Lustre and HDFS for MapReduce
September 29, 2014 by
In this video from the LAD’14 Lustre Administrators and Developers Conference in Reims, Rekha Singhal from Tata Consultancy Services presents: Performance Comparison of Intel Enterprise Edition Lustre and HDFS for MapReduce Applications.
insideAI News Guide to Big Data Solutions in the Cloud
September 23, 2014 by
For a long time, the industry’s biggest technical challenge was squeezing as many compute cycles as possible out of silicon chips so they could get on with solving the really important, and often gigantic problems in science and engineering faster than was ever thought possible. Now, by clustering computers to work together on problems, scientists are free to consider even larger and more complex real-world problems to compute, and data to analyze.
Attaining High-Performance Scalable Storage
September 3, 2014 by
As compute speed advanced towards its theoretical maximum, the HPC community quickly discovered that the speed of storage devices and the underlying the Network File System (NFS) developed decades ago had not kept pace. As CPUs got faster, storage became the main bottleneck in high data-volume environments.
Designing a High Performance Lustre Storage System: A Case Study
August 19, 2014 by
Intel’s White Paper, “Architecting a High-Performance Storage System,” shows you the step-by-step process in the design of a Lustre file system. It is available for download at insideAI News White Paper Library. “Although a good system is well-balanced, designing it is not straight forward. Fitting components together and making the adjustments needed for peak performance is challenging. The process begins with a requirement analysis followed by a design structure (a common structure was selected for the paper) and component choices.”
Intel Enterprise Edition Software for Lustre
August 13, 2014 by
Intel Enterprise Edition Software for Lustre can transform vast amounts of data into data-driven decisions. And using the software with Hadoop makes storage management simpler with single Lustre file systems rather than partitioned, hard-to-manage storage.
