insideAI News Guide to Big Data Solutions in the Cloud

For a long time, the industry’s biggest technical challenge was squeezing as many compute cycles as possible out of silicon chips so they could get on with solving the really important, and often gigantic problems in science and engineering faster than was ever thought possible. Now, by clustering computers to work together on problems, scientists are free to consider even larger and more complex real-world problems to compute, and data to analyze.

Attaining High-Performance Scalable Storage

As compute speed advanced towards its theoretical maximum, the HPC community quickly discovered that the speed of storage devices and the underlying the Network File System (NFS) developed decades ago had not kept pace. As CPUs got faster, storage became the main bottleneck in high data-volume environments.

Designing a High Performance Lustre Storage System: A Case Study

Intel’s White Paper, “Architecting a High-Performance Storage System,” shows you the step-by-step process in the design of a Lustre file system. It is available for download at insideAI News White Paper Library. “Although a good system is well-balanced, designing it is not straight forward. Fitting components together and making the adjustments needed for peak performance is challenging. The process begins with a requirement analysis followed by a design structure (a common structure was selected for the paper) and component choices.”

Intel Enterprise Edition Software for Lustre

Intel Enterprise Edition Software for Lustre can transform vast amounts of data into data-driven decisions. And using the software with Hadoop makes storage management simpler with single Lustre file systems rather than partitioned, hard-to-manage storage.