Hewlett Packard Enterprise Acquires Pachyderm to  Expand AI-at-Scale Capabilities with Reproducible AI  

Hewlett Packard Enterprise (NYSE: HPE) today announced an expansion to its AI-at-scale offerings with the acquisition of Pachyderm, a startup that delivers software, based on open-source technology, to automate reproducible machine learning pipelines that target large-scale AI applications. 

Reproducing a machine learning pipeline enables use of the same dataset to achieve the same  results each time to increase transparency, trustworthiness, and accuracy in predictions while  optimizing time and resources. It is critical to successful AI-at-scale initiatives, which represent the  next revolutionary step in realizing AI’s potential to increase the accuracy of predictions and achieve results faster. To attain these outcomes, organizations need to adopt technologies to efficiently  build and train larger machine learning models that require a high volume of complex data.  

“As AI projects become larger and increasingly involve complex data sets, data scientists will need  reproducible AI solutions to efficiently maximize their machine learning initiatives, optimize their  infrastructure cost, and ensure data is reliable and safe no matter where they are in their AI  journey,” said Justin Hotard, executive vice president and general manager, HPC and AI, at HPE.  “Pachyderm’s unique reproducible AI software augments HPE’s existing AI-at-scale offerings to  automate and accelerate AI and unlock greater opportunities in image, video, and text analysis,  generative AI, and other emerging large-language-model needs to realize transformative  outcomes.” 

HPE expands AI at-scale portfolio with addition of Pachyderm

HPE unlocks AI-at-scale opportunities for its customers by bringing together its leading  supercomputing technologies that are foundational for optimized AI infrastructure, and the HPE  Machine Learning Development Environment, a machine-learning software that enables users to  rapidly develop, iterate, and scale high-quality models from proof-of-concept to production. The combined solution already helps users train more accurate AI models faster, and at scale, on several  of the world’s fastest supercomputers that have been purpose-built for demanding AI workloads. 

Building on to these solutions, HPE will integrate Pachyderm’s reproducible AI capabilities in one  integrated platform to deliver an advanced data-driven pipeline that automatically refines, prepares,  tracks, and manages repeatable machine learning algorithms used throughout the development and  training environment.  

Delivering an end-to-end machine learning software platform to enable production AI at-scale

AI-at-scale capabilities advance popular use cases involving natural language processing, computer  vision, and video and image processing that are growing across industries such as transportation, life  sciences, defense, financial services, and manufacturing. 

By integrating Pachyderm’s machine learning pipeline capabilities with its existing AI offerings, HPE will enable faster development and deployment of more accurate and performant large-scale AI  applications with the following benefits: 

  • Data lineage – Visibility on the origin of the data and where it moves over time during the  machine learning lifecycle and analytics process to easily trace errors back to the root cause.
  • Data versioning – Ability to track different versions of data to understand when data was created or changed at any point in time, to increase efficiency in making any changes.
  • Efficient incremental data processing – As data changes over time, only incremental data  needs to be processed to update AI applications. Pachyderm makes incremental data  processing automatic and efficient. 

Lockheed Martin deploys HPE’s AI at-scale solutions for mission-critical applications

Lockheed Martin’s AI Factory, an open architecture approach to AI-at-scale, integrates Pachyderm’s  software, HPE’s Machine Learning Development Environment, and other modular solutions as part  of their foundational AI ecosystem. Leveraging these capabilities allows Lockheed Martin to increase  trust, maximize performance, and standardize AI technologies across a broad range of contested  environments in support of national security missions. 

This acquisition builds on HPE’s February 2022 investment in Pachyderm through its venture capital arm, Hewlett Packard Pathfinder, to speed time-to-market for AI innovation at lower data processing  and operating costs. The transaction is not subject to any regulatory approvals and is expected to  close this month. 

Product integration and availability  

Pachyderm’s software is available today to integrate with HPE’s existing supercomputing and AI  software solutions. Additionally, HPE plans to integrate Pachyderm with upcoming versions of the  HPE Machine Learning Development System, which eliminates the complexity and cost to build and  train models with a complete, ready-to-use solution.  

Sign up for the free insideAI News newsletter.

Join us on Twitter: https://twitter.com/InsideBigData1

Join us on LinkedIn: https://www.linkedin.com/company/insidebigdata/

Join us on Facebook: https://www.facebook.com/insideAI NewsNOW