Pepperdata Adds Comprehensive Chargeback Reporting to Effectively Measure the True Cost of Workloads Across Distributed Systems

Pepperdata-logoPepperdata, a leader in real-time cluster optimization, announced it is eliminating a significant hurdle for Hadoop to become an essential enterprise data platform by adding a chargeback feature to more effectively measure and allocate the costs of increasing workloads across distributed systems. For the first time, IT can clearly see how much capacity each user or workload requires and allocate costs back to departments that share a centralized, multi-tenant Hadoop deployment.

While Hadoop adoption is rapidly growing among enterprises across every vertical industry, companies are struggling to make Hadoop a core part of their business processes. The primary roadblocks are predictability when supporting multiple users and departments on a single Hadoop cluster; effective utilization of systems running multiple, diverse workloads; and visibility into resource contention, performance issues, and cost allocation among departments.

To realize business-driving, transformational outcomes, companies need to solve for three key things – predictability, visibility, and utilization – collectively. That requires making thousands of decisions a second to ensure optimal performance,” said Sean Suchter, Co-founder and CEO, Pepperdata. “We’ve reached an inflection point where more Hadoop jobs from various departments are stomping on each other – organizations can’t count on those systems to run efficiently without the tools to optimize cluster performance at scale. That’s the software gap in core Hadoop that we’re solving for.”

New Comprehensive Chargeback Feature Allows IT to Allocate True Costs

As Hadoop becomes an increasingly large part of a company’s IT spend, it’s more important than ever that it be efficient and, as a shared resource, to ensure that SLAs are being met. When multiple departments are using shared infrastructure, one group’s use of Hadoop must not slow down other jobs. To provide “internal Hadoop as a service” offerings, with many departments and hundreds or thousands of users running jobs on the same cluster, detailed visibility is also key.

As organizations scale their workloads using Hadoop, IT groups don’t fully understand which departments or teams are most heavily using the shared infrastructure. Chargeback gives users the ability to track Hadoop usage per user, per workflow, and per department – all in real time.

Over the past year, Hadoop adoption has steadily accelerated from single instance applications in production environments to multi-tenant environments. Going forward, organizations need to be able to better manage these environments and measure and cost out the amount of resources consumed,” said Nik Rouda, senior analyst at Enterprise Strategy Group. “The level of visibility and granularity that Pepperdata is delivering with its chargeback feature is incredibly valuable for organizations looking to integrate Hadoop into their standard business processes and put in place internal Hadoop as a service offerings.”

The Pepperdata engineering team has been working with Hadoop since its earliest days – when a few groups were exploring its use for research and building experimental projects. As more teams of developers realized its power, scale, and flexibility, it matured, and an ecosystem developed around it. Now, as Hadoop has entered a third phase, where businesses have started to run more workloads on the same cluster, they are realizing that out-of-the-box Hadoop doesn’t have the predictability and performance needed to support broad use by every business unit within an organization.

The company’s new chargeback feature adds reporting of company-wide and department-specific usage of key hardware resources. For the first time, central IT departments can accurately allocate the costs of running multi-tenant clusters and can use this information to measure infrastructure investment and build internal service offerings that can be charged back to relevant business groups.

Sign up for the free insideAI News newsletter.