New Survey Reveals Businesses Are Bullish on Data Lakes

Print Friendly, PDF & Email

Arcadia Data, provider of the visual analytics and BI software native to big data, announced the results of a survey conducted by Eckerson Group, which paint an exciting picture of data lakes and the analytics tools, such as business intelligence (BI) on Apache Hadoop®, that surround them. The results highlight how today’s organizations understand that data lakes play a crucial role in generating value from data. The survey, conducted between March and June, 2018, drew more than 500 responses from business users across varying roles and industries.

The data lake has long served as a powerful tool for data scientists and data engineers. However, today’s business environment often requires that users without coding or scripting skills access the data stored in lakes. The survey results highlight how data lakes support more casual business users, enabling visual reporting and analysis and BI with point-and-click accessibility. Though some bemoan the idea data lakes are turning into data swamps, this survey underscores that businesses are bullish about data lakes and that some have already reaped the benefits of modern lake technologies. In fact, nearly 66 percent of respondents felt “business users can explore data (e.g., filter, drill) to get the views they want.” However, for all the praise data lakes receive, there are still gaps in how many casual users have access to the information and what they are able to do with it.

Though Cloud Increasingly Popular, Data Lakes Still Hadoop Territory

Despite the general growing reliance on the cloud, sixty-one percent of organizations deploy their data lakes on premises. It is clear that Hadoop deployments are viewed as useful tools for business.

  • Most often, organizations deploy their lakes on Hadoop clusters, where 62 percent of respondents have deployed their lakes.

Data Lakes, With the Right Technology, Offer Clear Value

Data lakes in and of themselves are a solid foundation, but it is the BI and analytics technology organizations use that make the difference. When implementing the right tools over data lakes, the benefits manifest themselves clearly:

  • Sixty-one percent of all respondents said “business users can author and edit reports and dashboards without coding.”
  • More than three-quarters (76 percent) of respondents believe “BI/analytics for our data lake increases the number and value of analytics for business users.”
  • Nearly three-quarters (72 percent) of respondents felt the data lake they use “fosters better decisions and actions by business users.”

Usage Gaps Show Through Aura of Excitement

For all the excitement around usability, the technology to extract value from data lakes requires some maturation. For more advanced analytic and data preparation capabilities, data lakes are show promise but are still lagging:

  • Nearly half (45 percent) of all organizations have fewer than 100 users who access their data lake.
  • Only half of all users are casual users.
  • Forty-two percent said administrators had to move or copy data to “create virtual data sets or semantic views.”
  • Only half of respondents felt “business users can blend data sets located inside or outside the data lake.”
  • A full 49 percent of respondents do not believe users can “view complex correlations.”

While some of these figures are encouraging, it is clear we have some work to do before every employee, casual or power user, feels comfortable using data lakes for analysis.

Performance, Governance Impressive With Opportunities for Growth

Though most analytics environments are expected to respond quickly to queries, there is still room for improvement for data lakes:

  • Just more than half (56 percent) of all respondents agreed that their data lakes provide “consistent, fast performance for all types of queries.”

Impressively, users also feel secure in the governance features their data lakes offer. Not only do they trust their data lakes with their information, but the tools offer flexible provisioning for optimal use.

  • Two-thirds (67 percent) of respondents “trust the accuracy of analytics running against the data lake.”
  • More than two-thirds (71 percent) said systems administrators “can set permissions for accessing data and analytic functionality at a granular level.”

Business users want direct, fast access to data. It’s no longer acceptable to be chained to mountains of data without a clear, simple and effective way to derive value from it. Even better — as illustrated by this survey — you don’t have to be a data scientist or a PhD to do so. The ability to create value is accessible to more people than ever before,” said Priyank Patel, co-founder and chief product officer, Arcadia Data. “That’s why Arcadia Data has built a BI and visual analytics platform that caters to business users. Running natively on the latest data platforms, including Hadoop and Azure Data Lake Store, Arcadia Data empowers business users to query and visually analyze large data sets in an intuitive, point-and-click manner.”

Survey Methodology

Eckerson Group created an online assessment with its Rate My Data platform, comprising 14 scored questions, five environmental questions, and three demographic questions. The assessment enabled respondents to measure the value their data lake provides to regular business users in seven categories ranging from “self-service” to “performance.” Respondents receive an overall score that they can compare to everyone else who has taken the assessment. They can also filter the aggregate assessment data to benchmark themselves against a peer group by company size, industry, and location.


Sign up for the free insideAI News newsletter.

Speak Your Mind