Research of Data Lakes

In Q2 2015, Radiant Advisors and Unisphere Research, a division of Information Today, Inc., released “The Definitive Guide to the Data Lake,” a joint research project that sought to clarify the industry’s emerging data lake concept through combining fundamental data and information management principles with the experiences of early implementations to provide deeper insight and a practicable perspective into how current data architectures are transforming into modern data platforms.

Following the success of that report, Radiant Advisors and Unisphere Research reconvened with the support of sponsoring vendors Teradata, Hortonworks, Attunity, and HP Data Security to launch a supplementary research study on the current state of data lake adoption maturity. By surveying both current and potential adopters in
the industry, this study was designed to document key perceptions, challenges, and successes by focusing on data organization, integration, security, and definitional clarification to address key areas of concern and interest in ongoing data lake adoption. The intent of the survey and this corresponding report is to understand and share the current and planned adoption of technologies in the Hadoop ecosystem, intended specifically for a data lake strategy, and to learn how adopting companies are addressing critical data lake success factors, including rethinking data for the long-term, establishing governance first, and tackling security needs upfront.
The survey and report also identify emergent areas of concern and new areas of clarification needed for data lake maturity.

We surveyed 385 IT practitioners and stakeholders at organizations within a variety of industries who subscribe to Database Trends and Applications
and Big Data Quarterly magazines; contributing respondents were, therefore, from a highly technical audience. Approximately 60% were IT and database administrators and staff or were otherwise identified with IT operations, while the remainder held CXO positions or other executive leader business roles. There was also a small sample of responders from academia, including roles as both student and professor (less than 5%).

At the organizational level, the most dominant organization sizes in the survey were those with
more than 20,000 employees worldwide (30%) and those of less than 250 employees (21.81%). Of these organizations, a broad spectrum of industry verticals were reached by the survey, with finance and software being the most highly represented at 13% and 11%, respectively. Other well-represented sectors included education, government, and manufacturing. Finally, regarding geographic engagement, North America represented the largest region represented in the survey, with over 80% of responders.