Matillion, a leading provider of data transformation for cloud data warehouses (CDWs), and IDG Research have released findings of an IDG Research MarketPulse survey, “Gaining Time, Savings, and Insights via Cloud-Powered Data Transformation.” The research exposes the challenges companies face in leveraging enterprise data for analytics and identifies data portability, time-to-value, and self-service for business users as top requirements to address these challenges.
“Over the course of several surveys, IDG has identified the key challenges organizations face preparing massive data sets for analytics engines,” said Tom Schmidt, Digital Content Director at IDG’s Strategic Marketing Services. “The latest study finds that powerful and scalable cloud-native solutions can help optimize this business-critical process.”
The survey polled more than 200 IT, data science, and data engineering professionals at North American organizations with at least 1,000 employees. The top takeaways include:
Businesses waste too much time wrangling and preparing data
- It takes about a week, on average, to aggregate and prep data so that it is useful for analysis
- Nearly half (45%) of time spent on data analytics projects is dedicated to data preparation, instead of on more strategic, high-value tasks
- Six in ten cited lack of scalability and flexibility as a challenge when preparing data for analytics
- Nearly all (97%) are searching for ways to accelerate the data transformation process
Enterprises deal with widespread operational and technical challenges in getting data analytics projects to production
- Nearly half (47%) of respondents said data control issues are the biggest challenge to data analytics projects
- Other top challenges indicated are lack of a scalable, reliable technology platform to process large data sets (45%), having too many manual processes (38%), and challenges cleansing and preparing data (36%)
- 40% cited lack of visibility and control of data silos as a challenge when collaborating with business users
Blending cloud data platforms presents new opportunities
- More than one-third (38%) are already using cloud data warehouses (CDWs). Long term, 43% expected to have all of their data in the cloud, with the remainder planning to pursue hybrid models that leverage both cloud and on-premises data warehouses
- While the use of CDWs is already widespread, only 16% currently use data lakes. More than half (56%) plan to use data lakes in the future, and another 26% are considering doing so
- 57% will leverage a hybrid cloud strategy (on premises and cloud) for data management, while 22% are planning a multi-cloud strategy, and 21% will use a single cloud provider to manage all their cloud-based data
Enterprises require data portability, time-to-value, and self-service for business users to overcome current challenges
- Respondents said data portability (57%), ease of on-boarding (57%), and cost effectiveness (52%) are the top features of an analytics platform that can help them move past current obstacles
- IT professionals favored user-friendly (50%) and easier connections to data sources (50%) as top features
- Data professionals (data engineers, data scientists, data architects, etc) cited time-to-value (57%) and ability to provide self-service (51%) as important capabilities
“There is clearly still a struggle for enterprises to perform data transformation faster to provide the business with analytics-ready insights, said Matthew Scullion, CEO. “This research highlights not only what holds them back, but more importantly, the capabilities and features needed from a solution to deliver business value through transformed data. The power and the economics of cloud-native solutions help companies achieve faster time to value that can make a big impact in creating a data-driven culture for IT and business users.”
For the full results of the Matillion IDG Research MarketPulse survey, visit HERE.
Sign up for the free insideAI News newsletter.
