#BigDataProblems: What to Do When the Data Isn’t So Big

shashiIn this special guest feature, Shashi Upadhyay, CEO of Lattice Engines, focuses on how small, or sparse, data and present a unique challenge to marketers who are not yet equipped to accommodate this type of data. Shashi is responsible for advancing Lattice’s vision to deliver the power of prediction to sales and marketing organizations. His unique background as a Cornell data scientist, turned McKinsey partner, drove the founding of Lattice. Today Lattice has changed the lives of tens of thousands of sales and marketing professionals with award-winning technology that enables them to use advanced data science to be more successful at their jobs and their companies to drive more revenue. Shashi holds an undergraduate degree from the Indian Institute of Technology at Kanpur and a Ph.D. in Physics from Cornell University.

As digital technology advances, businesses are collecting a tremendous amount of unstructured data, pushing them to employ machine learning, predictive analytics and highly-advanced algorithms to tackle the challenge of sorting, analyzing and acting on the influx of data at their disposal. Companies are also collecting more specific, real-time data that can be used to discover current conditions and timely insights. As such, the newest challenge marketing teams must tackle is leveraging the small data that is being collected about their prospects and customers.

Instances of small, or sparse, data—like how the purchase of one product relates to the purchase of another—present a unique challenge to marketers who are not yet equipped to accommodate this type of data. This is largely because machines themselves haven’t been fully equipped to process and analyze small data. However, some recent Open AI developments tout small data capabilities, like Amazon’s DSSTNE, giving marketers access to new data that helps complete buyer profiles. In order to gather the most complete insights into buyer propensity, marketers must embrace both big and small data.

So how can marketers who are accustomed to big data best incorporate small data into their existing workflows?

One way is to welcome advances in AI and actively search for toolkits that embrace advanced machine learning. A World Economic Forum report warned that AI, robots and other tech advances will take more than 5 million jobs from humans over the next five years. While it’s true that machines will replace manpower across industries, humans will always play a critical role in the data analysis process, especially when it comes to small data. Marketing teams must take the outputs from AI and machine learning systems, and infuse that information into marketing processes so they can tell strategic, personalized stories to customers and prospects. Using the data-driven insights from AI solutions and then applying human processes and storytelling will be integral for companies looking to use both big and small data in their successful marketing campaigns.

Another way marketing teams can better incorporate small data into their programs is by hiring more team members with an interest or background in data science. Named the hottest job of 2016, data scientists are no longer coming from a strictly mathematical background, nor are they limited to working in a company’s analytics department. These new citizen data scientists—those that leverage analytics even though their primary job function is outside the world of statistics–will play an important role in incorporating small data into targeted marketing campaigns. As such, smart marketing teams should recruit data scientists to help bolster small data capabilities within existing marketing campaigns.

Furthermore, it’s important for marketers to note that the possibilities presented by machine learning systems are only starting to be realized. With projects like Elon Musk’s nonprofit Open AI and Amazon’s DSSTNE that seeks to extend deep learning to include search and recommendations, it’s clear that these technologies have only just scratched the surface of their potential. As these collaborative projects grow and succeed in stretching the boundaries of machine learning, smart data scientists will be able to analyze small data sets and draw correlations that marketing teams can utilize.

For anyone who has attended marketing conferences or webinars over the past 12 months, it should come as no surprise that today’s successful marketing teams are incredibly dependent on data. Marketing is no longer the land of the creative brand designers. It’s now owned by data-driven demand gen leaders who are discovering and implementing new ways to pull data into their systems and make sure that every program is targeting the right person, with the right message, at the right time. For these savvy demand gen marketers, small data is the next frontier. The teams that learn how to integrate small data into their programs and processes will undoubtedly see increased customer engagement and company revenue

 

Sign up for the free insideAI News newsletter.