Use of Natural Language Processing and AI Driving Big Data Innovation to New Levels

In this edition of Industry Perspectives, Sean Keenan, Co-founder and Vice President of Product for Unifi Software, explores how tools like natural language processing (NLP) and artificial intelligence are changing the landscape of big data and driving innovation forward. 

Sean Keenan, Co-founder and Vice President of Product for Unifi Software

We’ve all heard the expression that ‘data is the new gold,’ and there’s no doubt that customer data in particular can fundamentally transform a business. However, data is rarely stored, located or created in one place—and it’s really hard to know what data does exist to have a holistic and accurate view of a business. Any business executive with accountability to the bottom line will tell you, the success of a strategic initiative is dependent on accurate and timely information. Today though, manual processes that require folks in IT to dig through data to get insights for everyone else simply doesn’t scale.

To really drive innovation to a whole new level we’re seeing a rise in the use of Natural Language Processing (NLP) and Artificial Intelligence (AI) to make finding and using trusted, quality data that much easier. In fact, leading industry analyst firms have noted that Machine Learning, AI and NLP are quickly becoming table stakes for analytics. That requires significant heavy lifting at the infrastructure level and it’s not an easy thing to do.

NLP, for example, requires a systematic view of grammar to understand the context of an attribute. In a query if someone types in ‘show me all the PII data’ the system must first understand all the words in the phrase where ‘show me’ equates to ‘execute a search’. ‘All’ would signal a search in every row, column and comment field specifically for ‘PII’ which encompasses a multitude of other variables such as a person’s name, their date of birth, social security numbers, credit card numbers, address fields, etc. Once the system figures out each word and then how one word relates to another it can execute a search.

We hear constantly from customers and prospects alike about the value of being able to search using natural language.

The open source community has made significant contributions to help make NLP and ML algorithms more mature. Open source software keeps evolving as developers are constantly putting code in the wild to trial and enhance. The open source community has led work in deep neural networks, AI, ML and NLP to help developers understand the underlying grammatical logic – what’s a noun, what’s a verb, how does one word relate to another. At Unifi Software our IP is in our execution of our NLP and AI and like many other companies that are innovating in this space we take full advantage of the frameworks in the open source community as a jumping off point.

We hear constantly from customers and prospects alike about the value of being able to search using natural language. The need and opportunity among healthcare providers and organizations for example, is incredible. Anil Srivastava, the Founder and President of Open Health Systems Laboratory, shared with us recently that he has a lofty goal—to build a global team science consortia to provide a quicker and better public health response to cancer by leveraging the best biomedical informatics, information and communication technology available. The technical challenge that they face is how to collect all the raw data from hundreds of cancer research projects around the world and provide an easy to use interface to help oncologists tap in to this global base of knowledge.

Within our connected enterprises we’ve amassed a rich metadata directory where we can extract value from defining relationships – even putting them into a knowledge graph and then building the NLP layer above that. The more data sources—the stronger the graphing capability. This is a very smart way to visualize the user experience and it’s another way to empower more business users to innovate faster through data-driven insights.

All of this drives the market forward and as we see NLP and AI technology maturing our use and implementation for data analytics will lead business transformation well into the next decade.

Sean Keenan, Co-founder and Vice President of Product for Unifi Software has deep experience in delivering large scale advanced analytic solutions to clients across a number of verticals. He is a career big data practitioner that has been involved in 100’s of data Implementations across the F1000. Sean has played pivotal roles at early start-ups including Greenplum, Alpine, and Platfora.