As we inch ever closer to GDPR in May, companies doing business in Europe need to start getting a grip on the sensitive private data they have. The trouble is that as companies move their data into data lakes, massive big data stores, it becomes more difficult to find data in a particular category. Clairvoyant, an Arizona company is releasing a tool called Kogni that could help.
Chandra Ambadipudi, Clairvoyant’s CEO, says the problem with most companies’ approach to big data is they don’t think about security until after they create the data lake, at which point it becomes quite challenging to understand what you have.
Kogni helps companies create a data dictionary of sensitive data to more easily identify the data that matters most to them. That could be credit card numbers or whatever a particular company feels is most important. In regulated industries understanding what sensitive data they possess is even more important. Yet the data lake architecture makes it more challenging to find it.
Kogni relies on machine learning to identify patterns and recognize the sensitive data types a company has defined. Ambadipudi says this isn’t just a one time scan. “The notion of understanding where sensitive data is has to be an ongoing process,” he explained. He says the software constantly monitors the data lake for new sensitive data.
Kogni Dashboard. Photo: Clairvoyant (cropped)
The software also includes mechanisms for securing the data, along with an open framework for customers who want to implement custom security solutions. It can also issue alerts in the event sensitive data is being used in an unsanctioned way.
As for GDPR, Ambadipudi is careful not to call this a GDPR compliance tool, but he does think it can help customers understand the sensitive data they have. ”We can help you build this sensitive data catalogue and figure out what is sensitive personal data.” The security and alerting component could also help in this regard.
Clairvoyant is actually a consulting firm that has worked with companies like Cloudera, which helps customers build data lakes on top of the open source Hadoop project. Clairvoyant provides data management services for these projects, and as part of that work has developed Kogni to help these clients better understand their data once they place it in a data lake.
The company, which is located in Chandler, Arizona, was founded in 2012 and has around 250 employees. It has bootstrapped without external investment to this point.