By Alex Babin, Co-Founder and CEO of ZERO Systems
When objects collide, mass matters, but so does speed. The new reality of Generative AI is entering the world of enterprise at warp speed. This is particularly true for Large Language Models (LLMs) like GPT or solutions like ChatGPT. As Marc Andreessen said 12 years ago, “Software is eating the world”, but what we see now is “AI is eating the software.” Companies driven by internal demand will either try to build generative-native apps from scratch or enhance their existing software applications with embedded generative capabilities. These apps will use a combination of general-purpose models like GPT4 in combination with internal and external enterprise tools. After all, models will become a commodity and a power supply for all AI applications. One thing is known for sure: The supply for these domain-specific models is data. Making internal, unstructured data available to these models during domain-specific fine-tuning will be critical to the quality of their output.
It sounds simple, but enterprises face significant challenges in connecting internal data to modern AI technologies. This relates to the data the enterprises generate and store themselves and to the data that belongs to their clients, which is even more restrictive. When bringing in new AI technologies, organizations consider the following:
- Risk of providing sensitive data to external systems and models.
- Risk of unauthorized access to the data inside the organization.
- Reliability of the output generated by the models.
- Cost of computing when processing unstructured data.
Our clients have raised these questions time and time again, and for the last 4 years, we’ve been solving them for the most risk-averse organizations in the world, which include large law firms as well as companies in the Fortune 500 list. Here is what we’ve learned at ZERO and how it affects the way we think AI companies can address the same issues.
Security concerns: keep your friends close, but your data closer
The effective utilization of LLMs in an enterprise requires connecting with internal data and systems. And this connection should be compliant with the data security policies of the enterprises.
For enterprises, especially in professional services like Finance, Insurance, and Legal, the data mainly consists of clients’ sensitive information, and exposing it to outside models presents a high risk.
There are two approaches a company can take to mitigate this risk. The first is to bring models within their security perimeter. The second is to focus on only using the depersonalized data with external models and only using external models for specific use cases when the internal models fall short. LLMs like GPT-4 are great at doing various things, but nothing beats a smaller model trained for a specific task and fine-tuned on the organization’s data.
To address this demand, we’ve built an AI engine, Hercules, which operates within a company’s security perimeter and prepares internal data to be used with external models of the client’s choosing while also supporting internal LLMs for various specific tasks. Besides balancing external and internal models, the engine does many other essential things like systematic data labeling, depersonalization, enrichment, and interconnectivity, acting as an abstraction layer between corporate data, internal corporate systems of records, internal and external models, orchestrating it in an efficient and secure flow.
Unauthorized internal information access: all accesses are equal, but some are more equal than others
Enterprises have multiple layers of security set up to protect their most sensitive data. This is a critical element of data governance, and even inside one organization, different levels of management have different levels of access to the information. This is why organizations have “ethical walls” set up for each internal system, and Hercules inherits the access control layers that are already set up for users inside these systems. This prevents exposing data to external models that should remain in the security perimeter of the organization and internal users not getting access to the output they are not authorized for.
Another critical element of “ethical walls” is related to model fine-tuning. A verification layer should be applied to check if the data is permitted to be used in fine-tuning the model. It’s essential to ensure data verification before indexing according to data governance rules.
Reliability of output: to trust or not to trust – that is the question
Models hallucinate. That is a fact. While they will become better over time, they will still fabricate. If you’re writing a blog post using ChatGPT and a mistake is made, it’s not a big deal. But imagine you’re dealing with critical financial data, and the error found its way into a financial statement and made its way up to your annual report for shareholders. That mistake might become costly. Using internal data as the source of truth is the only way to avoid such problems.
To alleviate this, Hercules implements fact-checking on indexed internal data to eliminate the chance of hallucinating essential facts and values. Cross-referencing the statements in the output against facts from internal sources is critical for every enterprise.
Cost of computing when processing unstructured data: To infinity and beyond
Cloud computing can be expensive on a large scale. Enterprises need to find a way to get the maximum out of training models while controlling costs. This is especially true if they need to label unstructured data in their systems.
Many companies can help enterprises to unlock the power of unstructured data. Gartner has a recent Cool Vendor report published about this – ZERO is mentioned there, too, but because we address this process differently.
Hercules is designed, built, and trained to provide systematic labeling of unstructured data at the point of ingestion. The data is processed at the edge and inside the security perimeter. This not only utilizes the processing power of existing infrastructure (CPUs are used for inferencing) to unlock the sea of metadata but also reduces computing costs for the client.
The future of generative AI: A new hope
More generative platforms will emerge, enabling companies to build apps that seamlessly integrate multiple models, internal and external tools, and knowledge sources. These platforms will optimize prompting and chaining, ensuring secure implementation of generated content while offering valuable insights, cognitive automation, and support.
In a nutshell, Hercules is an AI engine designed to unlock the potential of AI and Large Language Models (LLMs) for enterprises. It provides businesses with a secure and efficient way to leverage their data across various internal systems: Finance, CRM, DMS, HR, and others. Hercules’ main goal, though, is to provide end-to-end solutions to users. These components are called Skilled AI Modules (SAMs), and they are being built to augment cognitive processes, offering automation, insights, and proactive assistance to human workers. The main parameters for any AI application for the enterprise should include:
- Ability to balance between the efficiency of internal models and the horizontal power of external models like GPT.
- Seamless Integrations with a wide range of enterprise systems specific to different industries, providing a unified API layer for interaction between these systems and LLMs.
- Systematic Data Labeling for the ability to add multiple labels to the unstructured data while also mapping it to the enterprise ontology.
- Security and Compliance layers to ensure secure access to enterprise data.
- Indexing of unstructured data in vector DB for efficient search, retrieval, and verification.
- Output Validation and Correction to verify and correct the output to prevent risks associated with hallucinations.
- Hybrid Deployment Architecture that offers an on-premises installation in customer servers or VPC, ensuring a modular and interoperable solution.
The generative AI revolution is just beginning. Enterprise leaders across all industries, especially Fortune 500 companies, must pay close attention and move fast to disrupt their own processes or risk being disrupted by others.
By addressing the common AI pitfalls, we are bridging the gap and alleviating the risks that advanced AI technology presents in its current state, enabling enterprises to harness the full potential of generative AI.
Learn more about how ZERO Systems brings generative AI and Large Language Models to Fortune 500 companies. Visit our website: https://zerosystems.com/
