Building AI applications is not easy — you’re constantly trying to think how to make an inherently stochastic system reliable. We are working at Basejump AI to make the ‘zero shot’ or first response from the AI as accurate as possible, but to truly deliver on the promise of AI-enabled data analytics, humans and AI need to work together for better outcomes and higher accuracy.

We already rely a lot on the expertise of data professionals to fill out the metadata within Basejump AI. We’re now giving another feature that will provide even more control to ensure users are seeing what you want them to see and also the most accurate information possible. To help improve the feedback to the AI as well as improve the retrieval of accurate information, we’re introducing verification.

How Verification in Basejump Works

As a user is chatting with the AI, a user can now mark a result as verified.

What this means is that user verifies the correctness of this information. After marking something as verified, this information can then be returned for users that have access to the same connection via their teams and ask the same question. This is possible because of semantic caching¹.

This is an example of a response being returned as pre-verified due to another user verifying the information ahead of time. The chat bubble has a verification checkmark in the top right corner.

There are 2 types of semantically similar results that can be returned in the cache:

• Identical: These results will return the same information as was originally cached if it is current within the last day. If it is older than 1 day, then the result is refreshed.

• Similar: These results will return different information, but the SQL query only differs in the where clause being applied. For example, if someone asks ‘how many bagels are there?’ and someone else asks, ‘how many sesame seed bagels are there’, the 2nd query is similar enough to be verified (we know how to count bagels), but will be updated to filter to sesame seed bagels only.

For more detail, please refer to the documentation².

Verification Modes

So how can you get accuracy up to 100%? We think about that question in a few ways: one option is the accuracy the AI is able to get you without verification at all, another is with verification and then AI is still able to return non-verified responses, and finally is the option where the AI is only returned verified responses.

To get up to 100% accuracy for the time being, a user would need to pre-verify similar questions. The AI then can modify the filter on those data objects and return those to users. This will limit the users ability to explore information that has not already been reviewed by an admin interacting with the AI.

The setting to control whether allow unverified results to show in response to user questions is called ‘Explore’. Only allowing verified results to be returned is called ‘Strict’. These options can be set by the owner within the company page.

How Verification Builds Implicit Consensus and Trust

Verified results is a great step to achieving common consensus around common metrics. Every time a result is returned, users can verify the output and other members will get the same query ran for them as well or the same information if the questions are the same and it’s using the same datasource connection. These verified results start building up a library of metrics that can be used to get current relevant information.

The Basejump AI Platform is essentially crowdsourcing agreement on metrics by allowing users to verify and unverify information. The more questions that are asked and the more that is verified, the more other users can see that information and have pre-verified information returned to them.

We believe strongly that AI analytical systems that have the ability for people to collaborate and trust in their data will ultimately be the tools that businesses come to rely on.

Interested in learning more?

• Book a demo of the Basejump AI Data Analytics platform: https://basejump.ai/demo

• Try Basejump AI for free here: https://basejump.ai/

• Follow us on LinkedIn: https://www.linkedin.com/company/basejumpai/

References

1. What is Semantic Caching (2024), Redis

2. Basejump AI Documentation: Verification, Basejump AI