Is this even AI?

Since the first edition dropped, I've gotten several questions about how to tell real AI solutions from the fakes.

Since virtually every tech company in our space is now slapping "AI" on their marketing, I figured it was time to give you some tools to separate the real from the BS. Here are a few simple questions that'll separate the real from the imposters:

QUESTION 1
What's the underlying AI technology? Which LLM are you using?

Red flag: "Our AI tech is proprietary" or "We built our own AI engine"

Look, it's taken years and billions of dollars for companies like OpenAI and Anthropic to build these systems. If some dealership software company claims they built their own LLM, they're blowing smoke.

Solid answer: "We integrate with GPT-4" or "Built on Claude with custom training"

This means they're actually using real AI technology. From there, you can dig deeper into how that integration works and what makes it valuable for dealers.

QUESTION 2
What data is your AI trained on?

Red flag: "Our solution knows everything about our customers" or "we trained it on all industry data".

If they are claiming that they know everything about their dealer customers or the industry as a whole then they are full of it. Real AI training is expensive and requires well structured data sets.

Solid answer: "We trained it on 2 years of our customers CRM data consisting of thousands text conversations."

We're looking for specific answers are here. Real AI companies will tell you exactly what data they used. Training their AI model is what makes their AI unique and powerful. It requires a substantial amount of their time and investment, so they should be able to speak about it in depth.

QUESTION 3
Does your AI ever give incorrect or misleading answers?

Red flag: “It’s smarter than all of us combined” or “It always gives the best possible answer”

If they say their AI is perfect or never makes mistakes then they are lying. 

Good answer: “Of course. This is brand new tech. We closely monitor AI’s responses and fine tune it so the same mistake isn’t made twice.” Even the most state of the art AI softwares make mistakes. The key is that the AI will keep improving as it is constantly trained and fine tuned. The vendors need to be very hands on to monitor and constantly improve their system.

Now that we have some basic questions to vet AI vendors, we'll put them to the test in next week's edition. Stay tuned as we start testing and reviewing the AI tools hitting the market.