Skip to site content
New to Availity? Get Started

Demystifying AI in Healthcare: The Top Seven Questions to Ask Your AI Vendor

There’s been a lot of buzz in healthcare about the potential of artificial intelligence (AI) to streamline administrative processes, while ensuring timely care delivery, resource allocation, and patient satisfaction. This, combined with market drivers, including the global tech surge, increased competition, and the popularity of generative AI, have created a perfect storm of interest in AI technology, along with a myriad of companies offering AI solutions to fill this demand. However, despite this excitement, not all AI solutions are created equal.

For executives in an industry as unique as healthcare, where errors can have dire consequences, the stakes are especially high. Skepticism centers around the role and potential risks of AI. It is critical for both providers and payers to have a firm understanding of the various types of AI technology available to them, so they can best assess potential AI solutions for their various care determination and delivery workflows.

This blog breaks down the fundamentals of AI – offering an overview of the most common technologies available in the market and a list of questions to ask vendors when evaluating AI solutions.

AI 101: Breaking Down the Different Types of AI

AI technologies can be categorized into three groups: analytical AI, reactive AI, and generative AI. Solutions leveraging analytical AI analyze and interpret complex data sets, uncover insights, and make data-driven predictions or decisions. Reactive AI technologies operate based on predefined rules and are designed to perform specific tasks without the ability to learn or adapt.

Generative AI technologies generate new content including audio/video images, and text by drawing from learned patterns in existing data using Large Language Models (LLMs). LLMs use complex statistical methodologies to process natural language inputs and attempt to predict the next best word to respond to a prompt based on the data it’s been trained on. It then predicts the next word, and so on, until its answer or response is complete. While the ability of LLMs to understand language and generate answers relevant to the conversation’s context is remarkable, there are many limitations, including a propensity for “hallucinations,” which can result in factually incorrect responses. Also, LLMs are limited to the information provided to them when they are trained. Since these systems make determinations based on training data that may reflect human biases, generative AI has the potential to perpetuate societal biases as well. Furthermore, the “black box” nature of some generative AI systems has created growing concerns due to the inability to see how those AI systems make their decisions.

Questions to Ask Your AI Vendor

Given the inherent biases of AI and risk of inaccuracies, when evaluating AI solutions to meet organizational needs, healthcare organizations should be cautious to ensure responsible and ethical use of AI.

Below are the top questions to ask your AI vendor during the exploration process.

  • Have you done it before – cradle to grave – with AI? AI systems are not like other IT systems; they change over time, which means that extensive technical expertise is required to create, deploy, and maintain effective solutions. Ensure the vendor you are working with has at least five years of experience maintaining their AI solutions and ask how you will be notified of changes to the AI performance and modeling. If it is a learning system, ask if they will be monitoring the system. If the vendor doesn’t provide clear answers or they haven’t done it before, walk away to avoid being a test case that could deliver a negative outcome.
  • Can they explain the AI system to your clinical staff? Understandability is another key attribute to look for before adopting any new AI technology. The solution should use terminology in a way that your clinical staff understands and leverage language that is commonly used within your healthcare environment.
  • Is the AI technology observable? While there are many variations among AI approaches and technologies, transparency is an important quality to consider as well. During the evaluation process, you should be able to see what the AI is doing. Ask your vendor what data the solution uses to make decisions and what is the source of that data. Also understand what happens when the system doesn’t know or is unsure how to answer a question.
  • Does the AI provide the right analytics? To ensure confidence in the AI solution’s decisions, avoid “black box” approaches to AI. Ask the vendor if it’s possible to assess the data being produced by the AI solution. You should be able to review the determinations and share them throughout your organization. It’s also important to confirm that the solution delivers the right level of analytical information to enhance business insights.
  • Does the AI technology keep you in the loop? While automation technologies can be an integral tool for success when dealing with timely and costly workflows, they must always have human oversight. Ask your vendor if the solution keeps humans in the loop. It’s also important to have control and visibility to all the information being used and to ensure that the system makes recommendations instead of forcing decisions on your staff.
  • Is the data accessible? If the data from the AI tool is not easily accessible or integrated into clinical workflows, it can lead to reduced efficiency and productivity, creating frustration for healthcare providers and patients. For this reason, avoid vendors that only make data accessible behind a paywall or require payment for access to your data later. The AI solution should provide easy and immediate access to your data.
  • Did they say yes to everything you asked for? Despite its promise, AI can only solve limited problems, so avoid misconceptions about what it can do. Prioritize vendors that are open to discussing their technology’s limitations and evolutionary path. If a vendor says their system doesn’t make mistakes and answers yes to every problem you present, that’s a red flag that the technology being pitched by the vendor may be too good to be true.

For a full breakdown of the information detailed in this blog, download our Questions to Ask Your AI Vendor Cheat Sheet . Also visit Availity.com/AuthAI to learn how Availity is leveraging analytical AI to streamline prior authorization reviews.

References

Availity provides the information in this blog for education and awareness use only. The information provided here is for reference purposes only, and does not constitute the rendering of legal, financial, or other professional advice or recommendations by Availity.