Kristof Horompoloy, ValidMind VP of AI Risk Management, shares his top three tips for validating #generativeai (#GenAI) models.
Download the ValidMind technical brief, "Validating Generative AI" here: [ Ссылка ]
Transcript:
So, in terms of validating GenAI models, model validation teams and model risk teams will need to be considering three important things — and this is by no means exhaustive — but it's something to to consider. First, the the number of GenAI use cases that will be coming across their desks is going to be large, so it will be very important to set up that risk assessment process properly so that they can select the use cases that are truly high risk and truly need their attention because they will not have the bandwidth to be reviewing everything.
So, setting up that risk assessment process and basing it on what the use case is actually being used for is going to be incredibly important and this means not to be distracted by questions like explainability, transparency, or accountability but really looking at questions like, is it client facing? Does it have regulatory implications? What are the impacts on your customers when you use it? Thinking about these questions will help help model risk management teams really prioritize the the use cases to focus on.
Second, I would say they have to make sure that they involve the proper stakeholders in the validation process for traditional models. Model validation was really the key team in the second line of defense with GenAI. This has changed significantly because the risks are are so varied. You'll be dealing with compliance risks, with legal risks, with security risks, data privacy risks, so on and so forth, and you have to bring in the right stakeholders to be able to to validate and review those aspects of the model.
The third recommendation that I have is considering the scale and the magnitude of these use cases to automate as much of your process as you possibly can, and this means if you're looking at lower risk situations, make it a light touch review, automate the documentation as much as possible, have automatic steps in your workflow and your processes so that you can really keep your focus on the most important use cases that will affect your employees and your clients the most.
ValidMind can can help you through this process and all three of these points through various features that we provide.
First, for the risk assessment we can incorporate your risk assessment directly into our workflows, which allows you to branch your process into various processes depending on the risk of your use case. We also of course help configure your workflows in a way that you can bring in all of your stakeholders and automatically alert them when they are needed for a step in your workflows or in your process. And finally, of course, we provide a lot of help with the automation of the documentation of your of your use cases as well as um automating the testing which should be bespoke uh to your to your use cases in GenAI.
