Creating a good AI product is like building a house: you need a strong foundation. To make sure your AI is doing what it's supposed to, you have to test it regularly. Start by creating simple tests (like checking if the AI can find information correctly) and then get feedback from experts in the field. It's important to keep track of how the AI is doing over time and adjust it based on what you learn. You can also use another AI to help you check the work of your first AI, kind of like having a teacher check your homework. But don't forget the most important part: always look closely at the data yourself to see what's really going on and where your AI needs improvement.

https://hamel.dev/blog/posts/evals/

https://hamel.dev/blog/posts/llm-judge/