The Union

Retrieval augmented generation (RAG) is revolutionizing AI by infusing language models with timely and relevant external data. This technique is pivotal in delivering not just intelligent but informed AI responses. In this podcast, Chris and I explain what RAG is, how it functions, its impact on AI’s performance, and the challenges it helps overcome. 

Key Takeaways 

• Retrieval augmented generation works by integrating large language models (LLM) with real-time data retrieval to provide accurate, contextually relevant responses, which reduces computational and financial costs associated with inaccurate responses 
• RAG fills knowledge gaps by using vector databases for better information retrieval and regularly updating knowledge libraries to maintain response accuracy, addressing the limitations of static data in AI models. 
• The practical application of domain-specific augmented generation use in industries like retail and e-commerce, telecommunications, and manufacturing demonstrates improved service delivery. 

Unlocking LLM Potential with Retrieval Augmented Generation 

RAG is a method that significantly enhances the capabilities of LLMs. RAG functions as a prompt engineering technique, enriching the output of LLMs by integrating an information retrieval component into your systems of record and data sources like CRM, HR, and external knowledge bases. Doing so provides AI systems with timely, accurate, and domain-specific data - a marked improvement over conventional large language models that often operate with static or outdated training data. This improves the LLM’s ability to generate accurate responses and limit hallucinations. 

More at krista.ai

2024 AI Predictions

What does the internet say about AI?

What do the AI pundits think will happen?

We were curious, too.

In our quest to understand what was being predicted for AI in 2024, we reviewed a set of diverse sources to analyze and merge a myriad of predictions to provide a consolidated overview. This article cuts through the noise, delivering a straightforward perspective on AI trends. We've factored in common predictions and outliers, providing you with a balanced view of who predicts what when it comes to AI.

The Sources Behind AI Predictions

In our review of AI predictions, each source offered distinct insights reflecting their unique perspectives. I've linked each of the sources from Adobe, Forrester, Gartner, IBM, IDC, LA Times, NVIDIA, PWC, TechCrunch, and TechTarget that we reviewed and categorized. 

• IBM emphasized predictions at an enterprise level, focusing on how AI would reshape business operations and strategies.
• Gartner and Forrester focused on the impact of AI on individual task levels, highlighting how AI could enhance personal efficiency and workplace dynamics.
• IDC provided a more IT-centric view, exploring how AI would aid IT professionals in their roles, with an emphasis on shifting outcomes and the emergence of conversations as the standard user interface.
• LA Times, PWC, and TechTarget brought attention to the coming of age of open-source AI, stressing the importance of ethical AI and the need for transparency in AI operations.
• NVIDIA presented a broader spectrum of insights, reflecting the diversity of opinions from the 17 experts they consulted, covering a wide range of AI applications and implications across various sectors and disciplines.

The AI Landscape - A Consensus View

Across the board, experts agree that generative AI is set to skyrocket this year, bolstering productivity and spurring innovation. Businesses are bound to see a significant shift towards multimodal AI, which invites a more natural interaction with technology using voice, images, and text. As these technologies advance, tight AI regulation is expected to emerge, guiding their integration into the market. The consensus is clear — AI is not just a fleeting trend but an innovation that is fueling economic growth and investments.

Outliers - Unique Predictions and Their Significance

Not all forecasts follow a common thread. Gartner casts a spotlight on AI's role as an emerging economic indicator of national power by 2027. Meanwhile, TechCrunch raises concerns about AI's potential misuse in the 2024 elections. NVIDIA equates the race for AI supremacy to a new space race. These outlier predictions, while not widely echoed, provide insights for businesses to consider, presenting both opportunities and warnings.

More at krista.ai

Most third-party risk lifecycles adhere to a similar pattern: planning, due diligence, contract negotiations, ongoing monitoring, and termination. However, the management and responsibility of these processes differ significantly across organizations. Traditionally, the information security department carried this burden, but recent events like Covid, regional wars, political changes, and socially-focused laws have broadened organizations' risk perception beyond just IT. They now include geographical, reputational, concentration, and compliance risks. 

Different departments, leveraging their unique expertise, now seek information from third parties to manage diverse risk types. Third-party risk management expert, Tom Garrubba, practical advice to assist companies in tailoring third-party risk management activities to their size, risk profile, and risk management necessities. Regardless of where the organization situates third-party risk management, the ultimate responsibility rests with the third-party risk manager and the business owner. They must identify the necessities and required documentation for each vendor, enabling a thorough assessment and due diligence or ongoing monitoring. 

The assessment process presents challenges for both the vendor and the risk manager, often requiring over 40 hours to complete and validate. Midsize companies dealing with dozens to hundreds of third parties quickly face the reality of these complications. Additionally, vendors often feel overwhelmed with assessment requests from their many customers and may instead issue a "customer assurance packet" containing broad information sets for you to sift through to identify potential risks. 

Third-party risk management is essential, even for industries not legally required to do so. Those lacking a robust strategy and supporting technology risk overloading their vendors with assessments and distracting internal teams. Furthermore, if you operate in a regulated industry, expect your strategy and technology to face scrutiny eventually.

More at krista.ai

Embracing technological change via automation and artificial intelligence (AI) is no longer optional; it's a necessity. Delaying AI use in your company can hinder progress and put you farther behind your competitors. However, embracing AI adoption is not without its apprehensions. Your concerns about unknown outcomes and hallucinations are valid but are easily overcome with the right security, accuracy, performance, and cost strategies to limit your risk and exposure. Integrating AI is about continual progress over perfection, focusing on the transformative power of automated processes, rather than the pursuit of unattainable perfection. We will show you how to overcome AI fear, build confidence, choose the right process for AI and guide you toward the first steps for adopting AI.

More at krista.ai

Generative AI (GenAI) is influencing nearly all processes in our businesses and none so much as knowledge management. Employees want a better experience and they have found by already experimenting with GenAI; ask a question, and get an answer. But, the answers and the knowledge delivered to them via the public interfaces aren't always correct. 

Our guest speaker Julie Mohr is a principal analyst at Forrester covering IT service management and enterprise service management. Julie shares how knowledge management practices are evolving and how GenAI is accelerating change. Julie spotlights the shift from old-school waterfall techniques to agile knowledge management and describes how GenAI is set to overhaul how companies capture, update, and apply their knowledge. 

More at krista.ai

Generative AI vs Predictors and Categorizers

Generative AI is hot and has ignited our imaginations. However, it's important to highlight that there are other AI capabilities, like predictors and categorizers, that can produce significantly more value, particularly in enterprise settings. But, these capabilities aren't new; they have been around for quite some time and have proven their worth in many business applications. Predictors, for instance, are excellent for forecasting numbers or categories based on historical data, while categorizers excel in sorting data into predefined groups. Both play a vital role in enhancing efficiency and decision-making in businesses, demonstrating that while generative AI is indeed captivating, it is not the most valuable AI player.

Key Takeaways:

1. Generative AI vs Other AI Models: While generative AI has garnered a lot of attention and hype, there are other AI models, such as predictors and categorizers, that can offer substantial value in enterprise settings. 
2. Practical Applications of Predictors and Categorizers:
   • Predictors: Used for predicting numbers or categories based on historical data. 
   • Categorizers: Used for categorizing data into predefined categories. 
3. Bridging the Gap for Business Users: There is a need to make AI more accessible to business users, not just data scientists. 
4. Data Quality and Availability: Successful implementation of AI models requires good quality data.
5. Building Trust in AI Models: For AI models to be successfully adopted, users need to trust their predictions and recommendations. 
6. Starting with AI in Business: Businesses looking to implement AI should start by identifying processes that can benefit from predictors and categorizers. 

Questions for Reflection:

1. Identifying Opportunities for AI: In what areas of your business could predictors and categorizers be applied to improve efficiency or decision-making?
2. Building Trust in AI: How can you involve business users in the AI implementation process to build trust and ensure the accuracy of the AI models?
3. Data Quality and Preparation: What steps can you take to ensure that you have access to clean and relevant data for training your AI models?

More at krista.ai

Explore how AI orchestration is revolutionizing Third-Party Risk Management (TPRM). Learn how combining AI technologies, like document understanding, NLP, and generative AI, with process orchestration improves risk management practices. Discover how AI can read vendor documentation, complete assessments, identify risk outliers, and provide comprehensive data processing in minutes rather than weeks. Sam Abadir explains how this automation not only enhances risk awareness but also significantly reduces costs and improves efficiency. Using an AI-led orchestration platform to automatically populate assessments and create issues helps your risk managers perform more assessments in less time therefore increasing your risk awareness.

More at krista.ai

The Pressing Demand for Generative AI in Enterprise

Generative AI (GenAI) is promising unparalleled advancements and efficiencies for many types of use cases.  Boards and CEOs continue to experiment with the technology and imagine how it can improve workforces and increase throughput. The Wall Street Journal highlights how CEOs are pressuring CIOs and technology leaders to urgently install generative AI for fear of being left behind and CIOs are feeling the heat. However, with the dynamics and complexity of adopting generative AI in enterprise settings, it becomes clear that managing expectations is just as important as it is about technological integration.

Generative AI Sets False Expectations

The simplicity and efficiency of generative AI in personal use often paint an unintentionally misleading picture in an enterprise setting. When CEOs and other non-technical leaders personally interact with tools like ChatGPT, they're introduced to the potential of the technology in an uncomplicated, straightforward context. This magical experience often sets false expectations, leading them to question why such technology isn't already integrated into the broader systems of their companies. However, the reality is that scaling these tools for enterprise needs is a vastly more intricate process. It's akin to the difference between cooking a meal for oneself versus catering for a large event with complex dietary restrictions; the underlying task is the same, but the scope and complexity are dramatically different. This lack of understanding between personal uses and the intricacies of enterprise deployment highlights the need for clearer communication about the capabilities and limitations of AI tools in a business context.
The Intricacies of Enterprise Implementation

Deploying generative AI in an enterprise setting is more than meets the eye. While individuals might find generative AI to be a convenient solution for isolated tasks, integrating it within a business's broader systems demands addressing a series of complex challenges. As John points out while a user might see generative AI as solving 100% of a personal problem, it only covers about 5% of the challenges in a business context. The vast majority of the work comes from:

• Content ingestion: Importing data correctly is a massive challenge, especially when dealing with varied content like text, tables, images, and metadata. Properly importing, categorizing, and managing this data is a colossal task that requires precision to ensure you prompt an AI model with the right context and information.
• Real-time access: Unlike personal use scenarios, where static data is sufficient, enterprises operate in dynamic environments and require real-time data, which means integrating AI models with existing systems in a nimble and adaptable method.
• Data security:  Enterprises deal with vast amounts of sensitive data, and any AI model must operate securely within existing frameworks, ensuring that access is limited to only the appropriate roles and parties.
• Scalability and cost: Experimenting with public interfaces is free or inexpensive but deploying these models at scale can be extremely costly so enterprises need to be able to manage these costs and justify the investments.

The journey towards integrating generative AI in your enterprise is simple if you plan effectively and leverage the right tools. It involves more than simple adoption—it demands understanding, strategic planning, careful deployment, and continuous assessment. With the right approach, clear use cases, strong data governance, skillful training, and vigilant monitoring, generative AI can be effectively integrated to drive considerable value to your business, fostering innovation, and giving your organization a competitive edge.

More at krista.ai

In an era where Artificial Intelligence (AI) is transforming business processes, companies face a significant dilemma: whether to hire expensive AI specialists or pivot towards more cost-effective and scalable options. With industry giants like Netflix offering compensation packages as high as $900,000 for AI Product Managers, the perceived value of AI expertise is high. However, smaller companies wanting to apply AI might be scared off by these numbers, raising questions about the viability and affordability of such roles. This article explores the financial implications of implementing AI, the challenges of AI adoption, and the benefits of choosing the right data platform for AI implementation. We delve into some AI use cases managed through automation versus those handled by specialized AI engineers, and describe the role of platforms like Krista in making AI accessible to companies regardless of their size.

More at krista.ai

Different LLMs have varying strengths and weaknesses. Understanding the strengths and weaknesses of different language models is crucial, as it allows you to select the most efficient tool for your specific needs. When you evaluate and test various LLMs, you are essentially comparing their ability to answer different types of questions, their performance, and their cost-effectiveness. This comparison isn't merely an academic exercise but a critical step in identifying which model can best serve your needs in real-world applications. This process is straightforward and achievable with the right tools and guidance, so everyone can benefit from the advancements in artificial intelligence.

How to Evaluate and Test Different LLMs

Now that we have identified some key factors to consider, let's look at how you can evaluate and test different LLMs in a matter of minutes.

1. Start by identifying your specific use case for an LLM. This will help you narrow down the list of available options. A great first use case is an employee assistant since it is straightforward and more than likely you have the data to support the test. We chose this use case in Comparing Large Language Models for Your Enterprise: A Comprehensive Guide.
2. Gather the documents related to your use case. These documents will serve as the basis for generating questions to test the LLMs. If you want to run a similar test to ours you can use your employee handbook or another resource that you are familiar with.
3. Import your documents into Krista and write down the questions that you would like to ask of your data. If you have FAQs available based on your document, then you can use those questions, different ones, or a combination.
4. Choose 2-3 LLMs that you want to compare and run each question through them, recording their responses. Krista provides you with a conversational interface to ask questions about your document sets. 
5. Evaluate the accuracy, performance, and cost of each model's responses. 
6. Consider any other relevant factors, such as data security, when making your final decision.
7. Repeat the process with different sets of questions and documents to thoroughly test each LLM. If you want to connect a system like a ticketing system, CRM, or email inbox, contact us for help.

More at krista.ai