The Union

The prompt engineer. Do you need one? Can you train your people to generate prompts? Can this be automated? Well, it depends.

LLMs, like BARD, OpenAI GPT-3.5 and 4, Llama, or Flan, require a diligent and persistent approach to testing and refining prompts. Their variable performance for different use cases reveals the critical need for consistent testing and evaluation to achieve accurate results. Therefore, prompt engineering isn't a static task. It's an evolving practice that isn't an exact engineering principle. It's part science and part art, therefore it requires constant attention and refinement. Whether you're a seasoned professional in the field or just starting out, this article and corresponding podcast offer examples, and valuable insights into the challenges and solutions in prompt engineering.

More at krista.ai

To effectively limit LLM hallucinations, you need to treat LLMs more like journalists instead of storytellers. Journalists weave stories using factual, real-time information. Similarly, you need to feed your LLMs with real-time information to ensure their generated content aligns more closely with reality. Storytellers on the other hand don't require real-time information, since they are creating tales in a fictional world. 

In our experience, LLMs are primarily used to distribute static content, but static content queries only cover about 20% of the answers users seek. The majority of queries require real-time information. For instance, asking for a current cash forecast at 10 a.m. on a given day will have a different answer hours, if not minutes, later. Or, "Which sales opportunities have a chance of slipping into the next quarter?" Answers to these questions reside in your finance and accounting or customer relationship management software systems. You can't train an LLM on this fluid data but you can prompt an LLM with the data from these systems by integrating an LLM with your backend systems. This in essence will limit LLM hallucinations since you are prompting it with your real-time data to generate an answer that contextually makes sense to the person asking the question given they have permission to read the data.

Moreover, when individuals pose questions like these, they often aim to initiate a whole workflow. For instance, the sales opportunity question mentioned earlier about which deals may slip, cannot be resolved by referencing static content. Such requests will involve triggering other systems or human workflows that fall outside of an LLM's purview. A sales manager or chief revenue officer will seek to initiate some type of action if deals are slipping so they can maintain the sales forecast. They may want to offer a discount to accelerate a deal or offer an incentive if excess inventory is available. 

It's essential to realize that while LLMs are an important part of the solution, they are not the entire solution. To handle queries related to static content, real-time information, and integrated systems or workflows, you need to marry LLM capability with other systems. With this integrated approach, you can limit LLM hallucinations, ensuring the AI system provides more accurate and beneficial responses.

More at krista.ai

In this episode of The Union podcast, Chris and I review McKinsey &Co.'s "The state of AI in 2023: Generative AI’s breakout year," exploring its impact on various industries and job functions and how it's shaping the future of work.

How High Performers Achieve Results with Generative AI

McKinsey's report summarizes 1684 survey responses and highlights how companies are using generative AI, some of the opportunities they are taking advantage of and several concerns with the technology and how it integrates into applications and workflows. The report authors split the responses and highlight how differently high performers are leading the way in the use of AI. High performers are organizations that, according to respondents, attribute at least 20 percent of their EBIT to AI adoption. They're able to quickly identify opportunities and capitalize on them with a data-driven mindset, leveraging AI solutions to rapidly develop new products, services and processes--not just save money.

High Performers Are Using Generative AI Differently

How are high performers achieving a 20% lift in earnings with generative AI? This is a big lift and demonstrates that AI is truly transforming businesses. Interestingly, the survey showed that high-performing organizations state they focus on building new products using AI rather than cost savings. Building new products opens adjacent markets or enables one to grab more share and spend from current customers. I'd like to see this data further segmented by industry or use cases since they are getting such a lift. I assume that the high performers are likely to be companies with dedicated data science teams who understand how AI can contribute to product development. Companies just starting with AI might be more focused on potential cost savings, given their lesser familiarity with AI's potential.

Concerns with Generative AI are Consistent

As with any technological advancement, there are apprehensions surrounding generative AI adoption. Is it secure? Are the outputs accurate? Where does my data go? These are valid concerns and the report states generative AI inaccuracy, cybersecurity, and IP infringement as the top concerns. However, these risks can be mitigated by using AI integration platforms like Krista to help govern data, and prompt generative AI with data and answers from your internal systems thereby reducing inaccuracies or hallucinations, and cybersecurity risks.

How Are Companies Using Generative AI?

Of the individuals surveyed for the report, most have used generative AI at least once and one-third use it regularly. Industries using generative AI the most include tech, media, and telecom sectors followed by financial services and other business services. This suggests that the more data-intensive an industry is, the higher the usage of AI, and therefore a greater opportunity to make sense of the business and build new products.

More at krista.ai

Companies evaluating AI want the most accurate machine learning models (ML) to predict or summarize business outcomes. However, many times achieving high levels of accuracy and confidence inflates expenses and erodes value versus adding to it. In this episode, Luther Birdzell and Scott King discuss what is and what is not AI, evaluating ML model accuracy against costs, and how to maximize your ROI when building or maintaining your models.

More at krista.ai

Embracing Predictive Analytics in Decision-Making

In this episode of The Union Chris and I discuss predictive analytics and how to use it to help you make better decisions. We explored how advanced analytics can help business process owners and IT professionals boost their decision-making prowess. The most important topic in this episode is using machine learning to build machine learning. It's possible to capture and interpret business process data to streamline and improve them over time. This capability helps automate decisions that, until now, many may not have recognized as automatable.

Automating HR Processes with Machine Learning

In a given process, predictive analytics can supplement or could replace a manual decision with an automated one, increasing efficiency and speed. For instance, when an employee requests vacation, a machine learning model can evaluate multiple variables rather than a manager looking up the same information - such as the employee's leave balance, work capacity needs, and crew shift patterns - to make an informed decision about if or when the employee can take a leave of absence.

Enhancing Cash Flow Predictions

We also discussed an intriguing case in finance where predictive analytics helped improve the decision-making process. Accounting can tally the invoices sent out, but it's finance that often struggles to predict when these invoices will be paid. Machine learning can analyze patterns in past behavior to predict when a customer is likely to pay an invoice. This insight can assist finance in determining cash flow so they can pay invoices or invest. This kind of prediction is impossible to achieve manually, especially for large organizations with thousands of vendors. With machine learning assisting in evaluating data and helping predict outcomes you can boost accuracy and get a better grip on cash flow forecasting.

Strategic Decision-Making with Predictive Analytics

Then we tackled the topic of strategic decision-making. Companies need to move beyond using predictive analytics for making isolated decisions. Instead, they should harness it to drive strategic actions. For instance, once finance knows which invoices are likely to be paid soon, they can offer an early payment discount to accelerate cash flow. It's all about leveraging the predictive power of analytics to achieve business objectives and streamline operations.

Envisioning the Future of Predictive Analytics

Looking ahead, predictive analytics will continue to enhance business decisions as companies move processes to software. As more organizations recognize its potential, we'll see a shift from simply making faster decisions to automating actions based on these decisions. The future is about integrating predictive analytics within business processes to drive automated, data-driven outcomes to improve the ways businesses operate.

The Indispensable Value of Predictive Analytics

Predictive analytics and automation have the potential to automate decision-making, predict future trends, and drive strategic actions, all of which will revolutionize the way businesses operate. Embracing predictive analytics is no longer a choice; it's imperative for those seeking to boost efficiency and enhance both employee and customer digital experiences.

More at krista.ai

There's a lot of AI buzz and businesses everywhere are feeling the pressure to integrate this exciting technology. However, amidst all the hype, there's a need to clarify a few things. First off, AI isn't a panacea for all business woes. It needs a foundation of sound data and processes to be effective and it's intended to assist rather than replace human decision-making.

The power of automated workflows

A major component that tends to get lost in the AI conversation is the power of business workflows. These are the cogs and gears that keep an enterprise running smoothly, but they're often overlooked. Consider a scenario where a company needs to manufacture a specialty item for a valuable client that isn't on the standard price list. How can you effectively orchestrate such a process that is outside of the norm? Different decisions will need to be made. Other concurrent processes will need to be modified or stalled. How do you keep everyone informed in such a situation to make sure you provide superior customer service at the lowest costs?

Your people cause process delays

Now, let's extrapolate this to the reality of remote work and multiple stakeholders in a process. Many times the complexities of business operations across time zones or business units can lead to inefficiencies, especially if the responsibility of coordination falls on a single individual. Moving well-orchestrated workflows to software could greatly simplify these processes, allowing you to keep your global operations moving at machine speed.

In the absence of well-orchestrated workflows, we run into several issues. For instance, if someone is on vacation, who steps in to approve and follow up on their tasks? Without a well-defined process, tasks could easily fall through the cracks, leading to delays and errors.

Automating communication offers immediate ROI

But there's a solution within your reach - intelligent automation. Intelligent automation offers immediate ROI by identifying areas within business processes that can be streamlined. By automating tasks, you set the stage for AI integration, thereby further enhancing your operations. There's a wealth of evidence supporting the ROI of automation, from case studies to quantitative data. This prepares you for your iterative AI journey and paves the way for a smarter, more productive workforce.

You need good processes and data for AI

To sum it all up, while AI is undoubtedly a game-changer, we need to keep our focus on the big picture. AI can help us make better decisions based on good data, but the underlying processes that service our customers and employees are just as crucial. We can make immediate impacts by taking the time spent on emailing and follow-ups and transferring them to software. By focusing on orchestrating our workflows, we can improve our efficiency, better serve our customers and employees, and make room for transformative technologies like AI.

More at krista.ai

Generative AI is transforming how data and content are generated in many types of use cases. Its ability to quickly summarize data and enable people to interact with systems in a conversational manner is redefining what work is and the value associated with it. In a recent conversation with Chris Kraus, we reviewed McKinsey & Co.'s, "The economic potential of generative AI: The next productivity frontier" on how generative AI is shaping different roles across several industries, particularly in customer service, labor cost efficiency, and the implications for high-value work.

Generative AI assists in front and back office tasks

The line between back office and front office operations is blurring as generative AI takes on a more significant role. AI's ability to assist in automating tasks previously managed by humans in areas such as customer service is remarkable. For example, consider the scenario where AI meticulously examines incoming documents in a loan application process, distinguishes them, and then forwards them to a loan officer for approval. AI aids professionals by automating routine tasks, allowing them to focus on higher-priority matters.

Generative AI ROI

One cannot overlook the financial implications of AI. In certain regions where labor costs are low, the financial burden of resolving a problem with AI may outweigh the cost of human labor. However, relying on humans in this context merely postpones the inevitable adoption of AI.

High-value work, which is the cornerstone of any successful business operation, deserves our attention. Every employee is often bombarded with requests from peers and external stakeholders. Amid these demands, it's important to prioritize. Generative AI can help by taking on tedious tasks, freeing up individuals to focus on high-value projects that require creative and critical thinking.

Generative AI innovation accelerated

When reflecting on McKinsey's previous reports, the advancement of natural language understanding technology surpasses all expectations. Previously, experts predicted that AI technology capable of human-like tasks would emerge in 2027. However, as of 2023, we are witnessing this transition happening right now. This leap signifies a four-year acceleration, which has tremendous implications for businesses worldwide.

What are the practical uses of Generative AI?

The question remains: How should businesses perceive and respond to this accelerated development? While some might dismiss it as mere hype, the practical use cases of AI, as outlined in various research papers, are too compelling to ignore.

In one instance, a Krista partner who sought ten potential use cases for AI from customers ended up identifying a hundred viable scenarios, each with a distinct dollar value. With the rapid advancement of technology, waiting and watching is no longer an option. If you don't adopt AI now, your competitors will, leaving you trailing in their wake.

More at krista.ai

Artificial intelligence (AI) has an immeasurable impact on various industries, from finance to healthcare to customer service. It can automate repetitive tasks, derive insights from massive data sets, and even help manage and govern data. However, effectively governing AI data requires a well-thought-out strategy and proper implementation. Chris Kraus and I discussed the importance of data governance in AI and how to effectively manage it in this episode of The Union.

AI Should Involve People in the Same Context

Any AI conversation should maintain a 'shared context'. A shared context refers to the ability of an AI system to maintain a consistent understanding of a situation across multiple interactions and even multiple users. For example, in a customer service scenario, a shared context would allow a customer service agent to pick up where a previous interaction left off, saving the customer from having to repeat information. 

Maintaining a shared context across multiple interactions is crucial since customers may require help from multiple systems or people spanning several sessions. Therefore, AI systems should be able to manage long-running conversations and provide previous knowledge. Many conversations are not resolved on the first attempt so any AI system should be able to recognize that it is the same conversation and maintain the same context. 

AI Should Know Who Is Allowed to Know What!

When implementing generative AI for customer or employee assistants, it is critical to understand who has access to what data and for what reasons. With many different stakeholders involved, from employees to managers to end users, you cannot afford to expose data to those who should not have access to it.

For instance, a call center agent may need to consult with a manager to approve a customer interaction or an offer. This decision may involve accessing customer data that the agent does not have the privilege to view. An automated AI workflow can handle this situation by learning how such decisions have been made in the past and using that knowledge to decide on its own, without the need for human intervention. However, it's essential to ensure that AI does not provide answers to those who should not have them, maintaining data privacy and security. You wouldn't want employees to ask the HR or payroll systems questions about employee health, benefits, or salary information. 

AI Conversations Should Transfer from Chat in a Browser to Mobile Platforms to SMS

Today's customers expect a seamless experience across all platforms, whether it's desktop, mobile, or SMS. By enabling 'long-running conversations' across these platforms, businesses can enhance omnichannel customer experiences while also governing the data that supports them.

Effectively governing AI data involves maintaining a shared context across many people, understanding who is allowed to access what data, and managing data across multiple user interfaces. By focusing on these areas, you can reap the benefits of AI while also ensuring your data is managed effectively and securely.

More at krista.ai

Generative AI possesses great abilities to craft impressive content. Its capacities in generating text, photos, and videos, and engaging in conversational interactions are remarkable, yet they represent only one step in a customer's or employee's quest to complete a process. Imagine a customer reaching out to an organization for assistance. Generative AI can answer this person if its model supports the request or it can suggest another method. However, its true potential can only be exploited when it transcends conversational limitations and takes definitive actions to help you actually "do" something.

We must view AI not just as an intelligent service that generates content, but as an operative tool that propels actions and leads to substantial business outcomes. AI should be more than an impressive chatbot; it must guide your customers and employees based on the next best actions for the outcome they are seeking. This is a fundamental concept in the realm of AI: operationalization and optimization. And, this AI shouldn't merely exist in the realm of conversations but rather connect to real-time systems, actively engage with multiple people, and most importantly, facilitate actions that drive positive business outcomes.

This journey of operationalizing AI starts with a simple, yet important, concept. Eventually, you will want customers and employees to have one-on-one, one-to-many, or many-to-many conversations with your real-time systems. Doing so provides them the capability to complete an outcome by themselves. In essence, what we are striving for is an AI automation system that does more than just chatter—it must "do" things. Chris and I delve into this intriguing aspect of AI in this second of three episodes, breaking down AI complexities and understanding its capacity to "do" things, which ultimately leads to measurable outcomes.

More at krista.ai

Implementing generative AI or any other form of AI within a business framework demands more than mere installation. Businesses need to be confident that their data remains secure and that there are role-based administrative procedures present in managing the least privilege. The AI should be capable of synchronizing with current systems in real-time to provide pertinent information and precise responses to users in any format. In instances where a response or conversation involves additional personnel, the AI should maintain the same context for all of the parties. This method ensures everyone comprehends the narrative and collaborates towards the identical objective. Most of all, businesses need to connect and change AI services almost instantly. Businesses cannot continue with slow and resource-intensive deployment and integration cycles.

This is the first of three podcasts where we explore quickly connecting AI solutions. This first episode discusses three enterprise requirements for connecting generative AI into your business to improve employee and customer experiences.

1. You must connect generative AI to enterprise systems in real-time 
2. You must be able to instantly integrate and interchange different AI tools and services
3. You must be able to efficiently synthesize vast data volume

More at krista.ai