Knowledge Graph Insights

Juan Sequeda

Knowledge graph technology has been around for decades. The benefits so far accruing to only a few big enterprises and tech companies.

Juan Sequeda sees large language models as a critical enabler for the broader adoption of KGs. With their capacity to accelerate the acquisition and use of valuable business knowledge, LLMs offer a path to a better return on your enterprise's investment in semantics.

We talked about:

his work data.world as Principal scientist and the head of the AI lab at data.world

the new discovery and knowledge-acquisition capabilities that LLMs give knowledge engineers

a variety of business benefits that unfold from these new capabilities

the payoff of investing in semantics and knowledge: "one plus one is greater than two"

how semantic understanding and the move from a data-first world to a knowledge-first world helps businesses make better decisions and become more efficient

the pendulum swings in the history of the development of AI and knowledge systems

his research with Dean Allemang on how knowledge graphs can help LLMs improve the accuracy of answers of questions posed to enterprise relational databases

the role of industry benchmarks in understanding the return on your invest in semantics

the importance of treating semantics as a first-class citizen

how business leaders can recognize and take advantage of the semantics and knowledge work that is already happening in their organizations

Juan's bio

Juan Sequeda is the Principal Scientist and Head of the AI Lab at data.world. He holds a PhD in Computer Science from The University of Texas at Austin. Juan’s research and industry work has been on the intersection of data and AI, with the goal to reliably create knowledge from inscrutable data, specifically designing and building Knowledge Graph for enterprise data and metadata management. Juan is the co-author of the book “Designing and Building Enterprise Knowledge Graph” and the co-host of Catalog and Cocktails, an honest, no-bs, non-salesy data podcast.

Connect with Juan online

LinkedIn

Catalog & Cocktails podcast

Video

Here’s the video version of our conversation:

https://youtu.be/xZq12K7GvB8

Podcast intro transcript

This is the Knowledge Graph Insights podcast, episode number 19. The AI pendulum has been swinging back and forth for many decades. Juan Sequeda argues that we're now at a point in the advancement of AI technology where businesses can fully reap its long-promised benefits. The key is a semantic understanding of your business, captured in a knowledge graph. Juan sees large language models as a critical enabler of this capability, in particular the ability of LLMs to accelerate the acquisition and use of valuable business knowledge.

Interview transcript

Larry:

Hi, everyone. Welcome to episode number 19 of the Knowledge Graph Insights podcast. I am really delighted today to welcome to the show Juan Sequeda. Juan is the principal scientist and the head of the AI lab at data.world. He's also the co-host of the really good popular podcast, Catalog & Cocktails. So welcome, Juan. Tell the folks a little bit more about what you're up to these days.

Juan:

Hey, very great. Thank you so much for having me. Great to chat with you. So what am I up to now these days? Obviously, knowledge graphs is something that is my entire life of what I've been doing. This was before it was called knowledge graphs. I would say that the last year, year-and-a-half, almost two years now, I would say, is been understanding the relationship between knowledge graphs and LLMs. If people have been following our work, what we've been doing a lot has been on understanding how to use knowledge graphs to increase the accuracy for your chat with your data system, so be able to do question answering over your structured SQL databases and how knowledge graphs increase the accuracy of that. So we can chat about that.

Juan:

But what I'm interested next now is I'm calling this how do we extract knowledge from people's heads? And this isn't new, this is not new, this is back from the '80s and early '90s, all the work on knowledge acquisition. People wanted to create rule-based systems and expert systems, and you need to have knowledge engineers, who go talk to the domain experts and be able to extract their knowledge and codify that. Well, that was hard, and it was brittle, and it was not scalable. And now we have a very fantastic tool called LLMs that I'm very excited to use them to be able to find ways to scale that back. So I think what's old is new, and I think the opportunities to use these new tools to do the work that we've wanted to go do, but we haven't been able to go do, which is just to improve knowledge management.

Juan:

And one of the reasons why I argue that, we still try to solve the same problems. We're solving the same problems today that we had 30 years ago, but the core problems continue to be there, and I think it's this lack of knowledge. Now, we have a way to scale this out so people can stop complaining, saying, "Oh, it's just too hard, too expensive." That's my long-winded answer of what I'm up to.

Larry:

Yeah, well, I'd love to jump right into that, because there's all these intersecting things, at least in my world. I run in a lot of different worlds, the content world and knowledge management folks. Anyhow, like you said, in the AI world, there's this longstanding, the expert systems and all that kind of stuff, and then there's the knowledge engineers and enterprise knowledge management stuff. It's always been really hard work, but parts of that are now more amenable to, if not automation, at least acceleration. Can you talk a little bit about how LLMs have kind of advanced the stuff you've always been doing and always wanted to do more of?

Juan:

Well, so LLMs, first of all, we look at the LLMs, and I like to look at things like as inputs and outputs. So the input is a prompt, and the output is going to be some text around this prompt. That's at a super high level. But what's been fantastic is that if you give it some context as an input, and part of that context, you also give it a task of what you want it to go do, the output that comes out actually aligns very nicely to it. Now, yes, there's hallucinations and stuff, but look, if it gets 80%, 90% done, that's better than nothing. It's better than actually doing things completely manual. And guess what, if a human does it, they are not always getting things 100% correct anyways. So that's at a super high level.

Juan:

So the example I give all the time is organizations always complain, "Oh, how many customers do we have? Oh, we have so many different customers. It's so complicated." Okay, so what? You're just going to cross your hands and just live like that? No. How about you figure out at least how many different ones are there, and what are the implications of those things? But how would you do that? Well, you need to go start talking to people, and then somebody needs to go talk to all these people and start cataloging and combine all this stuff. Well, what if I actually just posed that question to somebody, saying, "Can you describe what is a customer for you in a sentence or in 10 seconds, in 15 seconds, right? Give me a minute rant of what is a customer for you?" Just take that, and you got a transcript, you got text right there.

Juan:

So now, I could prompt the LLM to say, "Hey, here's this definition of a customer. Can you please give some definitions?" Here's where the prompting, this is where the interesting experiments are going to come in. And then you can now do that for so many different other people. And then you can do further analysis to figure out, saying, "Hey, where are the overlaps of this stuff? Actually, you know what? Can you codify this? Can you turn this into code? Can you turn this into already some sort of a knowledge graph on it?" And I can now start comparing these things and realize, "Oh, look. There's really X different definitions of this, where the alignment is on 80%, and the 20% of this." Now we know these things. Now I can actually go back. That's the type of stuff that we can actually go do very quickly in the initial experiments that I've been doing. It's like, this stuff is working so freaking well.

Juan:

I wish we had this back 20, 30 years ago, but now it's time to go look at the problems that we're trying to go solve there. But back then and now we can actually solve them with these new tools. So that's what I'm really excited about and this is something that I really hope, one, I want to show that this can be done, but second, but most importantly is the so what? Once we've done this, what is the value for an organization? So I think those are the ties that we need to go do still. It's like we all complain about we don't know the definition of the customer. So what is the problem for that? What's the money that we're leaving on the table? How much money are we losing by not having a complete definition of a customer? Now that we do have that, we need to be able to say, well, I do have that now. How is that improving? I think that's really important for the next step.

Larry:

So we're a couple of years into this now, where is the value emerging? You've talked a lot about the workflow in making knowledge graphs and capturing knowledge, but how does that percolate up to business value? Where are folks finding that payoff?

Juan:

You mean from knowledge graphs in general or...?

Larry:

Well, just from this whole scheme you're describing, the ability to accelerate understanding of knowledge and what you just described is what I do. I've been doing content modeling for many years and it's sort of very similar kind of work and a lot of that back and forth. I've already seen it accelerated in my content modeling practice as well. And so the way you're talking about,...

Jesús Barrasa

Over his 20-year career, Jesús Barrasa has spanned the worlds of object-oriented property graphs and assertion-based knowledge graphs.

He knows as much about these two foundational technologies as anyone and offers pragmatic advice to help architects and engineers decide which approach will work best for their needs.

We talked about:

his role at Neo4j in which he helps companies adopt graph technology

his academic study of semantic technology and his early work on mapping relational data to ontologies and enterprise ontologies

his move across the graph spectrum from RDF graphs to property graphs, culminating in his current role at Neo4j

his take on the similarities and differences between RDF and property graph approaches, the key commonality being linked data and the key distinction between them being the level of abstraction that they employ

different ways to approach inference

the origins of the semantic web and how it has made data actionable and interoperable and smarter

how the professional backgrounds of software developers can affect their choice of graph technologies

the crucial role of interoperability in graph technology, and our ongoing inability to productively harness it

how semantics is managed and used in the property graph and RDF worlds

ontology as a technology-independent way of representing knowledge

the importance of staying focused on the needs of practitioners when advising them on how to make a graph technology choice

how knowledge graphs can balance the "opaque power" of large language models with "explicit, declarative, explainable power"

Jesús' bio

Dr. Jesús Barrasa is Neo4j's AI Field CTO and the company's resident expert in Knowledge Graphs and Semantic Technologies. He co-authored the O'Reilly book "Building Knowledge Graphs: A Practitioner's Guide" (released in July 2023) and combines over 20 years of professional experience in the data management space split between industry and research and academia.

Prior to joining Neo4j, Jesús worked for data integration companies like Denodo and Ontology Systems (now EXFO), where he gained first-hand experience with many successful enterprise-wide data integration deployments and large graph technology projects enhancing the operations and analytics of major companies worldwide.

Jesús' doctoral work in Artificial Intelligence and Knowledge Representation focused on the automatic repurposing of legacy data as knowledge graphs. He's an active thought leader in the graph and semantics communities and co-hosts the popular monthly webcast on knowledge graphs "Going Meta."

Connect with Jesús online

LinkedIn

Going Meta webcast.

Video

Here’s the video version of our conversation:

https://youtu.be/7WFP_oDQsxI

Podcast intro transcript

This is the Knowledge Graph Insights podcast, episode number 18. If you search for the term "knowledge graph," you're likely to get an equal number of results about property graphs and RDF-based graphs. Jesús Barrasa has been immersed in both of those technologies for more than 20 years. He takes a pragmatic approach to graph technology adoption, focusing on the needs of practitioners and on the ability of knowledge graphs to balance the "opaque power" of large language models with the explainable power of knowledge graphs.

Interview transcript

Larry:

Hi everyone. Welcome to episode number 18 of the Knowledge Graph Insights Podcast. I am really delighted to welcome to the show Jesús Barrasa. Jesús is about as graph-ey a person as it gets. I got to say he has a 20-year background in this stuff. He's currently the AI field CTO, Chief Technical Officer for the graph database company Neo4j. Welcome to the show, Jesús. Tell the folks a little bit more about what you're doing these days.

Jesús:

Hi Larry. Thank you very much. I'm really, really glad to be here. I mean, we've been trying to plan this for a while now. Really, really happy to have this conversation today. So yeah, you're right. I'm with Neo4j. And well, I've always been part of the... We call the field organization so we were not part of building our product, but helping our customers adopting it and adopting graph technology in the general case. These days we have a very, very strong focus on as it's inevitable in how we integrate with large language models, GenAI, which we'll talk a little bit about later on in the show but that's what we do. So basically I work all over the globe with all of our customers, many of our customers of course, and helping them adopting graph technology. So that's where I am today.

Larry:

That's great. This show, I really like to focus on adoption and use and practice around graph technology and you know as much about it as anyone. I don't always start these episodes with a biography, but your background is so interesting. Because I think virtually every guest I've had to this point comes out of the RDF-based, ontologically driven knowledge graph world. And you come out of that world originally, but you're currently in the property graph space. So tell me a little bit about your pathway from your original study and your career.

Jesús:

Yeah, absolutely. And that's going to age me I guess. But yeah, you're right. Well, I started many others as a university graduate. I went on and I did computer science and started doing software engineering. But what it really starts for our conversation today is when I decided to join the, it was called the Ontology Engineering Group in Madrid and started my PhD. And that's when I met these people that were using these super interesting technology that was called Semantics, the Semantic Web. Of course it was back in the day when Berners-Lee and company published the famous article in Scientific American.

Jesús:

And that was amazing. I really, really enjoyed it and finished my PhD and this was the early 2000s, so around 2008 or something like that. And what I did basically was coming up with a way of mapping relational data. I don't want to go too nerdy too early in the conversation, but mapping relational data. Which is basically where all the data lived to ontologies. Basically overlaying some form of semantic description of the meaning of the data. And like many others we try to come up with a declarative, structured mapping language to be able to basically leverage in the semantic web the content from relational databases. So that was great fun. And that's work that then later on people like Juan Sequeda, who we talked about earlier today followed and even built a company around it. But that's what I did.

Jesús:

And then from there on, I moved to the UK to London to join a company called Ontology that's still around under a different name that we're using the RDF stack. And they were focusing on the telecoms verticals. So we were building connected representations of, I would say all the elements involving the delivery of a telecom service. I mean from the physical infrastructure to the logical elements. And the services, the products, even the customers. And we were trying to solve problems like impact analysis, root cause analysis. So we did that for a few years and that was again, great fun.

Jesús:

And that's where I jumped to other side of the graph, let's call it, camp. Or let's call it spectrum, because more kind of a gradual thing. And a short stint at a data integration company called Denodo. Then I joined Neo4j. And I've been with Neo for the last nine years, so quite a while. Which means that I've been doing graphs for close to 20 years which is crazy. And like you say, I'm one of these unusual individuals out there that has spent as much time doing RDF and SPARQL as I have been using a property graph and Cypher, which is great. That puts me in a great place for conversations like this one. And it's been great fun. So that's kind of a bit of an overview of what I've been doing over the last over 20 years.

Larry:

And that's really... Because the way you just said that you're reminding me of, it's just data. And you're getting at it in different ways. You're querying it with SPARQL in some cases in Cypher in another context. But as much about the distinctions or how to navigate that spectrum from property graphs to RDF-based knowledge graphs. How would you describe the similarities? If you do a Google search for knowledge graph, it's a mashup of property graph and RDF-based graph stuff. How would you distinguish the two? And what unites them and what separates them, I guess?

Jesús:

Sure, yeah, I know, absolutely. And you're totally right. So I don't think we always do a great job at helping practitioner, which is ultimately what should be our main objective. So yeah, I mean the way I see it is they share the most important aspect, which is they have the same underlying abstraction, which is connected data. So we think of the world not in terms of tables or documents. We think of the world in terms of things connected to other things. Because that's how humans think. So we think of Larry is a person, Jesús is a person and we're friends and we're connected through a friendship relationship and we work for companies. And the world is a collection of things related to things.

Jesús:

Now, if you go down the RDF stack approach you're going to break down that into individual atomic statements that are called triples, which is what RDF is based on. And you would put it in your choice of model would typically, not always it shouldn't be and we can happy to talk about that later, determine kind of a technology stack. But ultimately it's kind of a level of abstraction. If you go the RDF route, you would describe it in terms of statements. So person 123 name Larry, person 123 lives in Amsterdam. Person 123 is connected to person 234. Person 234 is Jesús. So everything can be broken down into logical statement into triples. And that's one way, that's one approach.

...

Yaakov Belch

Yaakov Belch is an AI researcher with strong ideas about the role of humans in AI systems. Instead of "human in the loop," he argues, we should put "humans in control."

Yaakov's research looks at business contracts and how knowledge graphs and AI systems can both capture their meaning more accurately and help managers make better business decisions.

We talked about:

his assertion that we need humans in control, not just in the loop

his research on applying AI technology to business contracts, in particular the issue of resolving inconsistencies in language model results

reasons to put human concerns ahead of any particular technology

the importance of having humans in control when interpreting ambiguous business decisions

the importance of both accounting for business intent and asking the right questions of your data and how the loop between the two tightens over time

the responsibility of human users to understand how LLMs work and to prompt and otherwise interact with them accordingly

why he doesn't use the term "hallucination" when talking about LLM outputs

the role and implications of applying different kinds of logic in the use of knowledge graphs

an analogy that shows how the concept of a Git fork can help knowledge graph engineers account in their models for different versions of reality

the real-world applications of his research, especially how the practices he is exploring can create new business value

the importance of building any model off of real data and always thinking about which human needs to be in control

Yaakov's bio

As a mathematician and data scientist, Yaakov Belch brings a unique perspective to the world of AI and knowledge graphs. With a strong background in mathematics, including participation in prestigious International Mathematical Olympiads, Yaakov went on to earn a Ph.D. in pure mathematics from the University of Cambridge.

Yaakov's career has spanned both research and industry roles. He has worked as an Algorithm Programmer, collaborating with researchers in bioinformatics and economics, co-authoring academic papers. Yaakov also served as a Senior Data Scientist at Israeli e-commerce startups, where he tackled challenges in symbolic and semantic search from different angles.

Currently, Yaakov is on a sabbatical, working as an independent Data Scientist to develop his method of reliable business reasoning, precise contract understanding, and humans-in-control.ai. He sees an interesting connection between the problems from the International Mathematics Olympiads and taming the inconsistencies of large language models: "At one hand, the problems are hard and just don't open up to the known, standard methods of the field. But you know that there is a beautiful solution. You need to find the right perspective to appreciate the problem and to see that beautiful solution."

Connect with Yaakov online

humans-in-control.ai

LinkedIn

Video

Here’s the video version of our conversation:

https://youtu.be/ZS9r0bfkGQc

Podcast intro transcript

This is the Knowledge Graph Insights podcast, episode number 17. Machine learning architects often talk about the "human in the loop." Yaakov Belch thinks that when it comes to language models the right approach is to put "humans in control." Yaakov's research looks at how knowledge graphs and large language models can help put humans in control of business contracts, capturing the actual intent that underlies them and facilitating better business decision-making based on the discoveries that they enable.

Interview transcript

Larry:

Hi everyone. Welcome to episode number 17 of the Knowledge Graph Insights podcast. I am really delighted today to welcome to the show Yaakov Belch. Yaakov is an independent senior data scientist and he’s made this really provocative statement about … There’s all this talk in the AI world in general about humans in the loop. And Yaakov says, “No. We need humans in control.” So Yaakov, I would love to talk about that today.

Yaakov:

We don’t need humans in the loop. We need humans in control. This is a paradigm which crystallized over time in the research which I’m doing, about how to apply language models for contract understanding in the business context. Let’s unpack that and see where the human in control comes in. A business contract is a memorization, a written-down expression of agreement between two or more parties; with the intention to fulfil that agreement in the future. It contains provisions for what the parties promise to do, the rights and obligations they have. And also descriptions what you do when things go wrong. After you make an agreement, you actually want to execute it, you want to fulfill it.

Yaakov:

When you make a large contract, some people negotiate the contract and other people will be doing the work. There are situations where the people who actually do the work are not really aware about what exactly has been promised in the contract. The huge contract is written in a way as contracts need to be written. But it can’t be understood on the spot when the person needs to decide: Is this promised or not? There can be a disconnect between the expectations of the customer and the provider based on just not being aware of what’s in the contract. In a more advanced setting, you may want to understand the risks which are in the contract in hypothetical situations. When you draft a contract you want to make sure that you’re not setting up yourself for problems when things go in a specific way. Yet more advanced: When you purchase a property, you may acquire contracts which go with them, like liens, leases and bills. In your due diligence work, you want to check that every liability has been properly addressed, so that you don’t acquire additional risks with the property. And if there are problems, you want to resolve them.

Yaakov:

I want to understand contracts from a business perspective. I want to use language models to understand their language. It’s not a surprise anymore that language models can help with that. You put in part of a contract. You can ask a question. You can get an answer. What is problematic is that if you don’t do it right, these answers will not be consistent. You may ask the same question twice and get two different answers. Or you get an answer that is justifiable, but it’s not consistent with the intention of your question. It is against your business goals. Some answers may be completely disconnected from reality. The essence of this research project is how do we deal with this inconsistency? How do we create a reliable system where language models are just a part of it? We have knowledge graphs, we have logic, and we have humans in control … So that businesses can rely on it for their business needs. How do we detect and how do we deal with all the mistakes which happen with language models? But also in your data you have mistakes. And how do we detect them? How do we deal with them? That’s the research.

Yaakov:

In this research, I find that you really need to be careful to use the right paradigms of what you’re doing. If you just apply a playbook from machine learning or from expert systems and just try to do hard work, you may break yourself against language models being different. It’s a new technology with new characteristics. You need to adjust your goals, your paradigms of work, and the structure of your work to the capabilities and to the limitations of the problems of language models. One key point is to understand the right role that humans must play in the whole scheme. It’s not technology alone, it’s technology and people. We don’t need people in the loop. We need people in control.

Larry:

Okay. We actually had a long conversation last week to set this up and I was beginning to really get it, but that was an excellent summary of your research and the business insight and the technical challenge that comes into this. I’d love to really focus on how knowledge graphs figure into this. I’m inferring that there’s a business ontology and stuff that drives… But can you talk about the specific role of knowledge graph in how you see it helping address these new paradigms that we need?

Yaakov:

For sure. This is an excellent question, especially in the audience of Knowledge Graph Conference and knowledge graph podcast. I suggest, however, that before we talk about the knowledge graph, which is basically a technology which supports this work, to first understand: What does this really mean, human control? How does it work? What do we want to achieve? What is the goal? Then, let's understand the mistakes of language models. Their answers may be inconsistent with what you want. But they are not just making random mistakes. Where does this come from? And then we’ll see how knowledge graphs and logic play an important role. So maybe we go into three steps and get the knowledge graphs at the end.

Larry:

That sounds perfect, because that makes knowledge graph the culmination, which is what we’re looking for. It’s a knowledge graph podcast. But, no. So the first thing you said is, what does that really mean, humans in control? Yeah. I’d love to hear those three steps you just mentioned.

Yaakov:

Okay. Humans in control versus Humans in the loop. When you have a machine learning system, you know you need training data. When you get incorrect results, you want to double-check and correct them with a human in the loop. As a benefit, you create training data that will hopefully make your model more precise. But this usually doesn’t work. It works only when you do it right: When you do this with one human in control in addition to many humans in the loop. Without that, the people in the loop either don’t know the right answers, because they’re not experts. Or they are experts and don’t want to answer your question: you overload them with repetitive work,...

Michael Iantosca

Where content, knowledge management, and AI converge, you'll find Michael Iantosca.

As many in the AI world flock to probabilistic models like LLMs, Michael takes a deterministic approach to content management and knowledge engineering, using ontologies and knowledge graphs to ground content in a concrete facts.

This approach embodies his insight that content and the models that describe it are not static information but rather valuable, ever-evolving enterprise IP assets.

We talked about:

his 44-year career in content, knowledge management and localization/globalization roles

the three pillars of his work: content, knowledge management, and engineering

the need he sees in his work to move away from probabilistic, vector-based models to deterministic, neuro-symbolic models like knowledge graphs

how he decides which models are appropriate to use with each of the varied kinds of data he works with

his explorations of how to automatically construct a knowledge graph to use to power generative AI solutions

how he acquires and develops ontology skills in his team

how graph technology supports the "total content experiences" he builds

how the non-static nature of content makes it a poor candidate to be managed in a static system like a vector-based model

the relative merits and utility of 1) deterministic retrieval for structured content and 2) probabilistic retrieval for unstructured content

the power of combining content models, knowledge models, and ontologies and how they can become crucial enterprise IP assets

his belief that we are entering a golden age of content and knowledge engineering

Michael's bio

Michael Iantosca is the Senior Director of Knowledge Platforms and Engineering at Avalara, a sales tax automation company. With over four decades of leadership in technical content management, Michael has been a pioneer in advancing the profession, driving innovations in structured content, intelligent authoring, and scalable knowledge platforms. Renowned for bridging engineering and content teams, he has championed the adoption of AI and cutting-edge technologies to enhance user experience. A thought leader and mentor, Michael continues to shape the future of technical communication through his expertise and passion for innovation.

Connect with Michael online

LinkedIn

Medium

ThinkingDocumentation

Video

Here’s the video version of our conversation:

https://youtu.be/WG9Nl5OY3QI

Podcast intro transcript

This is the Knowledge Graph Insights podcast, episode number 16. A lot of work in the AI world these days is about vectorizing giant collections of static, unstructured content and data for LLMs. Michael Iantosca has worked for decades in a world where content is dynamic, always precisely structured, and contextualized with rich metadata. So he has a different take on architectural innovations like graph RAG, favoring knowledge-based deterministic retrieval of content over vector-based models and probabilistic methods.

Interview transcript

Larry:

Hi, everyone. Welcome to episode number 16 of the knowledge graph Insights podcast. I am really delighted today to welcome to the show, Michael Iantosca. Michael is currently the Senior Director of Knowledge Platforms and Engineering at Avalara, the big tax-compliance automation software company. He's also got a long history. He's spent a couple of decades, a few decades at IBM prior to his role at Avalara. Welcome to the show, Michael. Tell the folks a little bit more about what you're up to these days.

Michael:

Larry, thank you for having me. It's a pleasure and an honor to get a few minutes to talk to you today. Yeah, I have just started my 44th year primarily in the professional content space, but also in the knowledge management and localization globalization space as well. I have been involved with content since the early days of SGML that began the structured content revolution and worked my way up through the professional content ranks. I'm also an engineer in IBM's Grand Wisdom. I was trained for years as a developer, so I do span both worlds.

Michael:

My responsibility is to develop some of the world's most advanced content supply chains for creating content for customers, delivering it, as we like to say, "Deliver the right content to the right person at the right time and in the right experience." That's my north star that drives that. I lead an engineering team that builds those platforms that services multiple groups throughout the enterprise for their content creation and delivery needs, whether that's in product, user assistance, contextual help, knowledge centers, help centers, support sites, and a litany of other channels by consolidating that entire supply chain, so that we can write content once and deliver it in many different channels, including chatbots, generative AI.

Larry:

Cool. I love that ... I think, I'm trying to remember only 16 episodes in, but I think you're the most content-ey person I've had on the knowledge graph Insights podcast. One of the things we talked about before we went on the air was this notion of you're like, "Hey, guys, dear engineers, it's not just about data, it's equally about knowledge." And in this world, the knowledge graph engineers and ontologists, they're on board with that, but you also bring this content perspective to it. I'm really curious how those three concerns combine in your work. Do you approach data stuff differently with a content lens on? And especially the knowledge management part, because each of those is its whole other thing, but you're working, your title, you're a knowledge platforms guy, you're combining all three of those. Tell me how that manifests in your work.

Michael:

Yes, that's critical. We see content and knowledge management, and when I say knowledge management, I'm talking more about taxonomies and ontologies and knowledge graphs and engineering, the actual coding and infrastructure of building out models and solutions in the AI, especially generative AI space, as a holy trio, if you will, that have to have equal footing. Developing really advanced generative AI solutions is not just a coding problem. It is equally as much of a knowledge management problem and equally as much of a content challenge.

Michael:

Content isn't generic. Content constantly is changing. The management of that constant is constantly changing. We can't treat content any more generically than we can treat data that changes daily, sometimes weekly. The state of that content, the purpose of that content is not static. We need the content teams involved because they have the very fuel of our generative AI solutions, but we also need the people that understand advanced semantic knowledge management that can help power both that content and make it intelligent and then feed that to the generative AI models, so that these models can be far better than they are today.

Larry:

Yeah, and when you say today, that's going to be way different even tomorrow from when we drop this. We were talking, again, before we went on about trends in the development that's developing around the implementation of graph technologies across all of this stuff, but in particular to content. I wonder if you could talk about how you see those trends. One of the ways I've seen is from LLMs to RAG to graph RAG and now, Tony Seale and folks talking about neurosymbolic loops and hybrid AI architectures. How was that unfolding in your world?

Michael:

That's a really good question too. I think almost everybody who starts out in the generative AI space follows the same basic path. About three and a half years ago, I think it was, it took us about a week to take a simple vector database, Pinecone, I think we used, and maybe a couple of hundred lines of Python code, and we built a RAG, retrieval augmented generation model, because we didn't want to use the general large language model that uses the public content, and we didn't want to feed our public content to train a large language model. It was natural that we wanted to have a private data model of our own and use a vector database to do that. But that's really an old, old model at this stage. It is a probabilistic retrieval model and therein lies its core weakness as well. What we wanted to do was move away from probabilistic ... oh, are you there?

Larry:

Yeah. Oh, do we have an internet thing? I lost you. Let me check my internet.

Michael:

I apologize. I had a burp in connectivity.

Larry:

Oh, no worries. Okay.

Michael:

Let me pick up again.

Larry:

Yeah. Go ahead.

Michael:

What we want to do is move away from probabilistic models like relying completely on vector-based retrieval and move toward deterministic models, sometimes what we call neuro-symbolic models and use mechanisms such as knowledge graphs, which are far better at providing true reasoning and true inference based on a concrete set of facts or what we call the ground truth. I think what you're seeing now in the marketplace is the initial models that are being deployed are good. They're yielding value, people are excited. They're not perfect, but as development teams reach those plateaus, they want to get better. They want better than where they are. This is what I call the precision paradox. The precision paradox says that as models improve, the tolerance for errors and lack of accuracy or relevance or contextual truth declines. We're partying right now with these models, and then, eventually that party is going to end and we're going to have to get down and do some of the serious work necessary to move to these deterministic, reliable models that are based on ground truth that we control, not that in LLM controls.

Michael:

That's what I see the trend going to. I think, every morning I wake up, I think I read at least 10 to 20 articles, all different models and variations. Unfortunately,...

Fran Alexander

When Fran Alexander looks at the current AI landscape she sees some interesting parallels between the Alien vs Predator science fiction franchise and the way RAG and other architectures are combining LLMs and knowledge graphs.

We talked about:

the analogy she draws between the Alien and Predator science fiction franchise with LLMs and knowledge graphs

how the human-esque (if malevolent) cognitive and behavioral nature of Predators aligns more with knowledge graphs and how the unpredictable and stochastic nature of Aliens aligns more with LLMs

how the eloquence of LLM outputs can deceive humans

the lack of explainability and transparency in both Alien and LLM behavior, and the opposite in knowledge graphs

the difficulty of dealing with baked-in biases in LLMs

the lack of repeatability in LLMs and the opposite in KGs

the current trend of architectures and practices like RAG that draw on the strengths of KGs and LLMs to get better results, just as the Alien and Predator media franchises combined forces

how over the past year or so investment in LLMs has overshadowed all other investments, just as Aliens are out to wipe out anything that's not an Alien

her approach to AI architectures that combine LLMs and knowledge graphs

how different kinds of people consume LLM output

how she helps enterprise decision makers choose whether to address a use case with a knowledge graph or an LLM

how taxonomists and ontologists can use LLMs in their work

the Alien Loves Predator UK Facebook group and Alien and Predator on a seesaw

Alien and Predator cosplay actors on a seesaw

Fran's bio

Fran started her career as a writer and editor of dictionaries and thesauruses in the UK, and, as technology evolved, she specialised in information architecture, search systems, and digital archives, and more recently, the use of semantics in knowledge graphs and LLM applications. Having worked on reference publications including the Collins English Dictionary, and as Taxonomy Manager for the BBC Archive, she now lives in Montreal, Canada, and is the Senior Taxonomist for Expedia Group. She was Taxonomy Bootcamp London’s Taxonomy Practitioner of the Year 2023.

Connect with Fran online

LinkedIn

Video

Here’s the video version of our conversation:

https://youtu.be/VWwDBIws6G8

Podcast intro transcript

This is the Knowledge Graph Insights podcast, episode number 15. When two impressive domains converge, amazing things can happen. When the Alien and Predator science fiction franchises joined forces, both enjoyed new commercial success. Similarly, in the AI world right now, Fran Alexander sees knowledge graphs and large language models combining forces to create retrieval augmented generation and similar architectures that work together to create systems more useful and valuable than the sum of their individual capabilities.

Interview transcript

Larry:

Hi everyone. Welcome to episode number 15 of the Knowledge Graph Insights podcast. I am really delighted today to welcome to the show, Fran Alexander. Fran is an independent taxonomist and ontologist based in Montreal. And so, welcome Fran. Tell the folks a little bit more about what you're up to these days.

Fran:

Hi Larry. Well, it's nice to talk to you again. I really enjoyed talking to you on the previous podcast that we did a little while ago. And that one was kind of a bit of a general introduction to taxonomies, ontologies, thesauruses, knowledge modeling and semantics. But this time, I thought we could talk about knowledge graphs and LLMs. They're a big hot topic and I did a presentation earlier on in the year for Taxonomy Boot Camp London, Bite-sized Taxonomy Boot Camp London. That was a lot of fun and has been really popular. A lot of people have been asking me about it. I've revisited it a couple of times and that's LLMs versus knowledge graphs, Alien versus Predator.

Larry:

Okay. So why Alien versus Predator? Why not King Kong and Godzilla or...?

Fran:

I did think maybe King Kong versus Godzilla. Godzilla as LLMs and King Kong as knowledge graphs. Certainly, the idea is that it's a fun analogy. It's a fun way to start thinking about the differences between knowledge graphs and LLMs. It's not supposed to be a serious study of science fiction characters, but you certainly could pick your own pair of monsters and do the analogy there. I did consider, as I say, I did consider Godzilla versus King Kong. That would work, but personally, I happen to really like the Alien franchise. I'm a bit more familiar with the Alien franchise. I thought it was really, really fun to have the Alien versus Predator crossover and that's actually become quite a successful franchise in its own right. So yeah, so you could use many monsters, pick your own monsters and run your own analogy. But as a starting point for talking about LLMs, they can seem very Alien. They can seem very scary. So that was my starting point.

Larry:

Interesting. Yeah, and so I'm just trying to, I'm getting it as you talk about it. Well, tell me a little bit more about, because in ontology work, we figure out what the entities in the domain are and ascribe properties to them. What are the properties of an Alien that make you think they're like an LLM?

Fran:

So Aliens are very different from humans and one of the reasons why I like the Alien versus Predator analogy is in these characters, they're societies and the way they operate and what we know of them are very different. So Aliens, we don't really know much about Alien society. They're not at all like humans. They have acid for blood and what an Alien does is basically, just goes around killing everything in its path and making more Aliens. We can't really communicate with them, we don't know much about them. They're very, very different. Their approach to the universe is very, very different to ours.

Fran:

Whereas Predators, are still big, scary monsters, still very powerful, but they're much more humanoid. They kind of look humanoid. They move in a more humanoid way and Predators actually have a much more human-like society. So they have some kind of moral principles, maybe not that many. They're basically, they're mercenaries for hire, but they do have social structures, social hierarchies, complex societies. You can't go and hire an Alien to work for you in the way that you can hire a Predator.

Fran:

So Predators are starting to get into those kind of complexities, hierarchies. You talk about the way that we build knowledge graphs, they're usually very specific. So Predators will have a specific target. They're out to assassinate their designated target that they're doing for money. Knowledge graphs are very specific and targeted and precise. Whereas Aliens, they're not really involved in that at all. They're just going off to do their Alien thing in their own Alien way. So that was my starting point. You've got these two contrasting characters, one that's very, very strange and different. The acid for blood of Aliens, I compared to LLMs having maths for blood.

Fran:

The way that LLMs work in a kind of, and I'm not an expert and machine learning engineers and experts, I don't know whether they'd completely agree with my overview, but the way I look at how LLMs work is that, you take a big corpus of something - texts, documents, images - chop it up into lots of little pieces, and then you layer on lots and lots of algorithms and calculations and machine learning to figure out what the probability of one piece appearing next to another piece is. And that essentially is what LLMs are doing. They're very, very complicated probability engines. Whereas knowledge graphs are built up using structures and hierarchies and conceptual models that come from humans and come from people.

Fran:

Now I don't know anyone who learns by chopping things up and calculating probabilities of bits of text. Humans don't read books and learn from them like that and humans don't discuss and describe concepts and approach the world like that. But a human way of looking at the world is in thinking of things with structures and hierarchies that we're used to. Our taxonomies that are kind of like the backbone with knowledge graphs, that kind of parent-child broader narrower concept relationship is something very, very familiar to us. We talked last time about supermarkets being organized with the dairy section and milk within the dairy section and types of milk. That's very natural to us and we approach the world with a mental map or a mind map.

Fran:

It's very much like in ontology. So when you build a knowledge graph, you tend to start with subject matter specifics like the Predator having a specific target and a specific reason for going out after its targets, a specific motivation and you'll build up, you start to put your lists, your labels, your taxonomies, your ontologies. You're building them up from a very human perspective with your subject matter experts or your business drivers to come up with a knowledge base that answers specific questions in a focused and targeted way. So that's kind of where I started with the analogy.

Larry:

Yeah and especially the way you punctuated at the end there. But when you were talking earlier about the LLMs and instead of acid, they have math for blood, maths for blood. And that's so Alien and I think what's interesting to me, there's weird contrast there between, I agree with you that that's like a metaphor and an analogy that really works for me in feeling LLMs, but I think because of their conversational interface that they typically use, I think a lot of people perceive them as more, ascribe more humanity to them than they deserve. Does that make sense?

Fran:

Yeah, I think that makes sense. I think it's really interesting. I think it's one of the dangers actually and there are probably other sci-fi monsters....

Andreas Blumauer

Every enterprise nowadays is awash in data, content, and knowledge, the understanding of which is all too often available only in employees' heads.

Forward-thinking businesses are moving to knowledge graphs to capture that tacit knowledge so that they can better understand and use it.

Andreas Blumauer shows how those graphs work best when they're accompanied by a domain knowledge model, creating a "semantic layer" that provides a vivid map of your business knowledge.

We talked about:

the recent merger of his company, Semantic Web Company, with Ontotext to form a new company, Graphwise, which aims to help enterprises build their semantic layer

the 20-year-old origin story behind his definition and description of a semantic layer

the emerging trend he sees of data, content, and knowledge people coming together, often around explorations of language and content structure

how getting domain knowledge out of people's heads and into multimodal AI architectures can streamline business research

how a semantic layer provides a map of your business knowledge

the importance of a domain knowledge model in a semantic layer architecture

the inadequacy of "desktop data integration," the practice of calling colleagues, consulting varied sources, and otherwise searching through ad hoc enterprise knowledge sources, and the stress it can cause

how a domain knowledge model can connect and reveal knowledge across an organization

the surprisingly small footprint of the domain knowledge model in an enterprise knowledge graph, typically just 1% or so of the semantic layer

how LLMs can help in the discovery of data and the construction of knowledge graphs

the two main elements of the semantic layer: domain knowledge models (taxonomies, ontologies, etc.) and an automatically generated enterprise knowledge graph

the different perceptions of the value of the semantic layer across data and content professionals, and how the arrival of gen AI has resulted in them talking together more to each other

how the semantic layer can facilitate the alignment of vocabularies and the understanding of data across business divisions

the benefits of a hybrid centralized-decentralized/global-local "glocalization" semantic layer strategy

Andreas' bio

Andreas Blumauer is SVP Growth at Graphwise, and CEO and founder of the Semantic Web Company (SWC), provider and developer of the PoolParty Semantic Platform and leading solution provider in the field of semantic AI and RAG. For more than 20 years, he has worked with more than 200 organizations worldwide to deliver AI and semantic search solutions, knowledge platforms, content hubs and related data modeling and integration services. Most recently, Andreas has been involved in the development of AI-powered ESG solutions for companies and investors.

A globally recognized thought leader and author in the field of semantic AI and graph technologies, Andreas has helped define and implement knowledge and AI strategies for various industries and domains. He is the author of "The Knowledge Graph Cookbook: Recipes that Work", a practical guide to building and deploying knowledge graphs in organizations. He is passionate about enabling clients to harness the power of semantic AI and graph technologies to achieve their business goals and make a social contribution.

Since 2022, Andreas has focused primarily on developing AI-based solutions that help organizations implement ESG and sustainable systems.

Connect with Andreas online

LinkedIn

Graphwise

Resources mentioned in this episode

Knowledge Graphs, LLMs and Semantic AI LinkedIn group

From Data to Trust: Leveraging Knowledge Graphs for Enterprise AI Solutions webinar

Enterprise architecture model that includes a semantic layer

Video

Here’s the video version of our conversation:

https://youtu.be/dEQh6m6zoDE

Podcast intro transcript

This is the Knowledge Graph Insights podcast, episode number 14. Data, content, and other business professionals have long known the benefits of sharing company lore in enterprise knowledge systems. Many of those systems now include a knowledge graph. Andreas Blumauer says that the thing that really lets you leverage your knowledge graph is a semantic layer that includes alongside your graph a human-crafted domain knowledge model that captures, and represents in a computer-readable way, the tacit knowledge in your enterprise.

Interview transcript

Larry:

Hi everyone. Welcome to episode number 14 of the Knowledge Graph Insights podcast. I am really delighted today to welcome to the show Andreas Blumauer. Andreas, you may know him as the CEO and the founder of the Semantic Web Company, one of the venerable companies in the Semantic space. But more recently, his company has merged with Ontotext to form Graphwise, where he's now the SVP, the Senior Vice President of Growth and Marketing. So welcome Andreas, tell the folks a little bit more about what's going on these days.

Andreas:

Yeah, sure. Thanks Larry. Thanks for the invitation to this great podcast series. Yeah. So yeah, exciting days. Currently experiencing the merger between Ontotext and Semantic Web Company. And actually it has come quite naturally. So we have been working together for many years and inside of our Semantic suite, PoolParty Semantic Suite, we always have been using, at least for many years, GraphDB, the core product of Ontotext. So we got closer and closer, and finally the time has come to execute on this bigger vision, which is all about helping organizations create semantic layers. And this is where we currently stand, and it's definitely a very, very exciting time for all of us and looking forward to the next steps.

Larry:

Yeah, it's easily the biggest news in the industry lately, but yeah, the whole point of it is to support the semantic layer, and you're one of the first people to talk about that. I mean, if you Google it it shows up in a lot of different contexts. But in the context of these enterprise architectures that support knowledge representation tech, you're one of the first people to talk about that. I don't know how familiar people are or whether they need to have your notion of a semantic layer disambiguated from what they've got, but tell me a little bit about what you think of the semantic layer is.

Andreas:

Yeah, no, this has been a topic I always was, since the beginning of my career, quite a lot of discussions were around knowledge management and how to bring together digital assets to the actual knowledge people have in their heads, so to speak. And there was this, I would say, first wave of knowledge management discussions where it turned out to be quite clear that on the one side, at that time it was around 2005, six or so, rather let's say new digital community where documents contained data information, and the other side of the community said, it's not knowledge. So in a document, you will never find knowledge, it just can be translated or transformed into knowledge as soon as it has been recognized by human beings as such. And so I was always feeling between sitting between those two communities.

Andreas:

So on the one side was excited since day one. Okay, where does AI bring us? And on the other side, I was totally understanding, okay, we should not substitute human intelligence at all. So I was always in this kind of tried to bridge those two communities. And so I was thinking to myself, looking at the contemporary enterprise data architectures at those days there was a missing layer. There was something, there was the data layer, there were the document repositories, and on top there were the applications trying to represent some kind of business logics and nothing in between. So there was really a big, big hole. So I thought to myself, at that time there was topic maps around even still, and then RDF came up, semantic web standards, more and more semantic web standards got developed. So I was trying to introduce this new layer and they called it semantics layer at that time already, to the community.

Andreas:

And everybody stared at me as if I was an alien, really. So I felt like, okay, maybe I said something wrong, it doesn't make sense to the others. And I was a little bit puzzled by that, I thought, but this is still, I mean, and that really kept me going into this direction. I always thought to myself, there is something very important. And now I think we have arrived a new era where the data people, the content people and the knowledge people come together and now they discuss how should a semantic layer look like? And I totally understand each of those have different angles and different ideas how it should be set up. But let's take a closer look at it because it really depends on the use cases on top and the strategies of the AI, the overall AI strategy we all now want to develop driven by Gen AI. Obviously it has become even more important than ever before to have a semantic layer in place in our siloed world, in our siloed environments.

Larry:

Yeah. And that notion of the, as you were saying that you're making me realize that the LLMs are kind of at that, to my mind, there's emerging at the highest level that architecture, they're really a great interface to a lot of this knowledge that we have, but they're not the best keepers or understanders of that knowledge, whereas the RDF stack of knowledge representation stuff is. Can you maybe talk a little bit about the relationship between LLMs and other AI tech and knowledge graphs and how the semantic layer facilitates their interaction?

Andreas:

Right. I would start maybe like that. First of all, I think language is something which allows us to develop knowledge without our ability to talk and to use language. We're not able to develop anything more complex than let's say what we need for our survival mode typically....

Jessica Talisman

Jessica Talisman is a seasoned information architect with decades of experience across a variety of domains.

She's done a lot of education and outreach around her semantic and and information architecture practices. One of the most important lessons she's learned is the crucial role of standards like the W3C SKOS model to bring structure and semantics to information and knowledge systems.

Since there are never enough information architects in any organization, she supports the democratization of IA practices, but she's also quick to highlight the unique skills that you can only get with deep study.

We talked about:

her work as a senior information architect at Adobe and previously in GLAM (galleries, libraries, art, and museums) and other domains

how her work in GLAM showed her the importance of the concept of lineage and attribution and benefits of the FRBR (Functional Requirements for Bibliographic Records) framework

how standards and rules bring discipline and structure to information and data ecosystems

how capturing knowledge via the SKOS standard can provide on its own the structure, semantics, and disambiguation your data needs, as well as set you up for future successes

the importance of focusing on semantic fundamentals and how the ensuing understanding if your data assets can improve activities like a graph RAG implementation

the importance of collaborating and sharing ideas across domains

democratization, evangelism, and other kinds of information architecture outreach

the "Golden Spike" railroad metaphor she uses to illustrate cross-functional collaboration challenges

how linked data can help span organizational silos and align stakeholders on language and terminology

the importance of understanding your unique organizational fingerprint

how applying the library science concept of "scholarly communications" can move organizations forward and promote innovation

Jessica's bio

Jessica Talisman is a Senior Information Architect at Adobe. She has been building information systems to support human and machine information retrieval for more than 25 years. Jessica has worked in a variety of domains such as e-commerce, government, AdTech, EdTech and GLAM. Jessica holds a Masters in Library and Information Science with a concentration in Informatics. She lives in Santa Cruz, California with her partner Dave, and two dogs.

Connect with Jessica online

LinkedIn - Jessica is working on a new book about information architecture and is looking for anecdotes and other input. If you're an IA practitioner with good stories to share, she'd love to connect.

Video

Here’s the video version of our conversation:

https://youtu.be/1tlrZTJ52Vs

Podcast intro transcript

This is the Knowledge Graph Insights podcast, episode number 12. Anyone who has tried to discern how people in a domain talk about the concepts in it, and then try to align stakeholders in an organization around those concepts and the words that describes them, and then share that information with computers so that you can scale the impact of your work, knows that you need a good system to manage your taxonomies and other terminology. Jessica Talisman argues that the W3C SKOS standard is your best friend in such endeavors.

Interview transcript

Larry:

Okay. Hi everyone. Welcome to episode number 12 of the Knowledge Graph Insights Podcast. I am really delighted today to welcome to the show Jessica Talisman. Jessica's currently a senior information architect at Adobe, but she is extremely experienced in information architecture and knowledge graph stuff, so welcome Jessica, tell the folks a little bit more about what you're up to these days.

Jessica:

Thanks, Larry. So I'm currently, as Larry said, a senior information architect at Adobe, and before this, I was information architect over at Amazon. I've worked in many different domain spaces, but my original foundational experience is in information and library science and I worked as an academic librarian in the past with books, museum galleries, library, art, and that really gave me the foundational experience and information and knowledge necessary to do my job.

Larry:

Nice. We were talking before we went on the air about how, especially your art and gallery and that kind of curation, tell me a little bit about how that ... It seems like that was really foundational to your current practice or it's been influential. Tell me why. Is it there's something about the nature of those collections or what's going on there?

Jessica:

One of the most critical things when working in that space in galleries, libraries, art. It's called GLAM, which is actually a great acronym for galleries, library, art and museum, and within the GLAM space, provenance and lineage is something that cannot be ignored because you're dealing with works, whether it be print books, art obviously, an attribution to the artist or writer and that's instrumental to building information in library spaces within that domain space. And so that taught me a lot about linking records. There's a super interesting framework that from the library and information science background, it's a framework called FRBR, which is Functional Requirements for Bibliographic Records. And the whole idea is to maintain the manifestation and expression of works, which is the lineage of a piece of art or a book and being able to support proper provenance and attribution of works while maintaining lineage for the benefit of people and machines.

Larry:

Nice. And it's so clear when you think about how LLMs work and the current state of AI, it's clear for the need for that kind of thing. Well, and that's kind of jumping ahead a little bit too to what you do with this stuff after you've got it all organized. I think in almost every case, whether you're like a data scientist, an information architect, content strategist, a museum curator, whatever, you start with this pile of concepts and words and then people like you turn them in to something useful to both humans and computers. How does that look? Walk us through the top level overview of that.

Jessica:

So normally no matter what, when I come into an organization or any sort of information environment or data-rich environment, there's usually problems, identified problems. And those usually are not the only problems within an ecosystem, an information or data ecosystem. And so the idea of nomenclature or vocabularies is instrumental. It's the foundation of how we discover and find things within that ecosystem. And so having to look at the current vocabulary and the current environment of how words are implemented, that starting point in understanding culturally and otherwise, how words are used and vocabularies used to not only define a domain space but to help support information discovery and retrieval within a space.

Larry:

Yeah, because that's the classic ... I mean I think the way most people would think of, they don't know anything else about information architecture, it's about discovery and discoverability and findability. But there's sort of levels to that. I work more in the UX world and there you're like, here's how we're going to talk about things. With content designers and UX writers, that's a common thing. So you just end up with a controlled vocabulary or something like that where you talk about it, but there's sort of a progression from that level all the way out to a full-blown ontology. How does that escalation happen I guess from when does it become clear that you need, "Well we really need to define these things," so I guess that's a glossary right now?

Jessica:

Well, in looking at that environment, so you have a series or collections and I like to use the word collections of vocabularies. Often they occur or exist as flat lists and there are usually internal belief systems that determine how these vocabularies are structured and implemented. Some are closely guarded. I will give the forewarning that within these spaces sometimes branding and marketing will be also pretty protective over the vocabularies used, but not understanding that these are meant to work on the back end of systems to help facilitate information retrieval and discovery. And so when building for back end systems, the lowest common denominator aside from a flat list or controlled vocabulary is to use something like an ontology like SKOS, which is Simple Knowledge Ontology System. It's a very simple ontology that helps to structure a hierarchy and simple relationships that are machine-readable. And it's something that that one standard and lightweight or upper ontology, there's the characteristics that are included with that, but it's super machine-readable and translatable, but it's also human-readable, which is what's really critical about that structure.

Jessica:

And so you can not only define parent-child relationships, alt labels or aliases and encode those with ontological labels, but there's rule bases, there's standards included with that which lines up with several other standards that exist for information retrieval on the World Wide Web. So having to structure and include those rule bases that are standards-based as well helps to enforce a certain type of discipline and structure within that information ecosystem and data ecosystem, which helps to guide people towards best practices and not only why hierarchies or taxonomies are important. But the introduction of concepts like thesauri or thesaurus, you can actually build a thesaurus using SKOS. So whether people realize it or not, that actually helps to shape a thesaurus. And then you can also have a very sort of primitive knowledge graph but still a knowledge graph using SKOS. So it's like a nice little primer and entry to the world of structuring data and information to be disambiguated. So you naturally go through a disambiguation process and a struc

Tony Seale

With ten years of semantic data experience and an endless stream of insightful posts on LinkedIn, Tony Seale has earned the moniker "The Knowledge Graph Guy."

Tony says there's precious little time for enterprises to prepare their data with the interconnectedness and semantic meaning that it needs to be ready for the coming wave of more powerful AI technology.

We talked about:

his 10-year history of applying academic knowledge graph insights to commercial work, mostly in the finance industry

the yin-yang relationship in his "neuro-symbolic loop" concept that connects creative, generative LLMs and the reliable, structured knowledge provided by knowledge graphs

the contrast in reasoning capabilities between LLMs and knowledge graphs

how neither formal logic nor probabilistic systems are rarely the right answer on their own, hence the yin-yang analogy

the crucial role of understanding and consolidating data, the gold mine on which every enterprise is sitting that describes any organization's unique value

the power of understanding the "ontological core" of your business and then projecting it, selectively and strategically, to the world

the urgent threat posed by snake oil salesmen and other opportunists coming into the graph world and derailing enterprises' chances to properly exploit their unique data advantage

the two crucial characteristics of AI-ready data: connectedness and semantic meaning

his work chairing the Data Product Ontology (DPROD) working group, an effort to provide a semantic definition of what a data product is

Tony's bio

For over a decade, Tony has been passionate about linking data. His creative vision for integrating Large Language Models (LLMs) and Knowledge Graphs in large organisations has gained widespread attention, particularly through his popular weekly LinkedIn posts, earning him the reputation of 'The Knowledge Graph Guy.'

Tony’s journey into AI and knowledge graphs began as a secret side project, working from a computer under his desk while employed at an investment bank. What started as a personal passion quickly evolved into an area of deep expertise. Over the past decade, Tony has successfully delivered several mission-critical Knowledge Graphs into production for Tier 1 investment banks, helping these institutions better organise and leverage their data.

Now, Tony has just founded The Knowledge Graph Guys, a brand-new consultancy dedicated to making knowledge graphs accessible to organisations of all sizes. Through this venture, he aims to empower businesses with the tools and strategies needed to harness this powerful technology.

Connect with Tony online

LinkedIn

The Knowledge Graph Guys

Resources mentioned in this podcast

Connected Data London conference

Knowledge Graph Conference

GraphGeeks podcast

DPROD working group

Video

Here’s the video version of our conversation:

https://youtu.be/lkNvCzwhTRY

Podcast intro transcript

This is the Knowledge Graph Insights podcast, episode number 11. Whether they realize it or not, every business on the planet is sitting on a gold mine, the precious data that uniquely positions them in their industry and market. With ten years of AI practice and an endless stream of insightful social media posts, Tony Seale has earned the moniker "The Knowledge Graph Guy." Tony argues that enterprises that fail to grasp the urgent need to consolidate and understand their data will not survive the coming wave of more powerful AI.

Interview transcript

Larry:

Hi, everyone. Welcome to episode number 11 of the Knowledge Graph Insights podcast. I am really delighted today to welcome to the show Tony Seale. Tony is the Knowledge Graph Guy. That's how everybody knows him on LinkedIn, and I think he's earned that moniker. He also does a lot of consulting and work for big investment banks and things in the financial world. But welcome, Tony. Tell the folks a little bit more about what you're up to these days.

Tony:

Hi, Larry. Thanks for having me on here. Yeah, so as I was saying, I've basically been doing knowledge graphs now for the last 10 years, largely within investment bank, investment, large investment banks. So I'm kind of at the rubber meets the road. Well, that's where I have been. So taking the technology that's been largely in the academic space and actually applying that into production settings. And then I've started trying to share that knowledge and I've become obsessed with the idea of what it would be to connect an organization together. So what would be possible if most of the information within a given organization was connected together? And I'm on a mission to push that forward. As you say, I've just started doing a consultancy to try and accelerate that effort. The last really, I guess, two and a bit years now have been focused really on the space between large language models and knowledge graphs. And hopefully we can talk a bit about that.

Larry:

Yeah, because I attended the Semantics Conference last week and that's pretty much all anybody was talking about. I mean, there was other conversations of course, but you have a take, and I think one of the things that is becoming clear in my mind is this sort of evolution from just math and LLMs to graph to RAG to graph RAG. And then you have this concept of the neuro-symbolic loop which, is that the evolution of the integration of these technologies? Tell us some more about that.

Tony:

Yeah, so I guess maybe to frame it at a conceptual level, you can think of the large language models existing in this continuous space. So they're slightly fuzzy, they're probabilistic, they're generative. So they're always guessing at what the next right thing to do should be. They're not structured. And that has within it a huge amount of power because they can explore different pathways. They can be in at least some limited sense of the word creative and imaginative. They can write poetry and do everything that we are familiar with them doing. But you could sort of contrast that type of intelligence with I guess what you would call the existing and what people will rather derogatorily call good old-fashioned AI. The existing approach, which is sort of formal deductive logic and formal reasoning. And really that's what knowledge graphs represent from a data side, like the most flexible structure for doing reasoning over your data.

Tony:

So what then becomes interesting, it's like, well, can large language models actually do that formal reasoning? And obviously the big AI houses are trying really hard in order to make that happen. They're chucking a lot of money into making that happen. But I think to a certain extent, maybe one day they will kind of get so close that we won't know the difference between it. But to a certain extent it's just a kind of different paradigm, if you like. There an uncertainty there within a generative model, which is just very different from what deductive reasoning is going to be.

Tony:

So the idea of the neuro-symbolic loop is to try to bring these two systems like a yin and yang, bring the two systems in together closely. So that as close as possible and as lower grain possible level, you are looping between this system one and system two. So system one being the large language model, being a bit creative, being generative, very, very quick. And then the system two being the formal representation of the system in which you working through steps, being able to do structured querying, always getting back reliable results.

Larry:

I love the visual of the yin and yang symbol as the representation of that tightening loop, because that's a perfect way to look at that. But back to, I want to revisit the notion of reasoning a little bit because you mentioned that and that seems really important. And a lot of the AI fans on LinkedIn these days and the big companies themselves of course have been talking a lot about the reasoning that they can do. Can you contrast that? I heard it referred to at Semantics as pseudo-reasoning, versus the real reasoning that you just mentioned, that deductive logic based systems can bring. Can you talk a little bit more about that, the relative reasoning capabilities and whether... Are they just faking it till they make it or what's going on there?

Tony:

Yeah, so I mean you're always working in analogies with this stuff because at the end of the day, nobody, not even the people who are very close to this truly understand what's going on inside of a large language model. It's a bit of a mysterious thing what's happening in there. But here's one way of conceptualizing it, that effectively it's kind of doing not a database lookup, but almost like that. This very sophisticated lookup of its training data. So it's seen a huge number of samples of training data. It's able to a certain extent, it's kind of mapped all of that kind of training information as you're doing the different layers within the large language model. It's compressed some of the concepts in there, not in a way that we would necessarily be able to understand in this kind of vector representation of it. But so it's able to use that to go and retrieve these kind of patterns, and to a certain extent sort of merge them together.

Tony:

So what recently has been done with the kind of Q-star stroke, Strawberry stroke 01, and really all of the others are doing the same thing as well, is that you will take some domain where you have right and wrong answers, for instance coding or mathematics. And then what you will do is you'll get the large language model to simulate loads of different answers to that and then have its kind of reasoning steps of how it got to that particular answer. And then you'll go out to an external verifier, which in the coding thing it's like, "Well, okay, run this unit test. Does the unit test pass or with the maths? Okay well,...

Paco Nathan

Graph RAG is all the rage right now in the AI world. Paco Nathan is uniquely positioned to help the industry understand and contextualize this new technology.

Paco currently leads a knowledge graph practice at an AI startup, and he has been immersed in the AI community for more than 40 years.

His broad and deep understanding of the tech and business terrain, along with his "graph thinking" approach, provides executives and other decision makers a clear view of terrain that is often obfuscated by less experienced and knowledgeable advisors.

We talked about:

his work building out the knowledge graph practice at Senzing, and their focus on entity resolution

the importance of entity resolution in knowledge graph use cases like fraud detection

the high percentage of knowledge graph projects that we never hear about because of their sensitive or proprietary nature

his take on the concept of "graph thinking" and how he and colleagues illustrate it with a simple graph model of a medieval village

how graphs add structure and context to our understanding of the world

the importance of embracing complexity and the Cynefin framework in which he grounds various types of business challenges: simple, complicated, complex, and chaotic

how to apply insights discerned from a Cynefin framing in management

how knowledge graphs can help oranizations understand the complex environments in which they operate

the wide range of industries and government entities that are applying knowledge graphs to concerns like supply chains, ESG, etc.

his overview of RAG - retrieval augmented generation and graph RAG

the wide variety of uses of the term "graph" in the current technology landscape

Microsoft's graph RAG which uses NetworkX inside their graph RAG library, not a graph database

Neo4j's approach which creates a "lexical graph" based an an NLP analysis of text

"embedding graphs"

ontology-based graphs

Google's approach to RAG, using graph neural networks

graphs that do reasoning over LLM-created facts assertions

"graph of thought" graphs based on chain-of-prompt thinking

"causal graphs" that permit causal reasoning

"graph analytics" graphs that re-rank possible answers

the evolution of graph RAG libraries and the variety of design patterns they employ

the shift in discovery dominance from search to recommender systems, most of which use knowledge graphs

examples of graph RAG from LlamaIndex and LangChain, in addition to Microsoft's graph RAG

his prediction that we'll see more reinforcement learning, graph tech, and advanced math capabilities like causality in addition to LLMs in AI systems

his reflection on his efforts to advance graph thinking over the past 4 years and the current state of LLMs, graphs, graph RAG, and the open-source software community

the need for a shift in thinking in the industry, in particular the need for cross-pollination across tech proficiencies and enterprise teams

the "10:1 ratio for the number of graph RAG experts versus the number of people we've actually worked with a library"

Paco's bio

Paco Nathan leads DevRel for the Entity Resolved Knowledge Graph practice area at Senzing.com and is a computer scientist with +40 years of tech industry experience and core expertise in data science, natural language, graph technologies, and cloud computing. He's the author of numerous books, videos, and tutorials about these topics.

Paco advises Kurve.ai, EmergentMethods.ai, KungFu.ai, DataSpartan, and Argilla.io (acq. Hugging Face), and is lead committer for the pytextrank and kglab open source projects. Formerly: Director of Learning Group at O'Reilly Media; and Director of Community Evangelism at Databricks.

Connect with Paco online

LinkedIn

Sessionize

Derwen.ai

Senzing.com

Resources mentioned in this interview

Connected Data London conference

Knowledge Graph Conference

GraphGeeks community

REALM: Retrieval-Augmented Language Model Pre-Training, Guu, et al.

Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks, Lewis, et al.

NebulaGraph Launches Industry-First Graph RAG: Retrieval-Augmented Generation with LLM Based on Knowledge Graphs

Graph Retrieval-Augmented Generation: A Survey, Peng, et al.

Video

Here’s the video version of our conversation:

https://youtu.be/4pmV6BUSKmY

Podcast intro transcript

This is the Knowledge Graph Insights podcast, episode number 10. As enterprises and tech companies have looked to ground in factual knowledge the answers that their LLMs deliver, graph RAG architectures and products have sprung to the fore. With his deep background in Silicon Valley culture, the open-source software community, artificial intelligence practice, and knowledge graphs and semantic technology, Paco Nathan is one of the best-positioned people in the industry to help us understand the current state of graph RAG.

Interview transcript

xLarry:

Hi, everyone. Welcome to episode number 10 of the Knowledge Graph Insights podcast. I am really delighted today to welcome to the show Paco Nathan. Paco is the principal DevRel engineer for knowledge graphs at Senzing, the big company that does entity resolution for a large-scale mission-critical applications, really fancy high-end graph stuff.

So, welcome, Paco. Tell the folks a little bit more about what you're up to these days.

Paco:

Thank you very kindly, Larry. I appreciate. Yeah, I'm over at Senzing. Actually, I was presenting a master class about Senzing integrations at the Knowledge Graph conference last time we saw each other in Manhattan and then joined the company shortly thereafter.

Paco:

I'm building out the knowledge graph practice area because we do... I'm with this team that has been doing work for many years in entity resolution and most people have probably never heard of it, but most people have probably used it. So, the idea is, say you have a bunch of different tables or data sets and you want to try to find what are the consistent entities inside these tables.

Paco:

So, you might have Bob R. Smith and Bob Smith Jr. and they're both at 101 Main, but one of them is spelled 101 Main Street and the other is maybe spelled a different way or abbreviated a different way. And if you can think about that kind of problem, but spanning across billions of records in a lot of different data sources, how can you pull out the consistent entities?

Paco:

And it sounds like a trivial data science problem. We could just use string distance, Levenshtein distance, which is a typical thing. But when you take into account the fact of, well, what if you've got Bob R. Smith at 101 Main, Bob R. Smith Jr., but then you get Bob R. Smith Sr. at 101 Main, and they've both got voter registration. Is that the same person?

Paco:

Because your Levenshtein distance will tell you it is. If you set a threshold on string distance, they'll tell you they're the same person. So, when they try to register for vote, one of them will be denied voting rights. And so, this problem becomes very much complicated when you're working in a world where there are companies that have offshore subsidiaries and maybe you don't know the actual owners.

Paco:

You might know some of the directors and you get a very tangled web of some very bad people who are moving a lot of money around to do very bad things offshore, sorry, illegal fishing, illegal lumber, overthrowing democracies in Asia or in North America, for that matter. Basically, when you get the problem of trying to understand who's who and what's what, and a lot of different people or companies or ships that might have a registry somewhere, but you don't know exactly in a given business context who they are, how can you triangulate on them?

Paco:

And so, it's typically not a matter of just a string distance, it's a matter of, well, I have enough elements of their address that are in common even though there are five different ways to represent this address in Singapore. I can tell the difference between a company at the same address or a hundred companies that are in the same shopping mall, which actually in Singapore is really a hard problem to understand.

Paco:

And same thing for tax records or passport control. There's an area called UBO, which is ultimate beneficial owner, has a lot to do with sanctions compliance and catching oligarchs and understanding who is trying to do money laundering in an offshore tax haven, who is funneling billions of dollars out of Kremlin assets to try to influence a campaign somewhere. These are the kind of problems we work with.

Paco:

And so, the long and short is that these are... If you look at any episode of Homeland or The Wire or NCIS, any crime drama, inevitably, the protagonist goes up to a wall and they've got pincushion, they've got all these clippings and photos and notes, and they take yarn and draw a graph between them. And the thing is, the people who do that real work, if you're in the US, you're talking about three-letter agencies. If you're in the UK, you're talking about four-letter agencies.

Paco:

But the people who really do that work 24/7, they actually use knowledge graphs. They use collaborative knowledge graph tools like Aptitude Global, SiReN, GraphAware, Linkurious, Esri, ArcGIS Knowledge, Kineviz. There's a bunch of different tools that allow people to collaborate on building knowledge graphs to catch bad guys.

Paco:

In finance, we have acronyms like AML, anti-money laundering, or UBO, ultimate beneficial owner, or PEP, politically exposed persons. All of these things have to do with the fact that somebody has committed very large-scale crimes and governments have reacted by saying, "Okay, regulatory, we will not allow this to happen again." So, you end up having data sets like LIFE, was a multi-government response to the problems of 2009 global financial crisis.

...

Katariina Kari

For the past eight years, Katariina Kari has built knowledge graph teams at giant e-commerce companies like IKEA and Zalando. This practical, real-world experience puts her in an elite group of ontology and knowledge graph experts.

Knowledge graphs offer unique benefits to e-commerce merchants. From better product recommendations to more useful search results, the semantic capabilities that knowledge graphs provide routinely result in seven-figure sales increases.

The knowledge graphs that Katariina builds provide a semantic layer in the enterprise architecture that lets companies capture, use, and re-use the organization's unique domain knowledge in any number of applications.

Because knowledge graph isn't one application that does one thing. It's a paradigm shift in the way we work with data. It's a paradigm shift in the way we code, because now you don't need to put business logic into your code.

We talked about:

her work over the past eight years building knowledge graphs at companies like IKEA and Zalando

how using knowledge graphs to improve reccomendation and search routinely brings seven-figure business benefits

the set of skills and talents it takes to implement a knowledge graph project, most of which already exist in most companies

how LLMs and other AI tools can help transform structured or unstructured data into semantic data, a computable resource that captures business domain knowledge

some of the specific skills needed for KG work: ontology experts, back-end developers who understand the semantic web stack, data scientists and engineers, knowledge practitioners to capture domain knowledge, and product management

the need in each organization for a unique knowledge graph team tailored to the needs of the company and the talent available

the importance of user-centricity and use-case understanding in any knowledge graph project

the benefits of capturing business logic in a semantic layer which can be used and re-used in multiple applications

an interesting search-improvement use case that resulted in seven-figure sales increases, as well as experience-improving recommendation and info-box use cases

how capturing subject matter expertise in a knowlege graph can dramatically improve recommendation systems and deliver unexpected benefits to other

the importance of showing the benefits of knowledge graphs to organically advance enterprise adoption

her take on the difference between RDF-based knowledge graphs and labeled property graphs (LPGs) like Neo4j

the compelling case for knowledge graphs in e-commerce, which she has discovered in her eight years of practice

Katariina's bio

Katariina Kari is a leading expert in semantic web technologies, specializing in the development of ontologies and knowledge graphs. Over the past eight years, she has worked with prominent brands like IKEA and Zalando, building knowledge graphs that significantly enhance customer experiences by improving search functionalities and recommendations. Her extensive hands-on experience in creating enterprise knowledge graphs has established her as one of the global top talents in the field.

Katariina is frequently invited to speak at international events on the semantic web and knowledge graphs, sharing her insights and practical expertise with industry professionals. Her deep knowledge and passion for the semantic web have made her a sought-after keynote speaker and thought leader in the field.

Balancing a dual enthusiasm for technology and the arts, Katariina holds both a Master of Science degree and a Master of Music degree. From 2012 to 2016, she ran her own consultancy, where she worked closely with classical music organizations and artists, helping them navigate digital outreach. An art-loving and art-serving nerd, she seamlessly blends her love for music and technology in all her work, constantly pushing the boundaries of what’s possible in her field.

Connect with Katariina online

LinkedIn

Video

Here’s the video version of our conversation:

https://youtu.be/gMFAHlY7VL0

Podcast intro transcript

This is the Knowledge Graph Insights podcast, episode number nine. One of the main benefits of semantic technology is the ability to sort out business logic independent of data and data from the applications in which it's used. Katariina Kari has captured in knowledge graphs the business expertise of e-commerce giants like IKEA and Zalando to power better search and recommendation systems and to generally provide a better experience for both internal users and external customers, resulting in millions of dollars in new sales.

Interview transcript

Larry:

Hi, everyone. Welcome to episode number nine of the Knowledge Graph Insights Podcast. I am really delighted today to welcome to the show Katariina Kari. Katariina is a long-time, deeply embedded in the community knowledge graph professional. She's done a lot of e-commerce work at places like IKEA and Zalando. She's currently the head of data at a stealth internet, a stealth startup that we can't talk too much about. But welcome, Katariina. Tell the folks a little bit more about what you're up to these days.

Katariina:

Yeah, thank you Larry, and thank you for having me in your podcast. Yeah, I could say that I've been really lucky to have worked in the industries and especially in the lifestyle sector, e-commerce sector for the past eight years. So very early on, before even graph databases were really commercialized, I had the opportunity to start building knowledge graphs. And so now, when there's someone out there was like, "Oh, I need a knowledge graph," I can confidently say, "Well, I've done it a few times so I can tell you, I can advise you or I can even run a team for you that can build the knowledge graph." And it's just shown me a lot of practical things, it's given me a really good perspective on what works from theory and from research and what actually doesn't, or doesn't yet work, or isn't mature enough yet for an applied setting in commercial use.

Larry:

That point you just brought up, you have a PhD in something, right?

Katariina:

No, no, I don't have a PhD. I never really went into research. I have a few published articles, scientific articles that I did towards the end of my master's, but I actually just have two master's. I have a master's in technology and then I have a master's in music arts management, because I've always carried this love for both art as well as technology, and I always wanted to combine them.

Larry:

I didn't plan it this way, but the episode right before this one, number eight, was Vera Brozzoni, who's a metadata strategist at the BBC, and she comes out of classical music. So if you don't know Vera, you two have to meet and talk music and data and stuff. But one of the things you mentioned there is one of the things in this community is, and the reason I assumed, it's usually safe to assume in this world that somebody has a PhD, but you've always been more focused on the practice side than the research side, which is awesome, because I'm all about sharing practice, so thank you for being focused that way. And you have all this experience coming up on, what, eight years of experience at IKEA and Zalando, two of the biggest retail brands in the country, or in the world.

Larry:

And one of the things, and I want to gently take you to task for something in a talk I saw you do recently, this hour-long talk, brilliant stuff about a lot of this stuff we'll talk about today, but right in the middle of that talk, you just kind of matter-of-factly mentioned like, "Yeah, and we're realizing seven-figure business benefits across this." I'm like, "Wait, what?" And in my journalism training, we would call that burying the lede. But that's kind of at a top level... There's real obvious business benefits to adopting knowledge graph technology. Can you talk about what is it that's unique about the work you've done and this technology that permits these massive revenue gains?

Katariina:

I would say that one part of it is the work, but the other part of it is working with big brands like Zalando, Europe's biggest e-commerce fashion, and then IKEA, one of the most known trademarks or brands in the world. So their volumes in e-commerce are huge. So if you add a positive change to the customer experience, like giving quality recommendations or just improving a few of the worst-performing search terms, you get a lot of... The volume is so big, the benefit is really big, and it's already in that category of seven-figure sums. So that's why I think maybe not every little e-commerce site can invest in this technology first. It's great that these big brands are actually investing in so we can figure out exactly how to do it, and then that can be brought to maybe a smaller-volume e-commerce setting or smaller-volume industry so that they can then make sense of these best practices. That's at least the way I see it. But yeah, I mean just being able to improve a big website's performance, just doing little optimization is already moving the needle quite a lot.

Larry:

Right. And once you've articulated those best practices, you can picture it, smaller businesses benefiting from it. But right now, it takes quite a team to put this together. I've heard you talk a lot about the skills that it takes, the roles that you need to execute on those skills, and then the human element, the thing that our friend Ashleigh Faith calls the data therapy part of this. Can you talk a little bit, I guess, first, about what does it take, what are the skills that you need, the knowledge, the wherewithal in your organization to actually make a knowledge graph project like the ones you've worked on happen?

Katariina:

Well, when you work with these industries in e-commerce, they'll probably have already very brilliant backend developers,...