Austrian Artificial Intelligence Podcast

Austrian Artificial Intelligence Podcast

By Manuel PasiekaTechnology
Download on the App Store

All Austrian Artificial Intelligence Podcast episodes

  • 61. Jules Salzinger - AIT - Building explainable and generalizable AI Systems for Agriculture

    Today on the podcast I have to pleasure to talk to Jules Salzinger, Computer Vision Researcher at the Vision & Automation Center of the AIT, the Austrian Institute of Technology.


    Jules will share with us, his newest research on applying computer vision systems that analyze drone videos to perform remote plant phenotyping. This makes it possible to analyze plants growth, but as well how certain plant decease spreads within a field.


    We will discuss how the diversity im biology and agriculture makes it challenging to build AI systems that generalize between plants, locations and time.


    Jules will explain how in their latest research, they focus on performing experiments that provide insights on how to build effective AI systems for agriculture and how to apply them. All of this with the goal to build scalable AI system and to make their application not only possible but efficient and useful.


    ## TOC

    00:00:00 Beginning

    00:03:02 Guest Introduction

    00:15:04 Supporting Agriculture with AI

    00:22:56 Scalable Plant Phenotyping

    00:37:33 Paper: TriNet

    00:70:10 Major findings


    ### References

    - Jules Salzinger: https://www.linkedin.com/in/jules-salzinger/

    - VAC: https://www.ait.ac.at/en/about-the-ait/center/center-for-vision-automation-control

    - https://www.ait.ac.at/en/about-the-ait/center/center-for-vision-automation-control

    - AI in Agriculture: https://intellias.com/artificial-intelligence-in-agriculture/

    - TriNet: Exploring More Affordable and Generalisable Remote Phenotyping with Explainable Deep Models: https://www.mdpi.com/2504-446X/8/8/407

    1 hr 27 min
  • 60. Alexandre Paris - Proofcheck - LLM fine-tuning and customization

    ## Summary

    Today on the show I am talking to Proofreads CTO Alexandre Paris. Alex explains in great detail how they analyze digital books drafts to identify mistakes and instances within the document that dont follow guidelines and preferences of the user.


    Alex is explaining how they fine-tune LLMs like, Mistrals 7B to achieve efficient resource usage and provide customizations and serve multiple uses cases with a single base model and multiple lora adapters.


    We talk about the challenges and capabilities of fine-tuning, how to do it, when to apply it and when for example prompt engineering of an foundation model is the better choice.


    I think this episode is very interesting for listeners that are using LLMs in a specific domain. It shows how fine-tuning a base model on selected high quality corpus can be used to build solutions outperform generic offerings by OpenAI or Google.


    ## AAIP Community

    Join our discord server and ask guest directly or discuss related topics with the community.

    https://discord.gg/5Pj446VKNU


    ## TOC

    00:00:00 Beginning

    00:02:46 Guest Introduction

    00:06:12 Proofcheck Intro

    00:11:43 Document Processing Pipeline

    00:26:46 Customization Options

    00:29:49 LLM fine-tuning

    00:42:08 Prompt-engineering vs. fine-tuning


    ### References

    https://www.proofcheck.io/

    Alexandre Paris - https://www.linkedin.com/in/alexandre-paris-92446b22/

    54 min
  • 59. Philip Winter - VRVis - Continual Learning

    Today I am talking to Philip Winter, researcher at the Medical Imaging group of the VRVis, a research center for virtual realities and visualizations.


    Philip will explain the benefits and challenges in continual learning and will present his recent paper "PARMESAN: Parameter-Free Memory Search and Transduction for Dense Prediction Tasks". Where he and his colleagues have developed a system that uses a frozen hierarchical feature extractor to build a memory database out of the labeled training data. During inference the system identified training examples similar to the test data and prediction is performed through a combination of parameter-free correspondence matching and message passing based on the closes training datapoints.


    I hope you enjoy this episode and will find it useful.


    ## AAIP Community

    Join our discord server and ask guest directly or discuss related topics with the community.

    https://discord.gg/5Pj446VKNU


    ## TOC

    00:00:00 Beginning

    00:03:04 Guest Introduction

    00:06:50 What is continual learning?

    00:15:38 Catastrophic forgetting

    00:27:36 Paper: Parmesan

    00:40:14 Composing Ensembles

    00:46:12 How to build memory over time

    00:55:37 Limitations of Parmesan


    ### References

    Philip Winter - https://www.linkedin.com/in/philip-m-winter-msc-b15679129/

    VRVIS - https://www.vrvis.at/

    PARMESAN: Parameter-Free Memory Search and Transduction for Dense Prediction Tasks - https://arxiv.org/abs/2403.11743

    Continual Learning Survey: https://arxiv.org/pdf/1909.08383

    1 hr 7 min
  • 58. Christa Zoufal - Quantum Machine Learning

    ## Summary

    AI is currently dominated by Deep Learning and Large Language Models, but there is other very interesting research that has the potential to have great impact on our lives in the future; one of them being Quantum Machine Learning (QML)

    Today on the podcast, I am talking to Christa Zoufal, Quantum Machine Learning Researcher at IBM.

    Christa will explain how Quantum Computing (QC) and Quantum Machine Learning relates to classical computing (CC).

    We will discuss Qbit's, the fundamental information storage and processing unit of quantum computers that in comparison with bits in a classical computers, not only store a single state of zero or one, but a superposition of the two.

    Christa will explain how Quantum algorithms are created by building quantum circuits that operate on those qbits. We will talk about the challenges when operating on those physical qbits that need to be kept at very low temperatures in order to operate without noise perturbing their states, and why one combines multiple physical qbits to a single logical qbit in order to perform error correction.

    How current quantum circuits are limited in the number of logical qbits they can contain and the number of operations; in the sense of the size of an algorithm, they can perform.

    Why these properties make QC is the wrong choice for BigData applications, but QM could in the future be used for applications where very little data is available and quantum algorithms exist that outperform their known classical counterparts.


    ## AAIP Community

    Join our discord server and ask guest directly or discuss related topics with the community.

    https://discord.gg/5Pj446VKNU


    ## TOC

    00:00:00 Beginning

    00:03:21 Guest Introduction

    00:07:08 Quantum Computer (QC) & Quantum Machine Learning (QML)

    00:14:50 Difference between classical computing and QC

    00:24:59 What kind of programs one can run on a QC

    00:38:43 Examples of Quantum Machine Learning

    00:49:25 Current applications of QC

    ## References


    Christa Zoufal: https://www.linkedin.com/in/christa-zoufal/

    IBM Quantum Learning: https://learning.quantum.ibm.com/course/basics-of-quantum-information

    Quantum machine learning course: https://github.com/Qiskit/textbook/tree/main/notebooks/quantum-machine-learning

    Scott Aaronson @ Lex Podcast https://www.youtube.com/watch?v=nK9pzRevsHQ

    State of the Art Bottlenecks: https://arxiv.org/abs/2312.09121

    1 hr
  • 57. Eldar Kurtic - Efficient Inference through sparsity and quantization - Part 2/2

    Hello and welcome back to the AAIP


    This is the second part of my interview with Eldar Kurtic and his research on how to optimiz inference of deep neural networks.


    In the first part of the interview, we focused on sparsity and how high unstructured sparsity can be achieved without loosing model accuracy on CPU's and in part on GPU's.


    In this second part of the interview, we are going to focus on quantization. Quantization tries to reduce model size by finding ways to represent the model in numeric representations with less precision while retaining model performance. This means that a model that for example has been trained in a standard 32bit floating point representation is during post training quantization converted to a representation that is only using 8 bits. Reducing the model size to one forth.


    We will discuss how current quantization method can be applied to quantize model weights down to 4 bits while retaining most of the models performance and why doing so with the models activation is much more tricky.


    Eldar will explain how current GPU architectures, create two different type of bottlenecks. Memory bound and compute bound scenarios. Where in the case of memory bound situations, the model size causes most of the inference time to be spend in transferring model weights. Exactly in these situations, quantization has its biggest impact and reducing the models size can accelerate inference.


    Enjoy.


    ## AAIP Community

    Join our discord server and ask guest directly or discuss related topics with the community.

    https://discord.gg/5Pj446VKNU


    ### References

    Eldar Kurtic: https://www.linkedin.com/in/eldar-kurti%C4%87-77963b160/

    Neural Magic: https://neuralmagic.com/

    IST Austria Alistarh Group: https://ist.ac.at/en/research/alistarh-group/

    47 min
  • 56. Eldar Kurtic - Efficient Inference through sparsity and quantization - Part 1/2

    Hello and welcome back to the AAIP


    If you are an active Machine Learning engineer or are simply interested in Large Language models, I am sure you have seen the discussions around quantized models and all kind of new frameworks that have appeared recently and achieve astonishing inference performance of LLM's on consumer devices.


    If you are curious how modern Large Language Models with their billions of parameters can run on a simple laptop or even an embedded device, than this episode is for you.


    Today I am talking to Eldar Kurtic, researcher in the Alistarh group at the IST in lower Austrian and senior research engineer at the American startup Neural Magic.


    Eldar's research focuses on optimizing Inference of Deep Neural Networks. On the show he is going to explain in depth show sparsity and quantization works, and how they can be applied to accelerate inference of big models, like LLM's on devices with limited resources.


    Because of the length of the interview, I decided to split it into two parts.


    This one, the first part, is going to focus on sparsity to reduce model size and enable faster inference by reducing the amount of memory and compute that is needed to store and run models.

    The second part is going to focus on quantization as a mean to find representations of models with lower numeric precision that require less memory to store and process, while retaining accuracy.


    In this first part about sparsity, Eldar will explain fundamental concepts like structured and unstructured sparsity. How and why they work and how currently we can achieve performant inference of unstructured sparsity only on CPU's and far less on GPU's.


    We will discuss how to achieve crazy numbers of up to 95% unstructured sparsity while retaining the accuracy of models, but why it is difficult to leverage this under quoutes, reduction in model size, to actually accelerate model inference.


    Enjoy.


    ## AAIP Community

    Join our discord server and ask guest directly or discuss related topics with the community.

    https://discord.gg/5Pj446VKNU


    ### References

    Eldar Kurtic: https://www.linkedin.com/in/eldar-kurti%C4%87-77963b160/

    Neural Magic: https://neuralmagic.com/

    IST Austria Alistarh Group: https://ist.ac.at/en/research/alistarh-group/

    52 min
  • 55. Veronika Vishnevskaia - Ontec - Building RAG based Question-Answering Systems

    ## Summary

    Today on the show I am talking to Veronika Vishnevskaia. Solution Architect at ONTEC where she specialises in building RAG based Question-Answering systems.


    Veronika will provide a deep dive into all relevant steps to build a Question-Answering system. Starting from data extraction and transformation, followed by text embedding, chunking and hybrid retrieval to strategies and last but not least methods to mitigate hallucinations of LLMs during the answer creation.


    ## AAIP Community

    Join our discord server and ask guest directly or discuss related topics with the community.

    https://discord.gg/5Pj446VKNU


    ## TOC

    00:00:00 Beginning

    00:03:33 Guest Introduction

    00:08:51 Building Q/A Systems for businesses

    00:16:27 RAG: Data extraction & pre-processing

    00:26:08 RAG: Chunking & Embedding

    00:36:13 RAG: Information Retrieval

    00:48:59 Hallucinations

    01:02:21 Future RAG systems


    ## Sponsors

    - Quantics: Supply Chain Planning for the new normal - the never normal - https://quantics.io/

    - Belichberg GmbH: Software that Saves the Planet: The Future of Energy Begins Here - https://belichberg.com/


    ### References

    Veronika Vishnevskaia - https://www.linkedin.com/in/veronika-vishnevskaia/

    Ontec - www.ontec.at

    Review Hallucination Mitigation Techniques: https://arxiv.org/pdf/2401.01313.pdf

    Aleph-Alpha: https://aleph-alpha.com/de/technologie/

    1 hr 11 min
  • 54. Manuel Reinsperger - MLSec & LLM Security

    # Summary

    Today on the show I am talking to Manuel Reinsperger, Cybersecurity Expert and Penetration Tester. Manuel will provide us an introduction into the topic of Machine Learning Security with an emphasis on Chatbot and Large Language Model security.


    We are going to discuss topics like AI Red Teaming that focuses on identifying and testing AI systems within an holistic approach for system security. Another major theme of the episode are different Attack Scenarios against Chatbots and Agent systems.


    Manuel will explain to use, what Jailsbreak are and methods to exfiltrate information and cause harm through direct and indirect prompt injection.


    Machine Learning security is a topic I am specially interested in and I hope you are going to enjoy this episode and find it useful.


    ## AAIP Community

    Join our discord server and ask guest directly or discuss related topics with the community.

    https://discord.gg/5Pj446VKNU


    ## TOC

    00:00:00 Beginning

    00:02:05 Guest Introduction

    00:05:16 What is ML Security and how does it differ from Cybersecurity?

    00:25:56 Attacking chatbot systems

    00:41:12 Attacking RAGs with Indirect prompt injection

    00:54:43 Outlook on LLM security



    ## Sponsors

    - Quantics: Supply Chain Planning for the new normal - the never normal - https://quantics.io/

    - Belichberg GmbH: Software that Saves the Planet: The Future of Energy Begins Here - https://belichberg.com/


    ## References

    Manuel Reinsperger - https://manuel.reinsperger.org/

    Test your prompt hacking skills: https://gandalf.lakera.ai/

    Hacking Bing Chat: https://betterprogramming.pub/the-dark-side-of-llms-we-need-to-rethinInjectGPT: k-large-language-models-now-6212aca0581a

    AI-Attack Surface: https://danielmiessler.com/blog/the-ai-attack-surface-map-v1-0/

    https://blog.luitjes.it/posts/injectgpt-most-polite-exploit-ever/

    https://github.com/jiep/offensive-ai-compilation

    AI Security Reference List: https://github.com/DeepSpaceHarbor/Awesome-AI-Security

    Prompt Injection into GPT: https://kai-greshake.de/posts/puzzle-22745/

    1 hr 6 min
  • 53. Peter Jeitscko - Impact of EU AI Regulation on AI startups

    ## Summary

    At the end of last year, the EU-AI Act was finalized and it spawned many discussions and a lot of doubts about the future of European AI companies.


    Today on the show Peter Jeitschko, founder of JetHire an AI based recruiting platform that uses Large Language models to help recruiters find and work with candidates, talks about this perspective on the AI-Act.


    We talk about the impact of the EU AI-Act on their platform, and how it falls into a high-risk use-case under the new regulation. Peter describes how the AI-Act forced them to create their company in the US and what he believes are the downsides of the EU regulation.


    He describes his experience, that the EU regulations hinder innovation in Austria and Europe and how it increases legal costs and uncertainty, resulting in decision makers shying away in building and applying modern AI systems.


    I think this episode is valuable for decision makers and founders of AI companies, that are affected by the upcoming AI Act and struggle to make sense of it.


    ## AAIP Community

    Join our discord server and ask guest directly or discuss related topics with the community.

    https://discord.gg/5Pj446VKNU


    ## TOC

    00:00:00 Beginning

    00:03:09 Guest Introduction

    00:04:45 A founders perspective on the AI Act

    00:13:45 JetHire - A recruiting platform affected the the AI Act

    00:19:58 Achieving regulatory goals with good engineering

    00:35:22 The mismatch between regulations and real world applications

    00:48:12 European regulations vs. global AI services


    ## Sponsors

    - Quantics: Supply Chain Planning for the new normal - the never normal - https://quantics.io/

    - Belichberg GmbH: Software that Saves the Planet: The Future of Energy Begins Here - https://belichberg.com/


    ## References

    Peter Jeitschko - https://www.linkedin.com/in/pjeitschko/

    Peter Jeitschko - https://peterjeitschko.com/

    JetHire - https://jethire.ai/

    https://www.holisticai.com/blog/requirements-for-high-risk-ai-applications-overview-of-regulations


    58 min
  • 52. Markus Keiblinger - Texterous - Building custom LLM Solutions

    # Summary

    For the last two years AI has been flooded with news about LLMs and their successes, but how many companies are actually making use of them in their products and services?

    Today on the show I am talking to Markus Keiblinger, Managing partner of Texterous. A startup that focus on building custom LLM Solutions to help companies automate their business.

    Markus will tell us about his experience when talking and working with companies building such LLM focused solutions.

    Telling us about the expectations companies have on the capabilities of LLMs, as well on what companies need to have in order to be successfully implementing LLM projects.

    We will discuss how Textorous has successfully focused on Retriever Augmented Generation (RAG) use cases.

    RAGs is a mechanism that makes it possible to provide information to an LLM in a controlled menner, so the LLM can answer questions or follow instructions making use of that information. This enables companies to make use of their data to solve problems with LLMs, without having to train or even fine-tune models. On the show, Markus will tell us of one of these RAG projects and we will contrast building a RAG system based on Service Provider offerings like OpenAI or self hosted open source alternatives.

    Last but not least, we talk about new use cases emerging with multi-modal Models, and the long term perspective that exists for custom LLM Solutions Providers like them in focusing on building integrated solutions.


    ## AAIP Community

    Join our discord server and ask guest directly or discuss related topics with the community.

    https://discord.gg/5Pj446VKNU


    ## TOC

    00:00:00 Beginning

    00:03:31 Guest Introduction

    00:06:40 Challenges of applying AI in medical applications

    00:17:56 Homogeneous Ensemble Methods

    00:25:50 Combining base model predictions

    00:40:14 Composing Ensembles

    00:52:24 Explainability of Ensemble Methods


    ## Sponsors

    - Quantics: Supply Chain Planning for the new normal - the never normal - https://quantics.io/

    - Belichberg GmbH: Software that Saves the Planet: The Future of Energy Begins Here - https://belichberg.com/


    ### References

    - Markus Keiblinger: https://www.linkedin.com/in/markus-keiblinger

    - Texterous: https://texterous.com

    - Book: Conversations Plato Never Captured - but an AI did: https://www.amazon.de/Conversations-Plato-Never-Captured-but/dp/B0BPVS9H9R/

    47 min