Send us a text

When identical prompts yield different outputs and a request for "a cat running across the screen" generates four cylinders and a bobbing balloon, the non-deterministic nature of generative AI becomes impossible to ignore. Despite the power of these sophisticated models, we discovered that creating consistent, high-quality results demanded substantial human intervention – not less. Our journey moved from refining prompts to building extensive code example libraries, requiring the very software engineering expertise these systems supposedly reduce.

This experience directly contradicts the popular "vibe programming" approach where developers mindlessly shuttle between AI suggestions and error messages. While this might suffice for weekend projects (as its originator noted), it produces unmaintainable spaghetti code lacking the architectural vision essential for professional software. Real engineering demands creating systems that scale, maintain, and transfer to other developers – not just code that temporarily functions.

The lessons extend beyond development to any organization implementing AI in decision-making processes. These systems lack the contextual understanding to independently determine business priorities or handle nuanced human factors. The future belongs not to AI replacing humans, but to thoughtful partnerships harnessing each party's unique strengths.

Ready to dive deeper into AI's practical realities? Listen now and join the conversation about how we can responsibly integrate these powerful tools while maintaining human oversight and expertise. Share your experiences with us at [email protected] or @CloudOutLoudPod.

Jon Gallagher on LinkedIn

Logan Gallagher on LinkedIn

The Animation App on Github

Andrej Karpathy on vibe programming

*Extra!* Andrej Karpathy on "Privacy Hygiene" - protecting yourself online now

Send us a text

In this episode we try out what we've speculated AI, particularly GenAI is good for: expanding the capabilities of engineers as they create new applications.

This application uses Google's Cloud Run  and Google Cloud Storage to get user's specification of an animation, create the animation, then store and play the animation.

This the link to the Github repo for the application.

The AI framework for the application is Google's Gemini and the program generates the animation is Blender. The demo is programmed in Python and uses the LangChain framework to leverage Large Language Models (LLMs) in the application

Send us a text

Courtesy of Corey Quinn's "Last Week in AWS" Newsletter # 389 Logan and Jon discovered a presentation from Sec-T  by Chris Farris that does a great job describing the tension between the Cloud Providers and Customers when it comes to security.

The presentation does a great job of highlighting the fact that, though the cloud can be made secure, for many people the defaults they use do not make them as secure as possible. In addition, the cloud providers have to make sure that existing software and systems are not broken by the changes necessary to increase security.

Corey Quinn - Last Week in AWS

Chris Farris

Transcript

Google - Shared Fate

AWS - Security Pillar

Slack - Transitioning to IMDSv2

Send us a text

This episode reviews a manifesto written by Leopold Aschenbrenner, formerly of OpenAI, now a founder of an investment firm that is focused on AGI, Advanced General Intelligence. The manifesto is titled SITUATIONAL AWARENESS, The Decade Ahead.

This episode's discussion sets up who Leopold is, where he's been, and what his focus is. From there the focus is on one aspect of the document, Security. ANd here the discussion looks at what the paper cosiders vulnerabilities, versus the real-world vulnerabilities IT handles if faced with.

SITUATIONAL AWARENESS

GNU Manifesto

Hacker's Manifesto

(... oh what the heck)

Communist Manifesto

Send us a text

As with all the discussions of AI, and particularly LLM's and Generative AI, this is a discussion of the challenges of getting it right, making it useful, and using these tools successfully in a business.

Major Points:

1. There are a large number of commercial and open-source models available.
2. Tens and even hundreds of millions of dollars were spent, and equal amounts of person years were expended to create these models. Your business cannot and should not duplicate this effort and expense.
3. You therefore must know the best way to use the models, and that is to collect enough data about your operations to extend the models to know about your enterprise.
4. You also must have enough subject matter expertise to know when the models are producing incorrect answers, such as "hallucinations".
5. You must be aware of the security challenges the models represent, particularly in sensitive industries such as finance and health care.
6. You must be able to create testing to ensure you know and limit the risks of incorrect answers, and security problems

There is no such thing as a free lunch. You must expend the effort necessary to teach the models you use, test those models, and ensure that you have limited the scope of security problems from the models.

Send us a text

Send us a text

Unveiling the Future of Cloud and AI at Google Cloud Next 

Episode 28: Show Notes

In this episode, we delve into the most recent big cloud conference, Google Cloud Next. We discuss what we saw, felt, and heard at this year's iconic Google Cloud Next and provide listeners with our biggest takeaways from the event. We discuss the cutting-edge tools in generative AI and machine learning that Google has rolled out and the potential these new Google offerings hold for developing robust enterprise solutions. Gain insights into the products and services that have the most potential, the obvious shift in Google’s approach to providing enterprise solutions, the integration of its new tools into current business operations, and much more! We also offer listeners our expectations of Amazon's upcoming conference and why the future of cloud computing is brighter than ever. To stay on the cutting edge of the cloud and AI revolution, tune in now!

Key Points From This Episode:

• Logan shares his thoughts on the conference, particularly on the generative AI content.
• Hear highlights of the new features and services Google had on showcase.
• Discover exciting new generative AI and machine learning tools from Google.
• Other interesting new features that enhance productivity in the cloud.
• The potential that Google’s new features have for developing enterprise solutions.
• Integrating Google’s new tools into a business's process and operations.
• We unpack Google’s noticeable shift in focus and approach to business.
• What we think are the most exciting new features and services.
• An incredible data migration story that was shared at the conference.
• Our expectations for Amazon’s upcoming conference: AWS re: Invent.

Quotes:

“It was really clear that [Google’s] butts were kicked into gear by OpenAI and ChatGPT and they are playing a bit of catchup.” — Logan Gallagher [0:02:50]

“Google is finally embracing the real world [with] a lot of these new tools.” — Jon Gallagher [0:07:49]

“The hard work of an enterprise, the hard work of running a business is still being with the new tools from Google.” — Jon Gallagher [0:11:34]

Links Mentioned in Today’s Episode:

Google Cloud Next 2023

Duet AI

Vertex AI

GKE Enterprise

AWS re: Invent 2023

Jon Gallagher on LinkedIn

Logan Gallagher on LinkedIn

Send us a text

Episode 27: Show Notes

Welcome back to Cloud Out Loud as we continue our discussion on generative AI and machine learning. Today is all about exploring the risks of modern machine learning and how we can properly navigate them as a society. Jon and Logan walk us through the benefits of AI tools for software companies, the dangers of poorly-trained generative AI models, why good code may not always be the kept standard, and how to assess the cost-effectiveness of the machine learning models at your company. Then, we dive into our concerns about the data of large language models, what generative AI could mean for the future of the internet itself, the perils of hallucinated AI data, stochastic parrots and other security vulnerabilities of generative AI, and so much more! To hear about the importance of transparency in machine learning and to find out what we’ll be talking about next week, press play now. 

Key Points From This Episode:

• The risks to consider when implementing AI and/or machine learning in your company.
• Assessing the best AI tools for software companies and the benefits thereof. 
• The importance of accurately separating good code from bad code after the initial prompts. 
• Exploring the dangers of mistraining a generative AI model. 
• How to know when your AI output is valid and how to monitor the system for updates. 
• Balancing costs: how cost-effective is your machine learning model for your business?
• Why we’re concerned about the data that is going into large language models.  
• How we don’t yet know what machine learning models could mean for the internet’s future.  
• Our fears surrounding hallucinated AI data and the (possible) universal adoption of bad code. 
• Some careers that could experience a boom as a result of widespread AI adoption. 
• Stochastic parrots and the lesser known/discussed security vulnerabilities of generative AI.
• What we need to focus on to make generative AI and machine learning more secure.   
• Why more transparency is needed around the data that is produced by generative AI tools.
• Recapping everything we’ve discussed today and what you can look forward to next time. 

Tweetables:

“Cleaning and curating your data is the least sexy but most important part of getting any value out of any of these [generative AI] tools.” — Logan Gallagher [04:39]

“We may be increasingly reaching the point where the internet is going to be so full of AI-generated content that our subsequent versions of generative AI models will be a snake eating its own tail.” — Logan Gallagher [21:36]

“This is something that I worry about much more than Skynet — that we end up with fragile systems or we end up with unknown attack surfaces because of frameworks that are being generated for us without our ability to have an audit trail of how this came to be.” — Jon Gallagher [32:29] 

Links Mentioned in Today’s Episode:

ChatGPT 

GitHub Copilot 

‘Stochastic Parrots: A Novel Look at Large Language Models and Their Limitations’

‘Undetectable backdoors for machine learning models’

Jon Gallagher on LinkedIn
Logan Gallagher on LinkedIn

Send us a text

Generative AI and ChatGPT with Logan and Jon

Episode 26: Show Notes

Machine learning and AI are fast becoming integrated into our everyday lives. However, despite its rising popularity, there is still a lot of confusion and misunderstanding around the subject. In this episode, we unravel the fundamental principles of machine learning and artificial intelligence. We start by setting the context before diving into the technical and business side of AI. We explain the different terms used, why people are so interested in machine learning, and how it is going to shake up Silicon Valley. We also provide listeners with an overview of the benefits and drawbacks of AI and machine learning and discuss how using AI can go wrong. Learn about neural networks, the transformer algorithm, the cost of implementing AI, and how to effectively leverage these technologies. We examine both positive and negative use cases, debunk common misconceptions, and emphasize the continuous nature of AI implementation. Lastly, we navigate the landscape of cognitive computing, exploring the threats it presents along with the opportunities it brings. Tune in now to ensure you do not get left behind in the AI and machine learning race!

Key Points From This Episode:

• Useful definitions and different terms are explained.
• Find out the difference between AI and machine learning.
• What algorithms popular AI tools are based on.
• Hear about exciting new technologies emerging in the space.
• Learn about the power of the transformer algorithm.
• The limitation of AI and machine learning: data.
• How much AI and machine learning can cost companies. 
• Ways companies are leveraging AI to reduce costs.
• An overview of the good and bad use cases of AI and machine learning.
• Common misconceptions surrounding AI and machine learning. 
• Why implementing AI and machine learning is a continuous process.
• Threats and opportunities of cognitive computing. 

Tweetables:

“Artificial intelligence is a broad field of study. It is an umbrella term under which these technologies fit into.” — Logan Gallagher [0:02:42]

“When you are interacting with a model that uses transformer, it can generate very human-readable and human-intelligible text and outputs that pass off as very convincing.” — Logan Gallagher [0:05:52]

“[Deploying new versions of AI] is a continuous process. If you are standing still, you are going to get left behind.” — Logan Gallagher [0:19:43]

“The business opportunity [of AI] is huge here. Thus, we are not only engaged in the standard hype cycle of technology, but we are looking at a Silicon Valley that is figuring out what business it is going to be in.” — Jon Gallagher [0:22:12]

Links Mentioned in Today’s Episode:

ChatGPT

The Transformer Model Tutorial

‘Transformer: A Novel Neural Network Architecture for Language Understanding’

OpenAI

Jon Gallagher on LinkedIn

Logan Gallagher on LinkedIn

Send us a text

Best Environment for Machine Learning

Episode 25: Show Notes

Lately, there’s been a lot of hype about AI. In today’s podcast, we too are going to chat about AI, and specifically the subset of artificial intelligence called machine learning. Instead of talking about the political, social, and moral aspects of this subject, however, we’re going to speak about some of the more mundane aspects of deploying this technology. Tuning in, you’ll hear about some of the recent technologies that have been the subject of hype cycles, what Logan learned about the hype cycle of crypto and how it affected AWS, and how this relates to the current hype cycles of AIML. We then discuss the ML and production experience of the three major cloud platforms (AWS, GCP, and Azure), the ML APIs that these companies have made available, and how you can extend these APIs for your unique requirements. To learn more about how to differentiate between the different cloud providers, the importance of being able to update existing models, the necessity for the automated collection and evaluation of the current model, and so much more, tune in today!

Key Points From This Episode:

• Examples of recent technologies that have been the subject of hype cycles.
• What Logan learned from the hype cycle of crypto and how it affected AWS.
• Google’s level of maturity in terms of AIML, despite seeming behind in this current hype cycle.
• The importance of knowing how to integrate AIML. 
• Three major cloud platforms’ ML and production experience. 
• The types of ML APIs that these companies have made available and some examples.
• AutoML and how you can extend these APIs for your unique requirements. 
• Examples of how you can use this technology in your company, and possible pitfalls. 
• How to differentiate between the different cloud providers, and choosing the right one. 
• What Google’s BigQuery ML is and how it works.
• How each cloud provider has an AIML suite of tools that enables people to train their models.
• Why the ability to update existing models is so important. 
• The necessity for the automated collection and evaluation of the current model for ongoing development of improved models.
• How the software practices that we’ve been learning and implementing over the years, still apply.

Tweetables:

“Maybe some of these companies like OpenAI will emerge as major players moving forward, but I think we can be sure that one of the big winners is guaranteed to be the cloud platforms.” — Logan Gallagher [0:04:01]

“That is the real use case that we can identify for ML; the ability to extend the capabilities of the working software we have.” — Jon Gallagher [0:06:14]

“What’s maybe more important than deploying a model for production is having the ability to update that model.” — Logan Gallagher [0:22:39]

“With ML and AI, there is a temptation to treat this as something new and different, but I really see all of the important software practices that we’ve been learning and implementing over the years, still applying here.” — Logan Gallagher [0:29:01]

Links Mentioned in Today’s Episode:

ChatGPT

“AWS and Blockchain”

AWS

GCP

Azure 

petqts.com 

Snowflake

Jon Gallagher on LinkedIn