The Python Podcast.init

By Tobias MaceyTechnology Education

The Python Podcast.init episodes

July 22, 2017Moving to MongoDB with Michael Kennedy
Summary
There are dozens of decisions that need to be made when building an application. Sometimes this can lead to analysis paralysis and prevent you from making progress, so don’t let the perfect be the enemy of the good. This week Michael Kennedy shares his experience with evolving his application architecture when his business needs outgrew his initial designs.
Preface
Hello and welcome to Podcast.__init__, the podcast about Python and the people who make it great.
I would like to thank everyone who supports us on Patreon. Your contributions help to make the show sustainable.
When you’re ready to launch your next project you’ll need somewhere to deploy it. Check out Linode at www.podastinit.com/linode?utm_source=rss&utm_medium=rss and get a $20 credit to try out their fast and reliable Linux virtual servers for running your awesome app.
Visit the site to subscribe to the show, sign up for the newsletter, read the show notes, and get in touch.
To help other people find the show please leave a review on iTunes, or Google Play Music, tell your friends and co-workers, and share it on social media.
Your host as usual is Tobias Macey and today I’m interviewing Mike Kennedy about his work scaling his apps and his business
Interview
Introductions
How did you get introduced to Python?
In some of your recent episodes you have mentioned the work that you did to migrate your applications to run on MongoDB. Can you start by describing the business case for these applications and how you arrived at the initial design?
What was the limiting factor that led you to consider such a drastic shift in how you store and manage your data and what benefits did you gain when the work was complete?
If the issue was with scaling, how did you identify the choke points?
Why go from relational (SQLite) to document (Mongo) instead of what would seem a more obvious choice of a production grade relational engine such as PostGreSQL or MySQL?

Are there any particular synergies that arise from using a document as opposed to a relational store when working with Python and what are some of the main considerations when deciding between them?

What was happening in your business that precipitated the need for this work?

How are you talking to MongoDB from Python? Directly (via pymongo) or ORM-style?
Why did you make that choice?
How well is that working out? Advantages / drawbacks?

In addition to podcasting you have also been working to create a number of successful courses to teach people how to use Python. Is there anything specific to the language that translates into how you design the material?

For anyone who wants to learn more about the benefits and tradeoffs of using a document store with their Python applications, what are some resources that you recommend?

Keep In Touch
Michael
@mkennedy on Twitter
Websites
Talk Python
Python Bytes

Picks
Tobias
Org Mode
Levar Burton Reads

Mike
Newspaper
Robomongo (now Robo 3T)
The Dark Secret at the Heart of AI
Haibike SDURO Cross 4.0

Links
SQLAlchemy
SQLIte
MySQL
PostGreSQL
NoSQL
MongoDB
Database Normalization
Foreign Keys
Document Database
Rollbar
MongoEngine
Mongo Security Checkup
MLab
MongoDB Atlas
MongoDB World
O’Reilly Python Mongo Book
MongoDB For Python Developers
Momentum Dash
The intro and outro music is from Requiem for a Fish The Freak Fandango Orchestra / CC BY-SA
48 min
July 16, 2017 Zulip Chat with Tim Abbott
In modern work environments the email is being edged out by group chat as the preferred method of communication. The majority of the platforms used are commercial and closed source, but there is one project that is working to change that. Zulip is a project that aims to redefine how effective teams communicate and it is already gaining ground. This week Tim Abbott shares the story of how Zulip got started, how it is built, and why you might want to start using it.
1 hr 1 min
July 16, 2017Zulip Chat with Tim Abbott
Summary
In modern work environments the email is being edged out by group chat as the preferred method of communication. The majority of the platforms used are commercial and closed source, but there is one project that is working to change that. Zulip is a project that aims to redefine how effective teams communicate and it is already gaining ground. This week Tim Abbott shares the story of how Zulip got started, how it is built, and why you might want to start using it.
Preface
Hello and welcome to Podcast.__init__, the podcast about Python and the people who make it great.
I would like to thank everyone who supports us on Patreon. Your contributions help to make the show sustainable.
When you’re ready to launch your next project you’ll need somewhere to deploy it. Check out Linode at www.podastinit.com/linode?utm_source=rss&utm_medium=rss and get a $20 credit to try out their fast and reliable Linux virtual servers for running your awesome app.
Visit the site to subscribe to the show, sign up for the newsletter, read the show notes, and get in touch.
To help other people find the show please leave a review on iTunes, or Google Play Music, tell your friends and co-workers, and share it on social media.
Your host as usual is Tobias Macey and today I’m interviewing Tim Abbott about Zulip, a powerful open source group chat platform
Interview
Introductions
How did you get introduced to Python?
What is Zulip and what was the initial inspiration for creating it?
Where does the name come from?

My understanding is that the project was initally intended to be a commercial product. Can you share some of the history of the acquisition by Dropbox and the journey to open sourcing it?

How has your experience at Dropbox influenced the evolution and implementation of the Zulip project?

There are a large number of group chat platforms available, both commercial and open source. How does Zulip differentiate itself from other options such as Slack or Mattermost?

Typically real-time communication is difficult to achieve in a WSGI application. How is Zulip architected to allow for interactive communication?

What have been the most challenging aspects of building and maintaining the Zulip project?

What is involved in installing and running a Zulip server?
For a large installation, what are the options for scaling it out to handle greater load?

There is a large and healthy community that has built up around the Zulip project. What are some of the methods that you and others have used to foster that growth and engagement?

What has been the most unexpected aspect of working on Zulip, whether technically or in terms of the community around it?

What do you have planned for the future of Zulip?

Keep In Touch
Zulip
Chat
@zuliposs on Twitter

Tim
@tabbott3 on Twitter
Website

Picks
Tobias
Lego Mindstorms EV3

Tim
Checklist Manifesto
Tim’s Recipe Wiki

Links
Zulip
Ksplice
Electron
React Native
IFTTT
Zapier
Zephyr
Barn Owl
MyPy
Tornado
Django
Zulip Tornado Documentation
MySQL
PostGreSQL
ElasticSearch
Code Triage
Emoji
Podcast.__init__ Zulip Chat
The intro and outro music is from Requiem for a Fish The Freak Fandango Orchestra / CC BY-SA
1 hr 1 min
July 09, 2017 NAPALM with David Barroso and Mircea Ulinic
Routers and switches are the stitches in the invisible fabric of the internet which we all rely on. Managing that hardware has traditionally been a very manual process, but the NAPALM (Network Automation and Programmability Abstraction Layer with Multivendor support) is helping to change that. This week David Barroso and Mircea Ulinic explain how Python is being used to make sure that you can watch those cat videos.
59 min
July 09, 2017NAPALM with David Barroso and Mircea Ulinic
Summary
Routers and switches are the stitches in the invisible fabric of the internet which we all rely on. Managing that hardware has traditionally been a very manual process, but the NAPALM (Network Automation and Programmability Abstraction Layer with Multivendor support) is helping to change that. This week David Barroso and Mircea Ulinic explain how Python is being used to make sure that you can watch those cat videos.
Preface
Hello and welcome to Podcast.__init__, the podcast about Python and the people who make it great.
I would like to thank everyone who supports us on Patreon. Your contributions help to make the show sustainable.
When you’re ready to launch your next project you’ll need somewhere to deploy it. Check out Linode at www.podastinit.com/linode?utm_source=rss&utm_medium=rss and get a $20 credit to try out their fast and reliable Linux virtual servers for running your awesome app.
Visit the site to subscribe to the show, sign up for the newsletter, read the show notes, and get in touch.
To help other people find the show please leave a review on iTunes, or Google Play Music, tell your friends and co-workers, and share it on social media.
Your host as usual is Tobias Macey and today I’m interviewing David Barroso and Mircea Ulinic about NAPALM (Network Automation and Programmability Abstraction Layer with Multivendor support), the library for managing programmable network devices
Interview
Introductions
How did you get introduced to Python?
[david] 2012 trying to use django 1.4 to store data I had on confluence.
[mircea] August 2008, when I bought the Learning Python, Mark Lutz, 2nd edition

Can you start by explaining what NAPALM is and the problem that you were solving when you started working on it?
[david] trying to remove all the if vendor_a do this, elif vendor_b do this other thing instead
[mircea] only if I will feel there’s anything to add

What led you to choose Python as the language for implementing it?
[david] it’s what I knew best and vendors were starting to provide libraries to interact with their platforms so python seemed like a natural evolution as we could just provide an abstraction on top of those libraries that already existed.
[mircea] I didn’t implement NAPALM, I was fistly a user then contributor, now I’m one of the maintainers.

When working with network equipment it is easy to apply the wrong settings and bring down a large number of systems or lock yourself out entirely. Are there any tools in NAPALM to help prevent this from happening?
[david] We provide mechanisms to ensure proper peer reviewing; we let operators propose a configuration and get a diff. We have a rollback mechanism so if you detect an issue you can immediately rollback and we also added support to the autorollback feature some vendors have.

How have you architected the library to allow for easy integration of new classes of network devices?
[david] very simple architecture. Trying to avoid complex features like abstract classes, metaprogramming or decorators. Main reason is that I figured my main user base wasn’t going to be very python savvy so I wanted something simple. What I ended doing was simulating interfaces with with a base class that described the supported methods and how they were supposed to behave and an extensive testing framework that ensure the method signatures and the behaviors matched the expectations.

Designing and building a consistent API for such a wide variety of hardware and software platforms is a daunting task. How do you determine the lowest common set of functionality that you are going to expose as part of the core library vs delegating to the underlying dependencies?
[david] We don’t necessarily go with the lowest common denominator. Sometimes we try to emulate features. For example, if a platform doesn’t support atomic changes we might simulate it by trying to send the configuration as a block and rollback immediately. Obviously a feature likes this is clearly documented so people is aware that this might happen. What we try to avoid though is implementing things that are very specific to a single vendor. In any case the way it has worked so far falls into two categories:
configuration management. These are primitives like loading a candidate configuration for merging or replacing into the device, getting a diff back, commiting, discarding or rolling back configuration. These primitives were designed at the very begining of the project based on the netconf protocol and they have changed very little since then. When a primitive is not natively supported by a device we try to emulate it as with the atomicity example I gave before or we don’t implement it at all if it’s not possible.
The second category is what we call getters which are methods that retrieve information from the devices. Things like interface counters, bgp neighbors, etc. These are basically community driven. Someone opens an issue on github explaining the data that he or she needs, we discuss it, we define a model and then we work on it. Not all getters are supported on all platforms. People mostly implements them as they need.
Now there is a third category though. It is actually funny but I presented napalm for the first time a couple of years ago at NANOG64. It turns out the day after, at the same venue, Google was presenting Openconfig. Openconfig is an effort to design a common set of models to operate the network. So, for example, they have models for BGP neighbors, for interfaces, vlans, etc… Those models try to be vendor agnostic and you should, in theory, be able to use them to configure or to retrieve consistently information from any device. Problem is that, of course, vendors are slow implementing them, they don’t even have plans for all of them or for all the platforms, etc… So the sad truth is that two years later support for Openconfig is extremely limited. However, in the last few months I have been working on integrating napalm with opencofig so now we have a beta version of napalm where you can use python bindings that can translate native data from a device into an Openconfig object and viceversa. That has two direct implications:
Now we are not only operating all vendors with the same tool but we are also operating them with the same data structures. This means that I can get the configuration of a cisco device and translate it directly to junos configuration.
It also means that because now we are dealing with objects, I can do smart things like having an object that represents the candidate configuration, anotther object that represents a certain running state and simulate merges myself without having to rely on the device itself. I can even generate the exact commands to do the merge without having to rely on them doing the actual merge. I can also simulate the changes offline, I don’t even need access to the device anymore, I could be builting the objects from a backup or from the resulting configuration after merging different branches on github.

I have seen a few posts recently discussing the use of NAPALM in conjunction with configuration management platforms such as SaltStack and Ansible. What are the tradeoffs of using the library directly vs integrated with these other tools?
[david] napalm is a library in the strict sense. There is no business logic, no workflows, very little tooling embedded. Instead we try to implement as many primitives and be as flexible as possible so other tools can leverage on napalm to implement their workflows. What this means is that using napalm directly is great if you are writing a script to do backups or to solve a specific issue but if you want to build a whole framework for event driven automation or a configuration management system you are probably better off leveraging on napalm integration with salt/ansible/st2.

I noticed in the documentation that merging configuration is supported. How do you manage conflicts and priority of nested data structures?
[david] we try to make changes atomic. So if you make a change and trigger a conflict or you are missing some datastructure or some configuration is invalid configuration won’t be applied and the user will get an error. For platforms where changes can’t be atomic we try to apply the configuration changes in bulk and revert immediately if there is an error.

How does declarative modeling of network devices differ from general purpose operating systems and what unique challenges do they pose?
[david] lack of tooling like sed/awk/etc. Lots of state. Configuration is state itself and in most cases you can’t even reload it. Which means you have to type the exact commands to go from state a to state b. Like trying to configure the network stack of linux with only the iproute2 tooling available.

What are the most technically challenging aspects of managing different network hardware programmatically?
[david] Inconsistencies and buggy code. Not even inconsistencies across different platforms but across minor revisions of the same platforms. Small API changes that are not backwards compatible, small differences on output commands that break regular expressions and APIs that break every second call.

What are some of the most interesting or unusual uses of NAPALM that you have seen?
[david]
I have seen people replacing their SNMP based monitoring system with napalm.
I have built myself what you could call “immutable infrastructure for the network”. So for example, when you have to do a configuration change you don’t apply that configuration change. What you do instead is compile a full configuration for the device and fully reload the state of the device. That ensures you are always into a known state. So if a user would connect to the device and do a change outside the change control system because you are fully deploying state you can be certain that the manual change will be wipeout. So there is no way out of the automation.
We also have this validate functionality integrated into napalm. With this functionality you can define a desired state, for example certain BGP neighbors have to be up and I must be receiving N prefixes from them. Napalm can then read those rules, figure out which data to retrieve and validate the data retrieved complies so I know some people using this state validation instead of using the traditional times series type of monitoring where you keep retrieving data constantly and alerting when you reach certain thresholds. I guess you could call this test driven monitoring?
[mircea]
SNMP thing

For someone who is interested in learning more about network management, what resources do you recommend?
[david]
networktocode.com has some resources, labs, the slack community behing the organization is very active as well.
ipspace.com has some good resources as well.
pynet.twb-tech.com is also another great place to check for courses
o’reilly has a book on Network Programmability and Automation which I haven’t read but I know the authors are very good so I am confident the content will be of high quality.
[mircea]
I blog about NAPALM & generally networking and network automation on my personal space: mirceaulinic.net
packetpushers.net

Keep In Touch
David
@dbarrosop on LinkedIn, GitHub and Twitter
Blog

Mircea
Blog
@mirceaulinic on LinkedIn, GitHub and Twitter

NAPALM
@napalm_auto on Twitter
Documentation

Picks
Tobias
The Twelve Networking Truths
Falsehoods Programmers Believe About Networking

David
The fear saga
59 min
July 02, 2017 Automat State Machines with Glyph Lefkowitz
The venerable 'if' statement is a cornerstone of program flow and busines logic, but sometimes it can grow unwieldy and lead to unmaintainable software. One alternative that can result in cleaner and easier to understand code is a state machine. This week Glyph explains how Automat was created and how it has been used to upgrade portions of the Twisted project.
50 min
July 02, 2017Automat State Machines with Glyph Lefkowitz
Summary
The venerable ‘if’ statement is a cornerstone of program flow and busines logic, but sometimes it can grow unwieldy and lead to unmaintainable software. One alternative that can result in cleaner and easier to understand code is a state machine. This week Glyph explains how Automat was created and how it has been used to upgrade portions of the Twisted project.
Preface
Hello and welcome to Podcast.__init__, the podcast about Python and the people who make it great.
I would like to thank everyone who supports us on Patreon. Your contributions help to make the show sustainable.
When you’re ready to launch your next project you’ll need somewhere to deploy it. Check out Linode at www.podastinit.com/linode?utm_source=rss&utm_medium=rss and get a $20 credit to try out their fast and reliable Linux virtual servers for running your awesome app.
Visit the site to subscribe to the show, sign up for the newsletter, read the show notes, and get in touch.
To help other people find the show please leave a review on iTunes, or Google Play Music, tell your friends and co-workers, and share it on social media.
Your host as usual is Tobias Macey and today I’m interviewing Glyph about Automat, a library that provides self-service finite-state machines for the programmer on the go.
Interview
Introductions
How did you get introduced to Python?
What is a state machine and when might you want to use one?
There are a number of libraries available on PyPI that facilitate the creation of state machines. Why did you feel the need to build a new option and how does it differ from what was already available?
Why do you think developers fall into the trap of complicated conditional structures rather than reaching for a state machine?
For someone who wants to integrate Automat into their project how would they go about that and what are some of the gotchas that they should keep in mind?
What do the internals of Automat look like and how did you approach the overall design of the project?
What are some of the more difficult aspects of designing and implementing state machines properly?
What are some of the technical hurdles that you have been faced with in the process of building a library for implementing state machines?
What do you have planned for the future of Automat?
What are some of the most interesting use cases of Automat that you have seen?
Keep In Touch
Email
@glyph on Twitter
Glyph on GitHub
Picks
Tobias
Commercial Electric color changing LED puck lights

Glyph
OmniFocus
GTD

Links
Automat
Glyph Interview About Software Ethics
Finite State Automaton
Yacc
Bison
Flex Parser Generator
PyPI State Machine
Pure Mealy Machine
Moore Machine
Mealy vs. Moore Machines
Leaky Abstraction
The Law of Leaky Abstraction

Twisted

Python Descriptor

GraphViz

Hypothesis

PyCon Talk – TLS State Machine

The intro and outro music is from Requiem for a Fish The Freak Fandango Orchestra / CC BY-SA
50 min
June 24, 2017 Nuclear Engineering with Dr. Katy Huff
Access to affordable and consistent electricity is one of the big challenges facing our modern society. Nuclear energy is one answer because of its reliable output and carbon-free operation. To make this energy accessible to a larger portion of the global population further reasearch and innovation in reactor design and fuel sources is necessary, and that is where Python can help. This week Dr. Katy Huff talks about the research that she is doing, the problems facing the nuclear industry, and how she uses Python to make it happen.
39 min
June 24, 2017Nuclear Engineering with Dr. Katy Huff
Summary
Access to affordable and consistent electricity is one of the big challenges facing our modern society. Nuclear energy is one answer because of its reliable output and carbon-free operation. To make this energy accessible to a larger portion of the global population further reasearch and innovation in reactor design and fuel sources is necessary, and that is where Python can help. This week Dr. Katy Huff talks about the research that she is doing, the problems facing the nuclear industry, and how she uses Python to make it happen.
Preface
Hello and welcome to Podcast.__init__, the podcast about Python and the people who make it great.
I would like to thank everyone who supports us on Patreon. Your contributions help to make the show sustainable.
When you’re ready to launch your next project you’ll need somewhere to deploy it. Check out Linode at www.podastinit.com/linode?utm_source=rss&utm_medium=rss and get a $20 credit to try out their fast and reliable Linux virtual servers for running your awesome app.
Visit the site to subscribe to the show, sign up for the newsletter, read the show notes, and get in touch.
To help other people find the show please leave a review on iTunes, or Google Play Music, tell your friends and co-workers, and share it on social media.
Your host as usual is Tobias Macey and today I’m interviewing Dr. Katy Huff about using Python for nuclear engineering
Interview
Introductions
How did you get introduced to Python?
Can you start by explaining what nuclear engineering is and give some examples of current research in the field?
The most widely used and recognized form of nuclear plant is the light water reactor, which, to my understanding, is also the most susceptible to melt-downs and release of radioactive material carried by escaped steam. What are some of the reactor types that are currently being researched to improve safety and efficiency?
One of the major policy and logistics issues regarding nuclear power plants is the problem of how to handle spent fuel rods. What are some of the methods that are being researched to solve this problem?
In your PyCon presentation you mentioned the Cyclus and PyNE projects as tools that you use in your research. Can you provide a brief overview of each and explain how you use them?
What are some of the most pressing issues in nuclear engineering and how are you leveraging Python to help with addressing them?
How does open source software relate to open science, and how do they impact the impact the ways that research is performed?
What are some of the current or future developments in nuclear engineering that you are most excited about?
Keep In Touch
Website
Twitter
Research
Picks
Tobias
Ryobi Tools

Katy
Atomic Awakening
Atomic Accidents
Atomic Adventures

Links
Plasma
Nuclear Energy
Thorium
Uranium
Molten Salt Reactor
Spent fuel rods
Yucca Mountain
Nuclear Fuel Reprocessing
Sodium Cooled Fast Reactor
PyCon Keynote
PyNE
Cyclus
Anthony Scopatz
Moose Framework
Partial Differential Equations
REPL (Read Eval Print Loop)
Stellarator
Toroidal Fusion Device
Journal of Open Source Software (JOSS)
American Nuclear Society
NEI
IAEA
The intro and outro music is from Requiem for a Fish The Freak Fandango Orchestra / CC BY-SA
39 min
June 18, 2017 Industrial Automation with Jonas Neubert
We all use items that are produced in factories, but do you ever stop to think about the code that powers that production? This week Jonas Neubert takes us behind the scenes and talks about the systems and software that power modern facilities, the development workflows, and how Python gets used to tie everything together.
1 hr 3 min