Pulling the Strings

In this episode of the Puppet Labs Podcast, I was joined by Gareth Rushgrove, who works for the UK Government and is the person responsible for the DevOps Weekly Newsletter. He goes by garethr on the Puppet Forge. I was also joined by Ryan Coleman, product owner for the Forge at Puppet Labs.

We talked about quite a few module best practices, with a focus on reusability of modules.  One important thing to keep in mind when designing reusable modules is how you abstract the data associated with modules and how you design the entry points for others. Using other people's modules as dependencies is another way to improve reusability, but you also need to make sure that your dependencies are solid, reliable modules.

One of the most important aspects of module development is testing. Gareth and Ryan both talked about various ways to test your modules to improve reliability and check for unintended consequences when you make change to your module. Rspec-puppet is one good option for testing your modules if you are just getting started.

To learn more about module best practices and reliability, you will need to listen to the podcast! You can also check our many other recent podcasts by visiting our podcast page or subscribing in your favorite podcast tool.

• Listen to our other podcasts.

• Join the Puppet Community.

• Watch Gareth's PuppetConf talk about Module Reusability or other great PuppetConf module talks.

• Visit the Puppet Forge to learn more about how to build great modules or use other people's modules.

Solomon Hykes is the founder and CTO of Docker (formerly Dotcloud), and he talks about Docker and the rise of Linux containers. Hykes compares and contrasts the differences between a container and a virtual machine, and about how Docker can make it easier for developers to develop their application within a controlled Linux environment. Ops can receive containers from developers, and then deploy them in any way that they want, whether it's on the public or private cloud.

Solomon also briefly mentions how configuration management solutions and containers can co-exist. Since the recording of this interview, there a Docker module has been released on the Puppet Forge. Yelp's Tomas Doran also recently gave a talk at Puppet Camp London, "Taking control of chaos with Docker and Puppet."

Be sure to check out James Turnbull, a former executive at Puppet Labs (and now at Docker), talking about how he sees the co-existence of configuration management and lightweight containers.

• Check out Gareth Rushgrove's Docker module on the Puppet Forge.

Eric Redmond is an engineer at Basho, which produces the NoSQL database Riak. We talked with Eric about design patterns for distributed systems.

Eric wanted to de-mystify the creation of highly available systems, so he wrote an example in Ruby for his OSCON presentation. He wanted to show that it's doable by an average programmer, and that you don't have to use an industrial-strength programming language like Erlang.

Eric talks about the many different tradeoffs involved in implementing some of these different distributed design patterns, including speed, capacity, uptime and data integrity

Eric describes Riak as a distributed system that persists data rather than a database that has been given distributed functionality. He explains the primary distribution mechanisms of replication and sharding, and how Riak deals with a cascading failures through primary and secondary replication.

It's that ability to deal with failure that Eric says makes Riak operations friendly: It's able to handle the loss of multiple nodes—he's seen up to half go down in a Riak deployment—while maintaining the ability to write and read all of the data.

Finally, he talks about some of the upcoming features of Riak, including commutative replicated data types (CRDT), which makes it easier to merge and resolve conflicts between replicated states.

• For more information, be sure to check out Eric's OSCON presentation "Distributed Patterns in Action"

In this episode of the Puppet Labs Podcast, I was joined by Kenn Hussey, Michelle Johansen and Heidi Pio to talk about how project managers help bridge the gap between developers and operations.

For example, our recent migration from Redmine to JIRA has required extensive collaboration between our developers and our operations teams across many months to make sure that we could complete the migration without disrupting work on our products.

We also have quite a few cross-functional teams between engineering and operations,  pulled together to handle issues with developer infrastructure, security concerns and much more on an ad hoc basis. The Puppet Forge engineering team has a representative from our operations team who attends its meetings. Across all of our projects, we try to get operations involved at the very beginning of each project.

For much more about project management and DevOps, please have a listen! You can also check our many other recent podcasts by visiting our podcast page or subscribing in your favorite podcast tool.

If all of this sounds interesting and you want to join us, we are also hiring two new project managers, an Engineering Project Manager and a Technical Operations Project Manager. Come work with us, we have whiskey.

Learn More

• Take the DevOps survey, help us understand DevOps uptake in enterprise IT, and get a chance to win a PS4, XBox One or some great tech books.

• Listen to our other podcasts.

• Join the Puppet Community.

• Follow our DevOps December content

Conversations With Puppet Labs is a regular podcast featuring voices from the IT and open source communities. Subscribe here.

Chandra Krintz is the CTO of AppScale Systems, an open source, cloud platform that mirrors all of the Google App Engine's APIs. You can use AppScale to run any Google App Engine application on your own private cloud infrastructure.

Since, according to Krintz, APIs aren't protected by copyright, companies like AppScale are free to reverse engineer the internal functionality of Google's App Engine. They don't have complete visibility into how features are implemented, but they can test their service by running a variety of different applications to see if it produces the same results. AppScale is also a Google Cloud Platform technology partner, so get some insight into future changes.

AppScale started as an academic research project. Chandra was interested in answering a couple of questions: How can you profile application workloads in order to make them run more efficiently? How can you simplify the application development process?

In order to test some of their theoretical performance innovations, they decided to mirror Google's App Engine. That provided them with a large set of existing applications they could use to test their ideas against real application workloads.

She also talks about some of the other features they've implemented, including auto-scaling, fault tolerance and elasticity, automatic failover and migration of running workloads.

AppScale hopes to make it easier for more and more people to get involved developing applications. They support a number of languages commonly used for web services, including Java, Python, Go and PHP.

• Read about Puppet Enterprise support for Google Compute Engine.

Denise Paolucci is a self-taught programmer who works at Dreamwidth Studios. She's been talking about a cultural trend nearly everyone experiences, but few want to talk about: impostor syndrome.

According to Paolucci, "Impostor syndrome is that feeling that you don't know what you're doing, that you're not as competent as everyone around you, and that at any minute everyone around you is going to figure out that you're faking and and expose you as a fraud."

When you're suffering from impostor syndrome, you convince yourself that you're the only person who feels that way, so it's spread through a vicious cycle of silence. Denise is trying to change that by providing a forum to discuss the issue, and to help people understand they're not the only ones experiencing impostor syndrome.

Denise said about three-quarters of the room raised their hands to say that they've experienced one of these common impostor syndrome thoughts:

• "I'm a failure"

Denise says that the more people talk about impostor syndrome in public, in person, and within the context of a safe group, then the more people will want to discuss it, and learn how to cope with it. It's part of demystifying the taboo around how people experience impostor syndrome, and the impact that it has on a culture -- like making it impossible for you to take pride in your work or to admit that you don't know something.

Denise provided some great guidance for how to deal with impostor syndrome:

• First off, be willing to talk about the issue with others and check in with your friends to get more perspective.

Impostor syndrome is really difficult to change if there's not leadership and buy-in from the top. If you work with a management culture that doesn't accept failure, then it can create an environment where people try to cover up or ignore their mistakes and breeds a culture of only having overly confident people or people who are really good at faking it.

Denise suggests that you talk to your managers, show them the evidence, and encourage them to do the following things:

• Be willing to help other people out and don't be dismissive of questions, especially if you're in a position of power.

For even more, be sure to check out her deck:

Let us know if you've experienced impostor syndrome, and how you dealt with it.

• Learn how to impostor syndrome-proof yourself and your community from Mary Gardiner and Valerie Aurora of the Ada Initiative

In the latest Puppet Podcast, Community team member Kara Sowles is joined by long-time Puppet developer Jeff McCune along with Cody Herriges and Matt Kirby from the technical operations Team.

The engineering staff at Puppet Labs has been growing quickly, which has meant that a lot of development and testing infrastructure has had to adapt and change. It used to be feasible to have each developer test Puppet on either their own laptop or within a single testing infrastructure. But with Puppet supporting more and more operating systems, including AIX and Windows, it's no longer feasible for all the tests to happen locally. Having one QA environment that any developer can render useless also doesn't work too well at scale.

So in this podcast, we explore how the development and operations teams at Puppet Labs are collaborating on a project to create more isolated virtual and physical test environments for each developer. This is enabling developers to be a bit more experimental and free to break things in their environment without worrying about slowing anyone else down.

They're also coming up with a solution to eliminate all the time-consuming steps needed to get that space by creating a variety of self-service interfaces to different parts of their infrastructure.

Their goal is to give new developers the ability to write a patch to Puppet code on their first day of work, then run it through their own QA environment that's optimized for continuous delivery. Jeff talks about how he able to interact with VMware's vSphere API using a combination of fog and the rbvmomi Ruby library.

Jeff, Cody and Matt also talk about the some of their work building out software-defined infrastructure with a network virtualization layer. Tools like Razor, a bare-metal provisioning tool, are sometimes difficult to consistently test within the context of a laptop environment, and are much better tested within the VMware infrastructure maintained by operations. Some other challenges with Razor include changing the behavior of DHCP or PXE boot yourself while not interfering with other production environments or the primary wireless network. The group talks about implementing OpenStack's Neutron networking interface (formerly quantum) and VMware's NSX network in order to achieve this.

Cody also talks about some specific considerations building an automated cloud infrastructure with dynamic storage capabilities. Specific considerations include creating a distributed storage architecture that eliminates the possibility of single-points of failure taking out the entire storage infrastructure for other virtual machines. There are also different optimizations for handling different heavy read vs. heavy write workloads, as well as hosting it within a private cloud or public cloud.

One specific challenge involves gathering enough metrics to know both how to grow and when to expand capacity. Some of the tools being explored include setting up a Ceilometer infrastructure to collect measurements within OpenStack environments.

Finally, the group talks about the process of designing and architecting a boundless infrastructure that is architected for scalability, reliability and elasticity; abstracted enough to plug in new capabilities in the future, yet stable enough that the ops team can sleep through the night.

You can also check out our many other recent podcasts by visiting our podcast page or subscribing in your favorite podcast tool.

• Join the Puppet Community

We caught up with site engineer Jonathan Thurman about his "Five Unsung Tools of DevOps," wherein he highlights some of the lesser known tools in use at New Relic. The tools he uses every day include:

• MUltihost SSH Wrapper: Distributed SSH to send commands to multiple hosts either in sequence or parallel.

Some of the other tools brought up in the discussion include NetDisco and ClusterSSH.

The thing these tools all have in common is that they allow developers to get better access to their systems so they can get access to machines, get logs and run any relevant processes.

Thurman talks about how New Relic handles the boundaries between operations and development, since the lines are becoming more diffuse over time, and how DevOps culture and practices are more about working together. He also talks about how New Relic uses configuration management tools like Puppet in its toolchain. Finally, he sees the next big movement of DevOps as moving tools like Puppet down into the developer's workflow for even more collaboration between operations and developers.

For more information, be sure check out more details in Thurman's Unsung Tools of DevOps presentation slides from OSCON.

Thurman also recently converted this presentation into a free ebook from O'Reilly, which you can download from here.

• Read more DevOps December content, and get ready for the DevOps survey, coming soon.

Ross Gardler of Microsoft Open Technologies talks about how Microsoft is contributing to open source projects. The company wants not only to ensure that open source products run well on Windows, but also that Linux-based products can run well on Windows Azure.

Ross talks about the flexibility that Azure provides through interoperability with different operating systems. He also talks about the private and public cloud capabilities of Azure, and how you can migrate processes from one to the other through the same management interface.

Finally, he talks about the the process of doing a cloud-based trial of your product in order to attract more users or open source developers. Azure offers a range of different options from IaaS to PaaS to SaaS, and he talks about the process of building and deploying images to Azure.

• Microsoft contributes to a variety of open source communities and projects. Learn more about Azure and open source software.

Andy Piper is a developer advocate for Cloud Foundry, an open source, Platform-as-a-Service (PaaS) solution similar to Heroku and OpenShift.

We caught up with Andy at OSCON this year, and he talked about some of the differences between PaaS and Infrastructure as a Service (IaaS) solutions like OpenStack, Apache CloudStack, Amazon EC2 or VMware's vSphere. One of the things that differentiates Cloud Foundry from other cloud PaaS solutions is that it is agnostic about which cloud backend you choose to use. Unlike some other PaaS solutions, Cloud Foundry is open source, and all of its code can be found on the Cloud Foundry GitHub account.

Cloud Foundry focuses on providing an application-centric solution that enables the frictionless development and deployment of apps from your laptop to production. Piper talks a bit about how they use Warden containers, their lightweight provisioning solution.

Cloud Foundry makes it possible to deploy to different cloud infrastructures. Because of the differences in auto-scaling among the various platforms, scaling out applications often requires human intervention. Piper talks about the reasons for this, and why this is a difficult problem to solve. He hopes to see more policy-based scaling sometime in the future.

Finally, he talks about Cloud Foundry's high-availability and Blue/Green deployment capabilities, as well as how multi-datacenter deployments are achieved at the load balancing layer.

• Learn about Puppet Enterprise and cloud management.