The New Stack Podcast

At KubeCon in Copenhagen in May, many talks focused on the work required to build continuous integration and continuous deployment pipelines using containers. One of the major issue still remaining in the container world is specifically that last bit of the CI/CD pipeline: building, storing, and securing containers built for internal software projects.

Steve Speicher, principal product manager on the Red Hat OpenShift Team, spent a good deal of time at KubeCon looking into the solutions and remaining pain points that exist around dynamically building and managing containers within a more traditional agile development environment. "A lot of people want to leverage Kubernetes for the build in the pipeline itself, and so that's one of the things we're talking about here and learning more about what people are interested in to leverage the platform to do more CI/CD," said Speicher.

Ben Parees, Principal Engineer at Red Hat, "You have people who have build CI/CD farms and infrastructure and then their deployment platform. With Kubernetes and OpenShift you have the opportunity to put that all in one, so your cluster is both your build platform, your test platform, and your deployment platform. It's easy to scale up multiple instances, to test them, run your builds there," said Parees.

When building front-end software, it can be tricky to figure out just what works. As with any page layout endeavor, from the Web to the supermarket checkout line tabloids, there are plenty of nooks and crannies to explore with headlines, graphics, and colors. Any software shop earning money on the Web likely already knows about "A/B" testing: the practice of subtle changing your page design and gathering metrics on its effectiveness at converting visitors versus the existing version of the site.

Now that such testing regimes are commonplace in the enterprise, it is inevitable that every team eventually encounters the egregeous and exhausting existentialist crisis that is test management. Gathering metrics for a single test is one thing, but what happens when the entire enterprise is pushing tests across thousands of sites all the time?

Claire Vo is Silicon Valley success story: she sold her startup Experiment Engine to Optimizely in 2017. Her particular winning formula was to help solve this exact problem for enterprises: Managing experiments at scale across thousands of sites, and measuring results in order to effect actionable changes overall.

With all the excitement around containers and Kubernetes, it can be easy to forget that these systems still require the same types of help that older virtual machine and hardware-based systems needed. Chief among that list of needs is security. We sat down at KubeCon in Copenhagen to discuss this very topic with Liz Rice, Technology Evangelist at Aqua Security, Justin Cappos, Associate Professor Computer Science and Engineering at the NYU - Tandon School of Engineering.

Cappos is one of the driving forces behind the TUF Project, which stands for "The Update Framework." "We had a pretty long history of going and doing a lot of work with folks at the Tor project and other large software distributions and maybe had concerns about nation-state actors maybe stepping in. About 3 or 4 years ago the Docker community came together and build a really nice implementation of TUF Notary, and as of about 6 months ago, both the Docker implementation, which is the cloud focused implementation of TUF, and the TUF specification itself became CNCF Projects," said Cappos.

Watch on YouTube: https://youtu.be/mNFoqxnuecg

With all the excitement around containers and Kubernetes, it can be easy to forget that these systems still require the same types of help that older virtual machine and hardware-based systems needed. Chief among that list of needs is security. We sat down at KubeCon in Copenhagen to discuss this very topic with Liz Rice, Technology Evangelist at Aqua Security, Justin Cappos, Associate Professor Computer Science and Engineering at the NYU - Tandon School of Engineering.

Cappos is one of the driving forces behind the TUF Project, which stands for "The Update Framework." "We had a pretty long history of going and doing a lot of work with folks at the Tor project and other large software distributions and maybe had concerns about nation-state actors maybe stepping in. About 3 or 4 years ago the Docker community came together and build a really nice implementation of TUF Notary, and as of about 6 months ago, both the Docker implementation, which is the cloud focused implementation of TUF, and the TUF specification itself became CNCF Projects," said Cappos.

Watch on YouTube: https://youtu.be/G2V3IlVVls8

Data lakes are an enticing idea: pour all of a company's information--both structured and unstructured--into a single system, and then everyone can access it and analyze it on demand. It's a great idea, but in practice, actually getting that data back out of the lake and into the hands of the people who need it can be a lot more difficult than it sounds.

Kelly Stirman, CMO and vice president of Dremio, has been working with data for decades. In a past life, he worked on strategy at MongoDB, and before that, MarkLogic. In his latest role, he's tasked with helping companies find ways to solve their data lake problems. Chief among those problems, he said, is making that data usable by those who need it.

At ChefConf 2018 in Chicago in late May, we sat down with Chef CTO and co-founder Adam Jacob, and Senior VP of Products and Engineering Corey Scobie, to discuss the cultural currents in the enterprise. Specifically, they focused on the proliferation of DevOps and DevOps tools within the business community. One of the things that is potentially a roadblock for uptake of DevOps in the enterprise has been the internal culture of many organizations. With so many new ideas and tools out there, it can feel like your company doesn't get it at all, and even perhaps fears DevOps.

Jacob said that the whole IT organization inside of an enterprise must change to properly embrace DevOps, and he said this must even extend to the opinion IT has of itself. If an organization is doing the wrong thing, or is impeding the uptake of DevOps, said Jacob, administrators and IT workers must push to change those impediments, not resign themselves this being the state of things forever and always.

"When you know something's not right in the world, in your company, in your organization, or whatever, you have two choices. You go back to work and you accept that, or you go back to work and you do something about it. One of the themes we've been talking about this week is that IT needs to change their thinking in the enterprise to continue to be successful and relevant, and to actually serve the business," said Corey Scobie, Senior Vice President of Product and Engineering at Chef.

Watch on YouTube: https://www.youtube.com/watch?v=5WPx6M0fzO0

Yaron Haviv, CTO and co-founder of Iguazio joins TC Currie for this episode of The New Stack Makers.  Haviv’s deep technological experience in the fields of big data, cloud, storage and networking led to him leading the team creating Iguazio’s new product nuclio.

This open-source project, launched in December 2017, is an extremely fast serverless platform with a real-time processing engine.  Extremely fast meaning 100x faster than AWS Lambda. “Everyone knows that the value of data diminishes over time,” said Haviv, “so that means when the data starts flowing into the system, you need to start aggregating contextualizing and acting on it as it flows.”

The question was how to process data at extreme speed and at scale, moving from the notion that data is something that you store and query to thinking of data as something you continuously process.

Michael Hausenblas, Red Hat developer advocate for OpenShift, still prefers the term Functions as a Service. In the push to expand the serverless capabilities of Kubernetes, Hausenblas, at KubCon in May, told us that one of the key desires for enterprises is to be able to host Amazon Lambda-like functionality internally as well as in public clouds. He expects as more users adopt serverless functions, that developers will increasingly become comfortable with function accessibility via APIs.

Hausenblas then detailed some of the changes this shift would bring to developers and their tools. "On the upside, I see a lot of positive developments so that it's pretty easy to actually write something. You can write it any language, as long as it's in Node.js. From a developer point of view, I don't think this is a big problem. It's more on the operational side. So if you actually think about who is going to be on call when you're operating that at scale , since you don't necessarily have any administrators around any more," said Hausenblas.

Watch on YouTube: https://youtu.be/MwX2FPBCR2o

Artificial intelligence may be at the peak of its hype cycle for modern businesses, but for the IT administrator it is still a headache, requiring software and processes that may entirely outside the normal sphere of operations. But help is on the way. At Kubecon + CloudNativeCon 2018 last month, David Aronchick introduced the first working version (0.1) of Kubeflow, software that packages the most popular AI and machine learning software so it can be easily run on the Kubernetes container orchestration engine.

"We can't ask people to be experts in everything," Aronchick told the audience at Kubecon, in a keynote introducing the technology.  He noted that today's AI tools, such as Jupyter Notebooks or TensorFlow, have to be managed individually and often at great effort (the "bespoke" model). Kubeflow could rationalize this unruly set of software, but offering a common platform upon which they can all be based, and offered easily to the end-user, the data scientist.

Watch on YouTube: https://youtu.be/S7qpvr2bZ2U

In the race to make this weird, wild world of distributed, containerized applications compatible with the virtualized infrastructure upon which most enterprises depend, perhaps no project has made more progress than Kata Containers.  The product of collaboration between the Hyper.sh project and Intel’s Clear Containers, Kata aims to pair individual containers with hypervisors, creating that direct link with the hardware that typifies first-generation virtualization, and isolating host Linux kernels from one another.

Google’s recent gVisor project follows a similar path, creating a minimal Linux kernel for the container hosts that reduces the likelihood of exploit.

Some folks contend these architectures may render many of the more aggressive security systems being conceived for containerized environments unnecessary or redundant.  But in a conversation for The New Stack Makers, Aqua Security co-founder and CTO Amir Jerbi told us he believes that even the mode of process isolation gVisor and Kata introduce, would carry with it into practice some security challenges.  Try orchestrating a microservices environment with isolated instances in a multi-tenant environment, he suggests, and see what happens.