Software Daily

Software Daily

By SoftwareDaily.comNewsTech News
Download on the App Store

Software Daily episodes

  • Alluxio: Data Orchestration with Haoyuan Li

    In 2009, the Berkeley AMPLab was a center of innovation. 

    Three projects from AMPLab have turned into successful open source projects and companies: Spark, Mesos, and Alluxio. Haoyuan Li was the creator of Alluxio, and he returns to the show to discuss his journey taking Alluxio from a research project to a company that has customers including Alibaba, Baidu, Wells Fargo, and Samsung.

    Alluxio is a distributed, fault-tolerant, tiered storage system. Alluxio allows application developers to think in terms of the latency that they require from their infrastructure rather than the details of different storage systems. Haoyuan discusses the process of integrating with gigantic companies like cloud providers, telecoms, and huge ecommerce companies.

    Alluxio is also hosting an upcoming conference, the Data Orchestration Summit November 7th at the Computer History Museum in Mountain View California.

    November 7, 2019 is the first open source Data Orchestration Summit in the Computer History Museum in Mountain View, CA. The summit brings together data engineers, data platform engineers, and data scientists to share their challenges and learnings from building and using modern analytics, AI, and cloud technologies. Featuring tech talks covering use cases, demos, best practices, and tutorials by industry experts from EA, Walmart, DBS Bank, Netflix, AWS, Rakuten, Tencent, Google, Baidu, Alibaba and more, with a focus on how to build cloud native analytics & AI platforms. Use code “SDE” to get a 50% discount.

    0 min
  • Redis with Alvin Richards

    Redis is an in-memory database that persists to disk. Redis is commonly used as an object cache for web applications.

    Applications are composed of caches and databases. A cache typically stores the data in memory, and a database typically stores the data on disk. Memory has significantly faster access times, but is more expensive and is volatile, meaning that if the computer that is holding that piece of data in memory goes offline, the data will be lost.

    When a user makes a request to load their personal information, the server will try to load that data from a cache. If the cache does not contain the user’s information, the server will go to the database to find that information. 

    Alvin Richards is chief product officer with Redis Labs, and he joins the show to discuss how Redis works. We explore different design patterns for making Redis high availability, or using it as a volatile cache, and we talk through the read and write path for Redis data. Full disclosure: Redis Labs is a sponsor of Software Engineering Daily.

    0 min
  • LinkedIn Data Platform with Carl Steinbach

    LinkedIn is a social network with petabytes of data. 

    In order to store that data, LinkedIn distributes and replicates that data across a large cluster of machines running the Hadoop Distributed File System. In order to run calculations across its large data set, LinkedIn needs to split the computation up using MapReduce-style jobs.

    LinkedIn has been developing its data infrastructure since the early days of the Hadoop ecosystem. LinkedIn started using Hadoop in 2008, and in the last 11 years, the company has adopted streaming frameworks, distributed databases, and newer execution runtimes like Apache Spark.

    With the popularization of machine learning, there are more applications for data engineering than ever before. But the tooling around data engineering means that it is still hard for developers to find data sets, clean their data, and build reliable models. 

    Carl Steinbach is an engineer at LinkedIn working on tools for data engineering. In today’s episode, Carl discusses the data platform inside LinkedIn, and the strategies that the company has developed around storing and computing large amounts of data. 

    Full disclosure: LinkedIn is a sponsor of Software Engineering Daily.

    0 min
  • Crypto Businesses with Haseeb Qureshi

    Cryptocurrencies are a fundamental computer science invention. 

    Cryptocurrencies crashed in 2018 but the technology remains as promising as ever. Bitcoin is a decentralized currency, and a plausible end state that is implied by Bitcoin’s current trajectory is a permissionless, decentralized financial system.

    This idea of decentralized finance or “DeFi” begs numerous questions: who will build the companies that provide the infrastructure for decentralized finance? Who will be the lenders? Who will be the credit agencies? Who will be the escrow services? How big will the teams need to be? Will these systems be built on smart contracts, or can it be done with centralized cloud providers?

    Haseeb Qureshi is a managing partner with DragonFly Capital and a frequent guest on SE Daily. He returns to the show to discuss his thesis on what kinds of crypto companies make sense, and how he thinks about investments into crypto startups. Haseeb recently wrote an article about how to build a crypto startup.

    0 min
  • Dark Lang with Ellen Chisa and Paul Biggar

    Dark Lang is a programming language that is tightly integrated with the cloud. 

    Dark takes an opinionated approach that most developers are going to want to run their applications in the cloud, and this perspective influences how Dark looks at deployments, IDEs, exception handling, and other aspects of software development.

    Paul Biggar is the founder of CircleCI, and ran the company for eight years before leaving to found Dark with Ellen Chisa. Ellen is a software engineer and the CEO of Dark. Paul and Ellen join the show to give their perspective on modern software engineering, and why it was time to build a new high level language that assumes the presence of a cloud.

    It is difficult to get programmers to adopt a new language. It is even harder to get those programmers to pay for products built around that language. But the timing could be perfect for Dark. 

    Software development is undergoing tremendous change–and many of these changes work to Dark’s advantage, such as the growing adoption of feature flags, low code tools, and sophisticated continuous delivery workflows. Whether or not Dark is a success, it is a bold project, and the team is working on something they believe in.

    We also discussed AWS–and whether the largest cloud provider has an obligation to contribute back to the open source community.

    0 min
  • Gravity: Distributed Application Delivery with Ev Kontsevoy

    Modern applications are distributed systems. These applications require an installation mechanism that can run and update the software across multiple nodes. 

    When a SaaS company starts to work with large enterprise customers, that company needs to figure out a way to deliver their product to the enterprise. This requires the SaaS company to deploy the product to whatever infrastructure the enterprise is running.

    Some enterprises use on-prem infrastructure. Some use AWS. Some use a variety of cloud providers and on-premise servers. A SaaS company with limited resources must be able to have a standard deployment model that satisfies all of these different use cases.

    Ev Kontsevoy is the CEO of Gravitational, a company that builds software for application delivery. Ev’s company maintains Gravity, an open source tool for application imaging and delivery. Ev was also the founder of Mailgun, a popular email API service. Mailgun was acquired by Rackspace, and in his time running Mailgun, Ev became deeply aware of the problems faced by developers and operators who manage server infrastructure.

    Ev joins the show to discuss his experience building companies and the state of modern infrastructure. Full disclosure: Gravitational is a sponsor of Software Engineering Daily.

    0 min
  • Incident Reproduction with Tammy Butow

    Databases go offline. Services fail to scale up. Deployment errors can cause an application backend to get DDoS’d.

    When an event happens that prevents your company from operating as expected, it is known as an incident. Software teams respond to an incident by issuing a fix. Sometimes that fix returns the software to its ideal state. Other times the software remains in a degraded state, and it takes more fixing to return the software to the place it should be.

    One way that a software team can learn from an incident is through incident reproduction. When an incident is turned into a reproducible system, it becomes a predictable training exercise rather than a surprising and painful outage.

    Tammy Butow is an engineer with Gremlin, a company that makes chaos engineering software. Chaos engineering is the process of creating controlled experiments that simulate outages. Tammy joins the show to discuss common incident types, and how those can be made reproducible for training exercises.

    0 min
  • Incident Reproduction with Tammy Butow

    Databases go offline. Services fail to scale up. Deployment errors can cause an application backend to get DDoS’d.

    When an event happens that prevents your company from operating as expected, it is known as an incident. Software teams respond to an incident by issuing a fix. Sometimes that fix returns the software to its ideal state. Other times the software remains in a degraded state, and it takes more fixing to return the software to the place it should be.

    One way that a software team can learn from an incident is through incident reproduction. When an incident is turned into a reproducible system, it becomes a predictable training exercise rather than a surprising and painful outage.

    Tammy Butow is an engineer with Gremlin, a company that makes chaos engineering software. Chaos engineering is the process of creating controlled experiments that simulate outages. Tammy joins the show to discuss common incident types, and how those can be made reproducible for training exercises.

    0 min
  • Diffbot Infrastructure with Mike Tung

    Diffbot is a knowledge graph that allows developers to interface with the unstructured web as if it was a structured database. In today’s show, Diffbot CEO Mike Tung returns for a second discussion about how he has built Diffbot and how Diffbot is used.

    The web has many different entities. Web pages, topics, people, stories, articles, companies, and much more. Humans use a search engine to find answers to their questions within web pages. Machines need to find answers to these kinds of questions as well, but a machine is not sophisticated enough to figure out answers from an unstructured web page.

    Diffbot brings structure to those webpages, and gives them an API interface for developers to build on top of. In order to create this system in a cost-efficient manner, Diffbot runs its own data centers, where web scraping, machine learning, and API infrastructure are all used to build the Diffbot application.

    Mike joins me for an interview about creating Diffbot, as well as his strategy for running the business.

    0 min
  • How To Build A Cloud Provider with Anurag Goel

    Render is a cloud provider built on top of Amazon Web Services and Google Cloud. Render uses the compute abstractions provided by the major cloud providers to build a second layer cloud provider with the goal of providing a better user experience.

    Anurag Goel is the founder of Render, and he returns to the show to discuss how Render works, and why there is a need for a new cloud provider. 

    Everyone knows that the market for cloud providers is gigantic, so why are so few companies pursuing it? From Anurag’s perspective, there is no good reason. AWS, Google, Azure, Digital Ocean, and Heroku have only explored a small percentage of the potential ways a cloud provider could be built.

    Anurag shares his strategy for building Render, and also talks through his belief around modern software engineering, including his belief that developers mostly choose their tools based on what they read from popular websites rather than what solves their problems–a phenomenon which he describes as “fashion driven development.”

    0 min

About Software Daily

From the publisher's feed

Technical interviews about software topics.