Software Engineering Daily

Software Engineering Daily

By Software Engineering DailyNewsTechnologyTech News
Download on the App Store

Software Engineering Daily episodes

  • Distributed Systems and Exception Monitoring with Brian Rue
    Exception monitoring services and log management services are two sides of a gradient. Exception monitoring services capture and aggregate the problems that occur on your application. Log management services aggregate all of your logs, so that you can decide for yourself what constitutes a problem. Brian Rue from Rollbar joins the show today to talk about Rollbar’s exception monitoring architecture, and the competitive landscape of these technology products. Every softwareContinue reading...
    38 min
  • Google’s Container Management with Brendan Burns


    Kubernetes is an open source system for automating deployment, operations, and scaling of containerized applications. Google developed Kubernetes after fifteen years of running containers in production.
    Brendan Burns is a founder of the Kubernetes project, and he joins us to talk about the lessons learned as Google has built containerized applications to distribute across its massive infrastructure. We talk about Docker, Borg, Kubernetes, and other distributed systems technologies.

    Applications crash and engineers need to be able to quickly find the root cause of a crash. Apps have become distributed, and debugging workflows have changed. Developers need better tools to identify and troubleshoot problems with their apps.

    The post Google’s Container Management with Brendan Burns appeared first on Software Engineering Daily.

    45 min
  • Google’s Container Management with Brendan Burns
    Kubernetes is an open source system for automating deployment, operations, and scaling of containerized applications. Google developed Kubernetes after fifteen years of running containers in production. Brendan Burns is a founder of the Kubernetes project, and he joins us to talk about the lessons learned as Google has built containerized applications to distribute across its massive infrastructure. We talk about Docker, Borg, Kubernetes, and other distributed systems technologies. Applications crash,Continue reading...
    47 min
  • Search as a Service with Julien Lemoine

    “You need to build more things yourself to be highly available, but one of the very good consequences of being bare metal is that the prices are very low compared to what you could get on the cloud provider.”

    Engineers who want to add search to their application usually deploy Elasticsearch, or write their own search engine that uses TF-IDF. These solutions work well for large documents, but are less effective for large volumes of small records–which is how many modern web and mobile applications are structured.

    In today’s show, Julien Lemoine discusses how his company Algolia thinks about search. Algolia is a search as a service company that gives developers an easier way to search on their websites and applications.

    Questions
    • What are the unsolved problems in search?
    • What is TF-IDF, and how is it is used to rank search results?
    • Why did you decide to set up your own servers instead of going to a cloud provider?
    • Why is it useful to divide search into two different aspects?
    • How do you stress-test your system?
    • How did you respond to your recent data center connectivity issue?
    • How do you build a company oriented for the long-term?
      • Links
      • Algolia
      • Elasticsearch
      • tf-idf
      • Bare-metal servers
      • Multitenancy model
      • GAE
      • Chaosmonkey testing strategy
      • ZooKeeper
      • Raft
      • On premise
      • Julien on Twitter

        • The post Search as a Service with Julien Lemoine appeared first on Software Engineering Daily.

        51 min
      • Search as a Service with Julien Lemoine
        “You need to build more things yourself to be highly available, but one of the very good consequences of being bare metal is that the prices are very low compared to what you could get on the cloud provider.” Engineers who want to add search to their application usually deploy Elasticsearch, or write their own search engine that uses TF-IDF. These solutions work well for large documents, but are lessContinue reading...
        53 min
      • Managing a CDN with Carl Gustas

        “We’re not always in control of other people’s networks.”

        CDN stands for content delivery network. A content delivery network is a system of distributed servers that delivers web pages and other web content. Without CDNs, the internet would be much slower, because CDNs function as a caching layer for most web resources.

        Carl Gustas is an engineer at CacheFly, a popular content delivery network. He joins us today to discuss how CDNs work, and the different methods an engineer can use to take advantage of caching in a CDN.

        Questions
        • What is a proxy server, and how is it relevant to a CDN?
        • Why is a first hit to a website painful whereas subsequent hits are not?
        • Does a CDN provide resilience from DDOS attacks?
        • Do you use containers in production?
        • What is the replication factor for my files on a CDN?
        • What are the craziest stories you have from managing a CDN for 10 years?
          • Links
          • Ingress point (CDN overview)
          • CDN
          • Delivery edge
          • Cachefly
          • Carl on Twitter

            • The post Managing a CDN with Carl Gustas appeared first on Software Engineering Daily.

            40 min
          • Managing a CDN with Carl Gustas
            “We’re not always in control of other people’s networks.” CDN stands for content delivery network. A content delivery network is a system of distributed servers that delivers web pages and other web content. Without CDNs, the internet would be much slower, because CDNs function as a caching layer for most web resources. Carl Gustas is an engineer at CacheFly, a popular content delivery network. He joins us today to discussContinue reading...
            41 min
          • Logging and NoOps with Christian Beedgen

            “You write the code, but you don’t run it? That’s just preposterous.”

            Software applications are constantly generating logs. These logs are necessary to understand how an application is functioning, and logs are key to debugging. As applications have gotten more complex, logging infrastructure has become complex as well. Storing and managing all of our log data is such a big task that several companies have been started to tackle this problem.

            Today’s guest is Christian Beedgen, CTO at Sumo Logic. Sumo Logic is a cloud-based log management company. We discuss the elastic log-processing platform Sumo Logic has built to help software engineers with log management. It’s a great conversation about distributed systems, machine learning, and debugging applications.

            Questions
            • How has logging changed for applications in the cloud?
            • What do you mean when you say “log data is big data”?
            • What kinds of elasticity are important in your log architecture?
            • What is the ingestion path for the logs?
            • Are there any interested distributed systems problems at Sumo Logic?
            • Why do you want to replicate three times?
            • How do you use machine learning to improve log management?
            • Can you describe what NoOps is, and how it reflects Sumo Logic’s culture?
              • Links
              • Sumo Logic
              • tail
              • Elastic log processing (video)
              • Multitenant architecture
              • Inverted index
              • Lucene
              • JSON logs
              • NoOps Debate Grows Heated
              • Christian on Twitter

                • The post Logging and NoOps with Christian Beedgen appeared first on Software Engineering Daily.

                54 min
              • Logging and NoOps with Christian Beedgen
                “You write the code, but you don’t run it? That’s just preposterous.” Software applications are constantly generating logs. These logs are necessary to understand how an application is functioning, and logs are key to debugging. As applications have gotten more complex, logging infrastructure has become complex as well. Storing and managing all of our log data is such a big task that several companies have been started to tackle thisContinue reading...
                56 min
              • Scaling Email with J.R. Jasperson

                “As the scale continues to increase, certain effects of architecture become less and less efficient.”

                When you spend money online, you expect a receipt to come in your email. When you register for a new web site, you need to verify your sign up in your email. These types of emails are called “transactional email” and sending these types of email at scale is a complex engineering task.

                J.R. Jasperson is the chief architect at SendGrid, a transactional email platform. On this episode, we discuss how email works–from the basics to the massive scale that SendGrid operates on. We also talk about email spam and fraud in detail.

                Questions
                • Why is transactional email important for software engineers?
                • What are the engineering problems associated with sending lots of emails?
                • Where can emails “fall through the cracks” so to speak?
                • What are the scaling issues you face and how have you dealt with them?
                • How is scaling an email service different from scaling a high-availability website like Netflix?
                • What did you learn from the previous “tech crashes” and how does it reflect on the current climate?
                • What is an engineering problem around email that you would never have encountered in another business?
                  • Links
                  • Sendgrid
                  • SMTP
                  • Mail server
                  • Amazon SES
                  • Kafka
                  • Egress filtering
                  • “Nobody ever got fired for choosing IBM”
                  • J.R. on Twitter

                    • The post Scaling Email with J.R. Jasperson appeared first on Software Engineering Daily.

                    51 min

                  About Software Engineering Daily

                  From the publisher's feed

                  Technical interviews about software topics.