
Sign up to save your podcasts
Or


Augmented reality applications are slowly making their way into the world of the consumer. Pokemon Go created the magical experience of seeing Pokemon superimposed upon the real world. IKEA’s mobile app lets you see how a couch would fit into your living room, which has a significant improvement on the furniture buying process.
Augmented reality applications can have even more dramatic impact on industrial enterprises.
Have you ever set up a factory? You might need to build a conveyor belt. You might need to put together the parts of a giant machine that extrudes steel. You might need to fix a silicon wafer fabrication machine. It takes an expert to set up these heavy, complicated machines.
ScopeAR is a company that builds augmented reality tools. One of the ScopeAR products allows users to telepresence with each other to collaborate on the construction and maintenance of heavy machinery.
Imagine I am setting up my factory, and I have a complicated piece of machinery (let’s say a conveyor belt) in front of me. I have never constructed a conveyor belt before. I put on a HoloLens, and set up a VoIP call with an expert who has experience with that piece of machinery, and they point out what I need to do by superimposing 3-D arrows, text, and other instructions on my field of vision. They can share my experience and help guide me through the process.
This is such a flexible tool–you can imagine applications for augmented reality assistance being useful in medicine, construction, education and other fields.
Scott Montgomerie is the CEO of ScopeAR and in today’s episode, we talk about the state of AR, how the AR tools from Apple and Google compare, and how the similarity between tools used for mapping the world in AR relate to the tools used to map the world by autonomous cars.
Scott was a great guest, and I hope to have him back on in the future.
We have done some great shows about how to build augmented reality and virtual reality applications. To find these old episodes, you can download the Software Engineering Daily app for iOS and for Android. In other podcast players, you can only access the most recent 100 episodes. With these apps, we are building a new way to consume content about software engineering. They are open-sourced at github.com/softwareengineeringdaily. If you are looking for an open source project to get involved with, we would love to get your help.
Shout out to today’s featured contributor Edgar Pino. He is working on a real-time chat application for Software Engineering Daily, so that we can have chat rooms for people to discuss the episodes easily. Innovative work!
Transcript provided by We Edit Podcasts. Software Engineering Daily listeners can go to weeditpodcasts.com/sed to get 20% off the first two months of audio editing and transcription services. Thanks to We Edit Podcasts for partnering with SE Daily. Please click here to view this show’s transcript.
Computational load is the amount of demand that is being placed on a computer system. “Load” can take the form of memory, CPU, network bandwidth, disk space, and other finite resources.
When we design systems, we need to prepare for high-load events. On a social network, people are much more active in the mornings. On an e-commerce site, Black Friday causes many more users to come online for discount shopping. Our distributed application must be able to scale in response to these spikes in traffic.
Cloud computing has changed the popular software architecture patterns, and load balancing has changed along with it. With on-demand, infinite infrastructure, we don’t need to worry about ordering servers and provisioning. With infrastructure as code, it becomes simpler to manage lots of deployable units–so we can break up our monolith into microservices, and have hundreds or thousands of virtual machines or containers running.
Enterprises that were started before cloud computing have large on-premise server deployments–but today, many of them also use the cloud. The cloud can be used to augment their classic on-prem deployments with cloud platform-as-a-service features. The cloud can also be used as a reliable way to scale during high load events.
Today, a common architectural pattern is to have your application broken up into services. Each of those services has multiple instances. When the load on a particular service is under lots of demand, you create more instances to handle the increased load. How do you monitor the load on each service? How do you know when to spin up new instances of the service?
Load analysis and load balancing across different services can be implemented by placing “agents” throughout your infrastructure. These agents gather data about services and service instances, and route that data to a centralized place. The centralized “control plane” can be used to make decisions about load-balancing and traffic routing.
Ranga Rajagopalan worked on networking at Cisco for a decade before co-founding Avi Networks as CTO. Avi Networks builds modern load balancing software, and in today’s episode, Ranga describes the requirements of load balancing. We talked about the evolution of network infrastructure, the impact of the cloud, and the technical decisions that his team has made when architecting Avi Networks. Full disclosure: Avi Networks is a sponsor of Software Engineering Daily.
Transcript provided by We Edit Podcasts. Software Engineering Daily listeners can go to weeditpodcasts.com/sed to get 20% off the first two months of audio editing and transcription services. Thanks to We Edit Podcasts for partnering with SE Daily. Please click here to view this show’s transcript.
The New York Times is a newspaper that evolved into a digital publication. Across its 166 year history, The Times has been known for longform journalistic quality, in addition to its ability to quickly churn out news stories. Some content on the New York Times is old but timeless “evergreen” content.
Readers of the New York Times website are not only looking for the most recent news–they want to know what the headlines were the day after Pearl Harbor. They want to read editorials about Martin Luther King. Over the last 30 years, New York Times has moved itself online, bringing old material with it.
Since the 90s, several different content management systems (CMS) have been used by journalists within The Times. These different sources of content store data in different formats.
This is a data management problem. Users want to search over the entire history of articles published by The Times, which means that The Times needs to unify those articles in a single index. These are articles from the 1920s that were digitized using OCR, articles from 1998 that were written on a legacy CMS, and articles from 2017 that use the latest CMS.
Boerge Svingen is the director of engineering at NYT, and he wrote about this problem and its solution on Medium. This story describes the flexibility of Kafka; in contrast to the applications of Kafka as a place to buffer high volumes of data, the New York Times uses Kafka as a place to unify data and allow for other specific materialized views to be built on top of it.
We have covered Kafka in the past with interviews of some of its creators–including Jay Kreps and Neha Narkhede. To find these old episodes, you can download the Software Engineering Daily app for iOS and for Android. With these apps, we are building a new way to consume content about software engineering. They are open-sourced at github.com/softwareengineeringdaily. If you are looking for an open source project to get involved with, we would love to get your help.
Transcript provided by We Edit Podcasts. Software Engineering Daily listeners can go to weeditpodcasts.com/sed to get 20% off the first two months of audio editing and transcription services. Thanks to We Edit Podcasts for partnering with SE Daily. Please click here to view this show’s transcript.
A cryptocurrency has a distributed ledger called a blockchain. The blockchain keeps track of every transaction that occurs across the cryptocurrency. This blockchain must stay up-to-date and verified–which requires someone in the network to do that validation.
Bitcoin and Ethereum use the proof-of-work algorithm. Miners do computational work to validate the legitimacy of transactions across the network, and in return they are given cryptocurrency as a reward for that computational work.
In the future, cryptocurrencies could move towards a proof-of-stake model. If you own a significant amount of cryptocurrency, you have incentive to keep the validity of the blockchain up to date. Proof-of-stake algorithms can be significantly less energy intensive.
Vlad Zamfir is a researcher for the Ethereum Foundation, and he joins Haseeb Qureshi for a conversation about cryptoeconomics. This is an in-depth conversation between two active blockchain developers. We hope you enjoy it.
You can send us feedback on the show by emailing me [email protected] or joining us on the Slack channel at softwareengineeringdaily.com/slack.
Transcript provided by We Edit Podcasts. Software Engineering Daily listeners can go to weeditpodcasts.com/sed to get 20% off the first two months of audio editing and transcription services. Thanks to We Edit Podcasts for partnering with SE Daily. Please click here to view this show’s transcript.
In America, the tech companies we focus on are commonly known as FAANG: Facebook, Amazon, Apple, Netflix, Google. We all know what these companies do because they impact our daily lives. In Asia, there are three giant tech companies that have similar scale: Baidu, Alibaba, and Tencent, otherwise known as BAT.
Technology within a location is shaped by the pressures of that location. You might think we live in a global society, but tech in Asia is dramatically different than it is in America. Differences in culture lead to differences in product development.
In China, a different political system contributed to more rapid adoption of online payments. Because there is more payment data, people can be given loans more efficiently. Less of the population is “unbanked.” Online payments are mostly handled by WeChat, a social networking product from Tencent, and Alibaba, an ecommerce giant. If you live in the West, imagine that Facebook and Amazon handled most of your payments for everything. You would have a different relationship with those companies.
Bernard Leong is the host of Analyse Asia, a podcast about Asian developments in technology in business. After studying materials science in Singapore and theoretical physics at Cambridge, he made his way into business and journalism, and developed an interest in the Singularity–a subject that few people took seriously until recently (one topic we explored in this show is Masayoshi Son, the Japanese tycoon who wants to invest nearly a trillion dollars into technology companies; Masayoshi believes firmly that the Singularity is coming).
Shenzhen: The Silicon Valley of Hardware (Full Documentary) | Future Cities | WIRED
In the Plex by Steven Levy
The Hidden Forces Behind Toutiao: China’s Content King
Transcript provided by We Edit Podcasts. Software Engineering Daily listeners can go to weeditpodcasts.com/sed to get 20% off the first two months of audio editing and transcription services. Thanks to We Edit Podcasts for partnering with SE Daily. Please click here to view this show’s transcript.
It’s 9pm at night, and you are hungry. You order a pizza from Domino’s. You live on a street that’s dark, and so you have installed a smart lightbulb in front of your mailbox that lights up the address. When the pizza at Domino’s is ready, you want the lightbulb on your mailbox to light up so that the delivery person can read your address when they arrive in front of your house with the pizza.
The Internet should make it possible to have this kind of event-driven, connected world. Anything that is connected to the Internet should be able to send signals to anything else on the Internet, so that our lives gradually become more automated.
This is what IFTTT does. Users of IFTTT can easily create applets to wire different services together. You can use IFTTT to trigger an email whenever three of your friends retweet something on Twitter. You can use IFTTT to flash the lights in your house when Bitcoin hits new market highs. You can use IFTTT to order a pizza whenever Bitcoin crashes.
IFTTT makes it easy to connect different services together, and a lot of work goes into the infrastructure that enables these billions of events to process correctly. Nicky Leach from IFTTT’s engineering team joins the show to describe how IFTTT allows for integrations between services that were not built to integrate–and he talks about the scheduling, data engineering, and monitoring of the company’s software stack.
Transcript provided by We Edit Podcasts. Software Engineering Daily listeners can go to weeditpodcasts.com/sed to get 20% off the first two months of audio editing and transcription services. Thanks to We Edit Podcasts for partnering with SE Daily. Please click here to view this show’s transcript.
The MapReduce paper was published by Google in 2004. MapReduce is an algorithm that describes how to do large-scale data processing on large clusters of commodity hardware.
The MapReduce paper marked the beginning of the “big data” movement. The Hadoop project is an open source implementation of the MapReduce paper. Doug Cutting and Mike Cafarella wrote software that allowed anybody to use MapReduce, as long as they had significant server operations knowledge and a rack of commodity servers.
Hadoop got deployed first at companies with the internal engineering teams that could recognize its importance and implement it–companies like Yahoo and Microsoft. The word quickly spread about the leverage Hadoop could provide.
Around this time, every large company was waking up to the fact that they had tons of data and didn’t know how to take advantage of it. Billion dollar corporations in areas like banking, insurance, manufacturing, and agriculture all wanted to take advantage of this amazing new way of looking at their data. But these companies did not have the engineering expertise to deploy Hadoop clusters.
Three big companies were formed to help bring Hadoop to large enterprises: Cloudera, Hortonworks, and MapR. Each of these companies worked with hundreds of large enterprise clients to build out their Hadoop clusters and help them access their data. Tomer Shiran spent five years at MapR, seeing the data problems of these large enterprises and observing how much value could be created by solving these data problems.
In 2015, eleven years had passed since MapReduce was first published, and companies were still having data problems. Tomer started working on Dremio, a company that was in stealth for another two years. I interviewed Tomer two years ago, when he still could not say much about what Dremio was doing. We talked about Apache Drill, an open-source project related to what Dremio eventually built.
Earlier this year, two of Tomer’s colleagues Jacques Nadeau and Julien Le Dem came on to discuss columnar data storage and interoperability. What I took away from that conversation was that today, data within an average enterprise is accessible, but the different formats are a problem. Some data is in MySQL, some is in Amazon S3, some is in ElasticSearch, some is on HDFS stored in Parquet files. Different teams will set up different BI tools and charts that read from a specific silo of data.
At the lowest level, the different data formats are incompatible–you have to transform MySQL data in order to merge it with S3 data. On top of that, engineers doing data science work are using Spark, Pandas, and other tools that pull lots of data into memory–if the in-memory formats are not compatible, the data teams can’t get the most out of their work. On top of THAT, at the highest level, data analysts are working with different data analysis tools, so there is even more siloing.
Now I understand why Dremio took two years to bring to market.
They are trying to solve data interoperability by making it easy to transform data sets between different formats. They are trying to solve data access speed by creating a sophisticated caching system. And they are trying to improve the effectiveness of the data analysts by providing the right abstractions for someone who is not a software engineer to study the different data sets across an organization.
Dremio is an exciting project because it is rare to see a pure software company put so many years into up-front stealth product development. After talking to Tomer in this conversation, I’m looking forward to seeing Dremio come to market. It was fascinating to hear him talk about how data engineering has evolved to today.
Some of the best episodes of Software Engineering Daily cover the history of data engineering, including an interview with Mike Cafarella, the co-founder of Hadoop, and another episode called “The History of Hadoop” in which we explored how Hadoop made it from a Google research paper into a multibillion dollar industry.
To find these old episodes, you can download the Software Engineering Daily app for iOS and for Android. With these apps, we are building a new way to consume content about software engineering. They are open-sourced at github.com/softwareengineeringdaily. If you are looking for an open source project to get involved with, we would love to get your help.
Computer chips have physical limitations. When transistors get too small, electrons start to behave in ways that make the hardware modules less reliable. Our reliable technological progress has been enabled by Moore’s Law: the idea that the number of components we can fit on a chip doubles roughly every 12-18 months.
We can’t keep shrinking the size of these components, because physics is no longer complying.
Quantum computing allows us to operate on qubits rather than bits, giving us better parallelism and continued reliable technological progress. Quantum computing is still mostly an area of research rather than production systems–but it is rapidly approaching usability, and Zlatko Minev joins the show to explain how quantum computing works, and why software engineers should care.
Zlatko is a PhD candidate at the Yale Quantum Information Lab. Today he describes how qubits work, which algorithms quantum computing impacts, and which parts of modern computer architecture will work on a quantum computer. We may have to throw out the Von Neumann architecture when it comes to quantum!
Transcript provided by We Edit Podcasts. Software Engineering Daily listeners can go to weeditpodcasts.com/sed to get 20% off the first two months of audio editing and transcription services. Thanks to We Edit Podcasts for partnering with SE Daily. Please click here to view this show’s transcript.
Interviewing engineers is not a solved problem. Quite the opposite–everyone in the software industry will tell you their own personal issues with the hiring process.
One reason that technical interviews have not evolved significantly is the lack of standardized tooling. Some companies give you one phone screen, some give you two. Some companies have you solve brain teasers (“how many golf balls fit in a school bus”) and some make you fix bugs in their production codebase. During the on-site interview, some companies use whiteboards, some let you use a laptop.
Software companies do so much–they should be outsourcing the things that are not their core competency. Certainly they cannot outsource the entire hiring process–but they can outsource parts of it to a company like Interviewing.io.
Engineers come to Interviewing.io to practice their interview skills, where other engineers from top companies practice with them as an interviewer. When an engineer has practiced interviewing enough, they can use Interviewing.io to interview for real with real companies and find a job.
Aline Lerner is the CEO of Interviewing.io, and she knows about the software interviewing and recruiting process as much as anyone. After working as an engineer, she started studying recruiting, consulting with top companies to help them improve their process. From her observations, she created Interviewing.io. In this episode, we dissect the workflow that she created for engineers to improve at interviewing and find jobs, and also explore the insights that led her to starting Interviewing.io.
Transcript provided by We Edit Podcasts. Software Engineering Daily listeners can go to weeditpodcasts.com/sed to get 20% off the first two months of audio editing and transcription services. Thanks to We Edit Podcasts for partnering with SE Daily. Please click here to view this show’s transcript.
Machine learning models can be built by plotting points in space and optimizing a function based off of those points.
For example, I can plot every person in the United States in a 3 dimensional space: age, geographic location, and yearly salary. Then I can draw a function that minimizes the distance between my function and each of those data points. Once I define that function, you can give me your age and a geographic location, and I can predict your salary.
Plotting these points in space is called embedding. By embedding a rich data set, and then experimenting with different functions, we can build a model that makes predictions based on those data sets. Yufeng Guo is a developer advocate at Google working on CloudML. In this show, we described two separate examples for preparing data, embedding the data points, and iterating on the function in order to train the model.
In a future episode, Yufeng will discuss CloudML and more advanced concepts of machine learning.
Transcript provided by We Edit Podcasts. Software Engineering Daily listeners can go to weeditpodcasts.com/sed to get 20% off the first two months of audio editing and transcription services. Thanks to We Edit Podcasts for partnering with SE Daily. Please click here to view this show’s transcript.
From the publisher's feed