
Sign up to save your podcasts
Or


How would you build a system for indexing and monitoring the entire Internet?
Start by breaking the Internet up into IP address ranges. Give each of those address ranges to servers distributed around the world. On each of those servers, iterate through your list of IP addresses, sending packets to them. Depending on what sorts of packets those IP addresses respond to, and what those responses are, you can build a map of the devices on the Internet: what is running on those devices, and what they respond to.
Qadium is a company that indexes and monitors devices on the Internet, to help organizations understand the devices that are within corporate networks. If you are a large corporation, Qadium can probably do a better job of figuring out your Internet footprint than you can.
Matt Kraning is the CTO of Qadium, and in today’s show he describes the process by which Qadium maps the Internet. Matt used to work on data infrastructure at DARPA, and has deployed Hadoop in Afghanistan–so the infrastructure of Qadium seems relatively manageable. Our data conversations in this episode spam from talking about Storm and Hadoop to Google BigQuery, BigTable, and DataFlow.
Transcript provided by We Edit Podcasts. Software Engineering Daily listeners can go to weeditpodcasts.com/sed to get 20% off the first two months of audio editing and transcription services. Thanks to We Edit Podcasts for partnering with SE Daily. Please click here to view this show’s transcript.
Scala is a functional and object oriented programming language built on the JVM. Scala Native takes this language, loved by many, and brings it to bare metal. Scala Native is an optimizing ahead-of-time compiler and lightweight managed runtime designed specifically for Scala.
Denys Shabalin (dennis shuh-blin) is a Research Assistant at the EPFL and the primary creator of Scala Native. In this episode, Adam Bell interviews Denys about the motivations behind the Scala Native project, how it was implemented and future directions. He also briefly touches on how Scala Native made cold compilation times of Scala code twice as fast. If you are interested in functional programming, compiler design, or want to learn some interesting tidbits about garbage collector design and trade offs you will like this episode.
The mobile apps are open sourced at github.com/softwareengineeringdaily. If you are looking for an open source project to hack on, we would love to get your help! We are building a new way to consume software engineering content. We have the Android app, the iOS app, a recommendation system, and a web frontend–and more projects are coming soon. If you have ideas for how software engineering media content should be consumed, or if you are interested in contributing code, check out github.com/softwareengineeringdaily, or join our Slack channel (there’s a link on our website)–or send me an email: [email protected]
Transcript provided by We Edit Podcasts. Software Engineering Daily listeners can go to weeditpodcasts.com/sed to get 20% off the first two months of audio editing and transcription services. Thanks to We Edit Podcasts for partnering with SE Daily. Please click here to view this show’s transcript.
You have heard the phrase: every company is becoming a software company. An insurance company is now supposed to turn into a software company that sells insurance. A clothing retailer needs to reinvent itself to be able to build software to manage the production and distribution of its clothing.
Software applications provide so much leverage to an organization, it seems smart to develop in-house software teams to build those applications. But does it really make sense? Is there a better alternative?
In the 90s outsourcing was a common solution to this problem. If you didn’t have software expertise at your company, you would hire a large consulting firm. These firms would often hire inexperienced offshore developers, and the resulting code quality was not so great.
Because of the bad experiences of the first Internet boom, companies became more cautious about outsourcing their engineering work–which led to today, where the standard is to hire your own software team.
The world has changed in ways that have made outsourcing a more viable solution.
Programming best practices are more widely understood. There is an international community of software engineers that share information on places like Stack Overflow, Quora, and Twitter. Off-the-shelf collaboration tools make it much easier to communicate the requirements of a project to a team of developers.
Gigster is a company that is working to optimize the engineering of software projects. Large enterprises come to Gigster to build new projects from scratch–whether that project is a marketplace, a mobile application, or a machine learning model. Roger Dickey is the CEO of Gigster, and he joins the show to describe how Gigster works, and why it often makes sense for companies to focus on their core competency and outsource software engineering.
Some of our most popular episodes of Software Engineering Daily describe how leading software companies are being built–we have covered Giphy, Netflix, Digital Ocean, Stripe, and many others. Download the Software Engineering Daily app for iOS or Android to hear all of our old episodes. They are easily organized by category, and as you listen, the SE Daily app gets smarter, and recommends you content based on the episodes you are hearing. If you don’t like this episode, you can easily find something more interesting by using the recommendation system.
The mobile apps are open sourced at github.com/softwareengineeringdaily. If you are looking for an open source project to hack on, we would love to get your help! We are building a new way to consume software engineering content. We have the Android app, the iOS app, a recommendation system, and a web frontend–and more projects are coming soon. If you have ideas for how software engineering media content should be consumed, or if you are interested in contributing code, check out github.com/softwareengineeringdaily, or join our Slack channel (there’s a link on our website)–or send me an email: [email protected]
Transcript provided by We Edit Podcasts. Software Engineering Daily listeners can go to weeditpodcasts.com/sed to get 20% off the first two months of audio editing and transcription services. Thanks to We Edit Podcasts for partnering with SE Daily. Please click here to view this show’s transcript.
Ethereum is a decentralized transaction-based state machine. Ethereum was designed to make smart contracts more usable for developers. Smart contracts are decentralized programs that usually allow for some a transaction between the owner of the contract and anyone who would want to purchase something from the contract owner.
For example, I could set up a smart contract where a listener sends my smart contract some ether and I send the listener a podcast episode automatically. Smart contracts can also interact with each other, to network together complex transactions. In the same way that web development has been made easier by PaaS and SaaS, smart contracts will make building financial systems simple.
Preethi Kasireddy is a blockchain developer who writes extensively about cryptocurrencies. She joins the show to describe how the Ethereum platform works, including the steps involved in a smart contract transaction. This episode covers some advanced topics of Ethereum, and if you are out of your comfort zone, don’t worry–you aren’t alone.
The mobile apps are open sourced at github.com/softwareengineeringdaily. If you are looking for an open source project to hack on, we would love to get your help! We are building a new way to consume software engineering content. We have the Android app, the iOS app, a recommendation system, and a web frontend–and more projects are coming soon. If you have ideas for how software engineering media content should be consumed, or if you are interested in contributing code, check out github.com/softwareengineeringdaily, or join our Slack channel (there’s a link on our website)–or send me an email: [email protected]
Transcript provided by We Edit Podcasts. Software Engineering Daily listeners can go to weeditpodcasts.com/sed to get 20% off the first two months of audio editing and transcription services. Thanks to We Edit Podcasts for partnering with SE Daily. Please click here to view this show’s transcript.
Visa processes 1,600 transactions per second. PayPal processes 193 transactions per second. Bitcoin processes only 3-4 transactions per second. In order to fulfill the dreams of financial programming–in order to get decentralized, peer-to-peer micropayments–Bitcoin needs a much higher transaction throughput. Bitcoin’s scalability issues have led to debates within the community and changes in the software.
In this episode, Jordan Clifford gives an overview of some of the scaling limitations of Bitcoin, and discusses SegWit, a change to the Bitcoin protocol that improves scalability. Jordan was previously on the show to discuss the basics of Ethereum and Bitcoin. This episode covers some advanced topics of Bitcoin, and if you are out of your comfort zone, don’t worry–you aren’t alone.
Stay tuned at the end of the episode for Jeff Meyerson’s tip about assessing cultural fit at a company: brought to you by Indeed Prime.
The mobile apps are open sourced at github.com/softwareengineeringdaily. If you are looking for an open source project to hack on, we would love to get your help! We are building a new way to consume software engineering content. We have the Android app, the iOS app, a recommendation system, and a web frontend–and more projects are coming soon. If you have ideas for how software engineering media content should be consumed, or if you are interested in contributing code, check out github.com/softwareengineeringdaily, or join our Slack channel (there’s a link on our website)–or send me an email: [email protected]
Transcript provided by We Edit Podcasts. Software Engineering Daily listeners can go to weeditpodcasts.com/sed to get 20% off the first two months of audio editing and transcription services. Thanks to We Edit Podcasts for partnering with SE Daily. Please click here to view this show’s transcript.
Advertising fraud continues to plague the Internet. We do not know the scope and scale of that fraud. How many ads on the Internet are viewed by bots? Estimations range from 2% to 99%.
Advertisers are slowly becoming more educated about fraud, thanks in part to Dr. Augustine Fou. Dr. Fou is a full-time advertising fraud researcher. He looks at data sets of billions of ad impressions to figure out how fraud works and help victims of ad fraud make their case.
Last year, Dr. Fou came on the show to give an overview of his perspective on the world of ad fraud. Today, we dive into the importance of Twitter in ad fraud schemes. We also talk about the severity of fraud on mobile apps. If you downloaded a flashlight app, or an alarm clock app, or a keyboard, that app could be displaying hidden ads that never actually show up.
Stay tuned at the end of the episode for Jeff Meyerson’s tip about getting into a new job: brought to you by Indeed Prime.
The mobile apps are open sourced at github.com/softwareengineeringdaily. If you are looking for an open source project to hack on, we would love to get your help! The Software Engineering Daily open source community is building a new way to consume software engineering content. We have the Android app, the iOS app, a recommendation system, and a web frontend. If you are interested in contributing, check out github.com/softwareengineeringdaily–or send me an email: [email protected]
The applications that demand deep learning range from self-driving cars to healthcare, but the way that models are developed and trained is similar. A model is trained in the cloud and deployed to a device. The device engages with the real world, gathering more data. That data is sent back to the cloud, where it can improve the model.
From the processor level to the software frameworks at the top of the stack, the impact of deep learning is so significant that it is driving changes everywhere. At the hardware level, new chips are being designed to perform the matrix calculations at the heart of a neural net. At the software level, programmers are empowered by new frameworks like Neon and TensorFlow. In between the programmer and the hardware, middleware can transform software models into representations that can execute with better performance.
Milena Marinova is the senior director of AI solutions at the Intel AI products group, and joins the show today to talk about modern applications of machine learning and how those translate into Intel’s business strategy around hardware, software, and cloud. Full disclosure: Intel is a sponsor of Software Engineering Daily.
Question of the Week: What is your favorite continuous delivery or continuous integration tool? Email [email protected] and a winner will be chosen at random to receive a Software Engineering Daily hoodie.
Data Skeptic podcast: Generative Adversarial Networks
Machines understand the world through mathematical representations. In order to train a machine learning model, we need to describe everything in terms of numbers. Images, words, and sounds are too abstract for a computer. But a series of numbers is a representation that we can all agree on, whether we are a computer or a human.
In recent shows, we have explored how to train machine learning models to understand images and video. Today, we explore words. You might be thinking–”isn’t a word easy to understand? Can’t you just take the dictionary definition?” A dictionary definition does not capture the richness of a word. Dictionaries do not give you a way to measure similarity between one word and all other words in a given language.
Word2vec is a system for defining words in terms of the words that appear close to that word. For example, the sentence “Howard is sitting in a Starbucks cafe drinking a cup of coffee” gives an obvious indication that the words “cafe,” “cup,” and “coffee” are all related. With enough sentences like that, we can start to understand the entire language.
Adrian Colyer is a venture capitalist with Accel, and blogs about technical topics such as word2vec. We talked about word2vec specifically, and the deep learning space more generally. We also explored how the rapidly improving tools around deep learning are changing the venture investment landscape.
If you like this episode, we have done many other shows about machine learning with guests like Matt Zeiler, the founder of Clarif.ai and Francois Chollet, the creator of Keras. You can check out our back catalog by downloading the Software Engineering Daily app for iOS, where you can listen to all of our old episodes, and easily discover new topics that might interest you. You can upvote the episodes you like and get recommendations based on your listening history. With 600 episodes, it is hard to find the episodes that appeal to you, and we hope the app helps with that.
Question of the Week: What is your favorite continuous delivery or continuous integration tool? Email [email protected] and a winner will be chosen at random to receive a Software Engineering Daily hoodie.
The Decentralized Autonomous Organization (DAO) was a digital form of venture capital. It was an ambitious idea–to provide a new decentralized business model for organizing corporations on top of the Ethereum blockchain. Few people in the crypto community were opposed to this premise–but the timeline was short, the code requirements were tremendous, and in retrospect, a vulnerability was inevitable.
The DAO launched in May 2016, setting the record for the largest crowdfunding event in history. The following month, the DAO was hacked, millions of dollars of Ether were stolen, and the reverberations of the event were a referendum on how the Ethereum community governs itself.
Matt Leising is a reporter for Bloomberg who has chronicled the DAO in his article The Ether Thief. He continues to follow cryptocurrencies closely, as the Internet of money fractals increasingly into the public consciousness.
If you like this episode, we have done many other shows about cryptocurrencies and their implications. You can check out our back catalog by downloading the Software Engineering Daily app for iOS, where you can listen to all of our old episodes, and easily discover new topics that might interest you. You can upvote the episodes you like and get recommendations based on your listening history. With 600 episodes, it is hard to find the episodes that appeal to you, and we hope the app helps with that.
Errata: Coinbase now supports Bitcoin Cash.
Religious extremists use technology to recruit vulnerable individuals to a violent cause. Google is developing ways to combat this extremism through its platforms, namely YouTube. When a user looks for inflammatory religious or supremacist content, YouTube’s “Redirect Method” instead sends those users toward anti-terrorist videos.
Google’s fight against extremism compelled writer Lochlan Bloom to write an article called “The Coming Battle: AI, Extremism, and the New War of Ideas.” Lochlan joins the show to discuss the societal implications of giant internet providers controlling our information flow.
Lochlan is a science fiction writer, most recently of The Wave, a book that mixes existentialism with quantum physics. Our reality is defined by what we observe, and this theme courses through our conversation–from religion to Twitter to artificial intelligence.
The Coming Battle: AI, Extremism, and the New War of Ideas
From the publisher's feed