ELT is a process for copying data from a source system into a target system. It stands for “Extract, Load, Transform” and starts with extracting a copy of data from the source location. It’s loaded into the target system like a data warehouse, and then it’s ready to be transformed into a usable format for

In this episode we discuss plug and play auth, password management, and crypto with Sean Li, co-founder and CEO of Magic. This interview was also recorded as a video podcast. Check out the video on the Software Daily YouTube channel. Sponsorship inquiries: [email protected]

For some data problems, you may be more concerned with the state of data at a particular point. A ticket is booked, or it’s not. How many poetry submissions were made to the contest? This is relational data. For other problems, you’re concerned with the change in data over time. Solar energy consumption, for example,

The cloud has delivered amazing benefits like on-demand infrastructure that’s easy to use, pay-as-you-go subscription plans, and effortless scaling of applications. This flexibility minimizes the growing pains for businesses and explains why today’s startups and established companies are both building apps on the cloud.  However, the costs of using the cloud stack up once companies

Uber is one of many examples we’ve discussed on this show that has changed the world with big data analysis. With over 8 million users, 1 billion Uber trips and people driving for Uber in over 400 cities and 66 countries, Uber has redefined an entire industry in a very short time frame. It’s difficult

The quantity and quality of a company’s data can mean the difference between a major success or major failure. Companies like Google have used big data from its earliest days to steer their product suite in the direction consumers need. Other companies, like Apple, didn’t always use big data analytics to drive product design, but

In this episode we discuss software investing, business, and the future with David Rosenthal, co-host of the Acquired podcast. This interview was also recorded as our very first video podcast. Check out the video on the Software Daily YouTube channel. Sponsorship inquiries: [email protected]

Complete information games are games where every player has information about the game sequence, strategies, and payoffs throughout gameplay. Playing chess, for example, relies on knowing the location of every piece everywhere on the board. In an incomplete information game like Minecraft, you continually gain new information during gameplay. Until very recently, incomplete information was

Proof of Work cryptocurrency mining, as used on the Ethereum and Bitcoin blockchains, requires huge amounts of energy to validate transactions and generate new tokens. The alternative, Proof of Stake, needs large deposits of assets to be staked up front in order to work. While both consensus protocols have their own drawbacks, they are the

The company StreamSets is enabling DataOps practices in today’s enterprises. StreamSets is a data engineering platform designed to help engineers design, deploy, and operate smart data pipelines. StreamSets Data Collector is a codeless solution for designing pipelines, triggering CDC operations, and monitoring data in flight. StreamSets Transformer uses Apache Spark to generate insights about your