Trino Community Broadcast

Trino's initial use case was around replacing the Apache Hive runtime. As data lakes grew into prominence, it became clear that having a faster query engine didn't solve all problems. The Hive model itself was a huge bottleneck and didn't provide features that companies needed akin to data warehouses and databases. Apache Hudi is a new table format created out of Uber that aims to address many of these issues and usher in a new generation of data lake.

Tune in as we speak to the Trino Hudi connector contributor Sagar Sumit and Grace Lu who uses Trino and Hudi at Robinhood to discuss the new Hudi connector and future plans!

- Intro Music: 0:00

- Intro: 0:32

- Releases: 14:43

- Concept of the episode: Intro to Hudi and the Hudi connector: 22:29

- Concept of the episode: Merge on read and copy on write tables: 28:28

- Concept of the episode: Hudi metadata table: 39:24

- Concept of the episode: Hudi data layout: 46:39

- Concept of the episode: Robinhood Trino and Hudi use cases: 51:12

- Concept of the episode: Current state and roadmap for the Hudi connector: 1:03:15

- Pull request of the episode: PR 14445: Fault-tolerant execution for PostgreSQL and MySQL connectors: 1:08:14

- Demo of the episode: Using the Hudi Connector: 1:13:34

Show Notes: https://trino.io/episodes/41.html

Show Page: https://trino.io/broadcast/

Join us for this next episode of the broadcast, where we bring back Ryan Blue, the creator of Iceberg, to discuss some of the latest happenings in the Iceberg community. We also discuss and demo a bunch of new features that have come out in the Trino Iceberg connector. We also have a new guest, Tabular Developer Advocate Sam Redai, shedding light on this incredible community as well!

Since the first episodes, Iceberg has finalized the v2 spec and added a lot of new features along the way. Likewise, we've improved Trino's writing capabilities around Iceberg. So much so that you can use Trino as the sole query engine atop Iceberg to support your data lake. We'll talk about all of this and more so don't miss it!

- Intro Music: 0:00

- Intro: 0:32

- Releases: 6:27

- Concept of the episode: What is Iceberg?: 11:27

- Concept of the episode: Why Iceberg over other formats?: 16:50

- Concept of the episode: Metadata catalogs: 35:40

- Concept of the episode: Branching, tagging, and auditing, oh my!: 43:54

- Concept of the episode: The Puffin format: 50:53

- Concept of the episode: Trino Iceberg connector updates : 1:01:38

- Pull request of the episode: PR 13111: Scale table writers per task based on throughput: 1:11:37

- Demo of the episode: DML operations on Iceberg using Trino: 1:15:31

Show Notes: https://trino.io/episodes/40.html

Show Page: https://trino.io/broadcast/

In this episode we sit down with engineers, Steve Morgan and Edward Morgan, to discuss how they use Trino at Raft. Raft provides consulting services and is particularly skilled at DevSecOps. One particular challenge they face is dealing with fragmented government infrastructure. In this episode, we dive in to learn how Trino enables Raft to supply government sector clients with a data fabric solution. Raft takes a special stance on using and contributing to open source solutions that run well on the cloud.

- Intro: 0:00

- Intro: 0:32

- Releases: 5:14

- Concept of the episode: Trino at Raft: 12:08

- Concept of the episode: Software factory: 13:21

- Concept of the episode: Standards and anatomy of a stack: 16:24

- Concept of the episode: Data Fabric at Raft: 18:58

- Concept of the episode: Security concerns around Trino: 22:01

- Concept of the episode: Iron bank container repository : 29:27

- Concept of the episode: Data Fabric user perspective : 36:13

- Concept of the episode: Challenges for adoption : 45:52

- Pull request of the episode: PR 13354: Add S3 Select pushdown for JSON files: 1:05:34

- Demo of the episode: Running Great Expectations on a Trino Data Lakehouse Tutorial: 1:08:01

- Question of the episode: How can I deploy Trino on Kubernetes without using Helm chart?: 1:13:55

Show Notes: https://trino.io/episodes/39.html

Show Page: https://trino.io/broadcast/

We'll be doing a more focused look at a specific feature that's being added to Trino: polymorphic table functions. We're excited to talk about what they do, where we are so far, where we're going, and how you can leverage them to make Trino better than ever!

Show Notes: https://trino.io/episodes/38.html

Show Page: https://trino.io/broadcast/

YouTube Video: https://www.youtube.com/watch?v=90e5WxhwNas

This episode covers will introduce the benefits of having the Trino community around the Trino project. What is the purpose of communities in tech projects? Would the product be successful without a community or anyone to maintain it?

We introduce some new faces that will be stewards in our journey to growing the adoption of our favorite query engine, what each of them does, and how their work impacts you as a community member! Most importantly, you can learn how to get involved and help us learn how to best navigate ideas, issues, or any other contribution you may have that helps make our favorite query engine the best-in-class!

- Intro song: 00:00

- Intro: 00:32

- Releases: 9:37

- Concept of the episode: How to strengthen the Trino community: 15:07

- Concept of the episode: Pull request process: 30:33

- Concept of the episode: Impact of community and developer experience: 33:07

- Concept of the episode: Community metrics for better decision making: 44:00

- Pull requests of the episode: PR 12259: Support updating Iceberg table partitioning: 1:09:42

- Demo of the episode: Iceberg table partition migrations: 1:16:00

- Question of the episode: Can I force a pushdown join into a connected data source?: 1:28:40

Show Notes: https://trino.io/episodes/37.html

Show Page: https://trino.io/broadcast/

As Trino preps to jump to Java 17, we discuss the latest features added Java 11 to Java 17, talk with Martin through a few of the potential uses of new features like the Vector API, language improvements, and G1GC speedups, and finally, we will dive into discussing some of the features that we'll be implementing in the upcoming months under a new project in Trino!

- Intro song: 00:00

- Intro: 00:36

- Releases: 8:17

- Question of the episode: Will Trino be making a vectorized C++ version of Trino workers?: 19:22

- Concept of the episode: Java 17 and rearchitecting Trino: 36:39

- Java 17 Updates: Performance: 40:10

- Java 17 Updates: Garbage collectors: 46:45

- Java 17 Updates: Java auto-vectorization: 1:06:22

- Java 17 Updates: Java Vector API: 1:12:08

- Java 17 Updates: Language features: 1:17:14

- Rearchitecting Trino: Update to Java 17: 1:27:19

- Rearchitecting Trino: Revamping Trino: 1:32:36

- Rearchitecting Trino: Project Hummingbird: 1:39:40

- Pull requests of the episode: PR 4649: Disable JIT byte code recompilation cutoffs in default jvm.config: 1:42:31

- Demo of the episode: FizzBuzz - SIMD style!: 1:49:18

Show Notes: https://trino.io/episodes/36.html

Show Page: https://trino.io/broadcast/

In our Trino Community Broadcast episode 35 we are catching up on recent releases 375, 376, 377, and 378.

We then talk about how Trino is packaged as tarball, rpm, and docker container, what some of the differences are, and how you can customize either of them. Beyond we also look for your feedback and input on usage of the different packages.

As a next step we chat about adopting Java 17 is standard for Trino, and then we get a demo of a new feature of the web UI.

- Intro song: 00:00

- Intro: 00:32

- Releases: 4:22

- Concept of the episode: Packaging Trino: 21:28

- Additional topic of the episode: Modernizing Trino with Java 17: 46:49

- Pull requests of the episode: Worker stats in the Web UI: 55:25

- Question of the episode: HDFS supported by Delta Lake connector?: 1:01:52

- Demo of the episode: Tarball installation and new Web UI feature: 1:05:58

Show Notes: https://trino.io/episodes/35.html

Show Page: https://trino.io/broadcast/

News from the Trino releases 372, 373, and 374, and an update on Project Tardigrade are the start. Then we dive into the details of the new Delta Lake connector contributed to Trino by Starburst.

- Intro song: 00:00

- Intro: 00:37

- Releases: 2:05

- Project Tardigrade update: 9:21

- Concept of the episode: A new connector for Delta Lake object storage. 18:37

- Pull requests of the episode: Add Delta Lake connector and documentation. 26:10

- Demo of the episode: Delta Lake connector in action. 29:14

- Question of the episode: How do I secure the connection from a Trino cluster to the data source? 54:00

Show Notes: https://trino.io/episodes/33.html

Show Page: https://trino.io/broadcast/

Goldman Sachs uses Trino to reduce last-mile ETL and provide a unified way of accessing data through federated joins. Making a variety of data sets from different sources available in one spot for our data science team was a tall order. Data must be quickly accessible to data consumers and systems like Trino must be reliable for users to trust this singular access point for their data.

Join us on this next episode as we discuss with engineers from Goldman Sachs on how they integrated Trino and achieved scaling and high availability.

- Intro Song: 00:00​

- Intro: 00:28

- News: 8:39

- Concept of the month: High Availability with Trino: 20:23

- PR of the month: PR 8956 Add support for external db for schema management in mongodb connector: 1:04:09

- Bonus PR of the month: PR 8202 Metadata for alias in elasticsearch connector only uses the first mapping: 1:15:15

- Demo of the month: Trino Fiddle: A tool for easy online testing and sharing of Trino SQL problems and their solutions: 1:32:08

- Question of the month: Does trino hive connector supports CarbonData?: 1:38:09

Show Notes: https://trino.io/episodes/33.html

Show Page: https://trino.io/broadcast/