Generative AI initiatives require new data pipelines that prepare text files for querying by language models. Data engineers, scientists, and other stakeholders collaborate to design and implement these pipelines, which span text sources, tokens, vectors, vector databases, and LMs.
Published at:
https://www.eckerson.com/articles/the-new-data-pipeline-for-generative-ai-where-and-how-it-works

The New Data Pipeline For Generative AI: Where And How It Works - Audio Blog

Listen to data and analytics leaders share the secrets of their success. Wayne Eckerson, long-time global thought leader interviews guests who run data and analytics programs at Fortune 2000 organizations around the world. Tune in to stay abreast of the latest technologies, techniques, and trends in our fast-paced industry.

Business

Investing

Careers

Management & Marketing

Business News

Shopping

Data engineers and data scientists must manage pipelines for unstructured data to ensure healthy inputs for language models.
Published at:
https://www.eckerson.com/articles/why-and-how-data-engineers-will-enable-the-next-phase-of-generative-ai

Why and How Data Engineers Will Enable the Next Phase of Generative AI - Audio Blog

Companies that adopt DataOps increase the odds of success by making GenAI data pipelines what they should be: modular, scalable, robust, flexible, and governed.
Published: https://www.eckerson.com/articles/dataops-for-generative-ai-data-pipelines-part-ii-must-have-characteristics

DataOps for Generative AI Data Pipelines, Part II: Must-Have Characteristics - Audio Blog

Most data leaders want to deliver data products, but few are doing it. 

Let's face it: most data teams today function as internal service bureaus that fulfill customer requests that arrive via ticketing systems, email, handwritten notes, or calls from colleagues looking for a favor. Most work double time to keep their request backlogs from ballooning from weeks to months. In this environment, few data leaders have time or capacity to switch from a project management approach to a product management one.

Even if data leaders had time, most wouldn't know how to make this transition. Most have no experience in product management, nor do they have a good idea of a data product. So asking data leaders to deliver data products is like asking them to build a rocket ship that can travel to the moon.

In this episode, Wayne Eckerson interviews Henrik Strandberg, a strong proponent of running data teams using product management principles. Henrik Strandberg is a seasoned data transformation leader who, for the past 25 years, has helped numerous organizations bridge gaps between business and technology. In stints at publishing and gaming companies, Henrik has developed a unique understanding of building and delivering data products at scale that delight customers.

Achieving Success with Data Products: An Interview with Henrik Strandberg

GenAI can help data engineers become more productive, and data engineering can help GenAI drive new levels of innovation.
Published at:
https://www.eckerson.com/articles/achieving-fusion-how-genai-and-data-engineering-help-one-another

Achieving Fusion: How GenAI and Data Engineering Help One Another - Audio Blog

Discover how master data management (MDM)  provides language models with high-quality enterprise data to improve their response accuracy.
Published at:
https://www.eckerson.com/articles/improving-genai-accuracy-with-master-data-management

The New Data Pipeline For Generative AI: Where And How It Works - Audio Blog

Download our free app to listen on your phone