From Change Data Capture to AI: Feeding Your RAG with Debezium

From Change Data Capture to AI: Feeding Your RAG with Debezium

About Giovanni

From Change Data Capture to AI: Feeding Your RAG with Debezium

Agenda

1 Data Integration Patterns: Change Data Capture
2 Machine-to-Machine Semantics: Vector Databases
3 LLM Patterns: Retrieval-augmented generation
4 Debezium
5 Demo
6 Takeaways

From Change Data Capture to AI: Feeding Your RAG with Debezium

1 Data Integration Patterns: Change Data Capture

Change data capture (CDC) is a data integration pattern that captures changes made to data in a source system, such as inserts, updates, deletes and transmitts those changes to downstream systems.

Source CDC System Target Reads Changes Writes Changes Writes Changes Business Application Database Transaction LOG Data Pipeline Database

From Change Data Capture to AI: Feeding Your RAG with Debezium

1 Data Integration Patterns: Change Data Capture

Change data capture (CDC) is a data integration pattern that captures changes made to data in a source system, such as inserts, updates, deletes and transmitts those changes to downstream systems.

Source CDC System Target Reads Changes Writes Changes Writes Changes Business Application Database Transaction LOG Data Pipeline Database

From Change Data Capture to AI: Feeding Your RAG with Debezium

inserts, updates, deletes... are we in the '90?

The term Change Data Capture became popular during the rise of data warehousing. Early CDC implementations typically relied on:

  • Timestamp columns (last_updated)
  • Version numbers
  • Database triggers
  • Custom audit tables

Query-Based CDC Changed rows Next cycle Every N min Poll Result set No changes Target System Scheduler SELECT *WHERE updated > ? Source DB Diff

From Change Data Capture to AI: Feeding Your RAG with Debezium

Log Based Change Data Capture

Today the preferred solution is reading directly from the transaction log.
The benefits:

  • minimizes the performance impact on the source
  • transactional consistency
  • order & integrity of changes
From Change Data Capture to AI: Feeding Your RAG with Debezium

Ok, but still...I use mongoDB in production

While CDC is commonly associated with traditional databases, it is more broadly a pattern for capturing, recording, and propagating changes in data over time as that data evolves within a system.

Write-Ahead Log LSN 001 — INSERTusers (id=1, name='Alice') LSN 002 — UPDATEusers SET name='Bob' LSN 003 — INSERTorders (id=42, total=99.9) LSN 004 — DELETEusers WHERE id=1

From Change Data Capture to AI: Feeding Your RAG with Debezium

Data over time: the king of the castle

Information has been considered a valuable asset long before the emergence of computers, and its digital representation has been the subject of continuous research for decades.

CA CP AP Consistency Availability Partition Tolerance
From Change Data Capture to AI: Feeding Your RAG with Debezium

Data over time: the king of the castle

As data management needs have evolved, different types of data sources have emerged, each providing distinct physical and logical models for representing, storing, and querying information.

Text-based CSV JSON XML Binary Schema Avro Protobuf Columnar Parquet ORC Row-oriented id | name | city | age 1 | Alice | AMS | 30 2 | Bob | UTC | 25 Column-oriented id 1 2 name Alice Bob age 30 25 Choosing the right format impacts query speed, storage cost & schema evolution
From Change Data Capture to AI: Feeding Your RAG with Debezium

Moving data: the king's knight

In this context, Change Data Capture is the most efficient way to move data from one system to another while preserving formats, conventions, and the semantic meaning of the information, ensuring that changes are propagated consistently and accurately across heterogeneous systems.

Source CDC System Target Reads Changes Writes Changes Writes Changes Business Application Database Transaction LOG Data Pipeline Database

From Change Data Capture to AI: Feeding Your RAG with Debezium

1 Data Integration Patterns: Change Data Capture
2 Machine-to-Machine Semantics: Vector Databases
3 LLM Patterns: Retrieval-augmented generation
4 Debezium
5 Demo
6 Takeaways

From Change Data Capture to AI: Feeding Your RAG with Debezium

2 Machine-to-Machine Semantics: Vector Databases

Vector databases address a new challenge in the age of LLMs: enabling data to be organized and retrieved according to its semantic meaning for use by generative language models.

dim₁ dim₂ Animals cat dog horse bird fish Vehicles car truck bus bike Food pizza pasta bread rice query: "kitten" [0.23, 0.87, ...] 0.95 0.91 0.84
From Change Data Capture to AI: Feeding Your RAG with Debezium

Vector Databases: embeddings

The semantic meaning is captured by embeddings, a numerical vector representations of data allowing machines to compare and retrieve information based on cenceptual similarity rather than exact matches.

  • Vector Similarity Search
  • High-Dimensional Data Storage and Indexing
  • Metadata Filtering and Hybrid Search
From Change Data Capture to AI: Feeding Your RAG with Debezium

Embedding Function

Mathematical transformation that converts unstructured data into a dense vector of numbers in a high-dimensional space.

"king" "queen" "car" "bicycle" "pizza" f(x) embed Vector Space ℝⁿ king [0.91, 0.12, 0.85] queen [0.87, 0.21, 0.89] car [0.15, 0.88, 0.22] bicycle [0.18, 0.82, 0.31] pizza [0.45, 0.33, 0.72] similar → close similar → close
From Change Data Capture to AI: Feeding Your RAG with Debezium

Embedding Function: Embedding Service Providers

In most architectures, an application invokes an embedding model, either through a managed embedding service or a self-hosted deployment, to transform input data into vector representations.

From Change Data Capture to AI: Feeding Your RAG with Debezium

1 Data Integration Patterns: Change Data Capture
2 Machine-to-Machine Semantics: Vector Databases
3 LLM Patterns: Retrieval-augmented generation
4 Debezium
5 Demo
6 Takeaways

From Change Data Capture to AI: Feeding Your RAG with Debezium

3 LLM Patterns: Retrieval-augmented generation

LLM patterns are reusable architectural solutions that define how Large Language Models interact with:

  • Knowledge Sources
  • Tools
  • Users
From Change Data Capture to AI: Feeding Your RAG with Debezium

Retrieval-augmented generation

(RAG) is an architectural pattern that enhances Large Language Models by retrieving relevant external information and using it as context during response generation. The RAG pattern operates in two main steps:

  • Retrieval
  • Generation
From Change Data Capture to AI: Feeding Your RAG with Debezium

Simple RAG

Workflow:

  • Retrieval: the model searches a data source to find relevant information
  • Generation: based on the information, the model generates a response

Retrieval Generation question Question + information response Query Input Generated Output Data Source retriever Large Language Model

From Change Data Capture to AI: Feeding Your RAG with Debezium

The missing part: Ingestion

Since ingestion happens offline or continuously in the background and not during query execution, it is often treated as a separate indexing process rather than part of the RAG request flow.

Ingestion Retrieval Generation Load question Question + information response Query Input Generated Output Data Source ingestor Data Source retriever Large Language Model

From Change Data Capture to AI: Feeding Your RAG with Debezium

The missing part: Ingestion

The application ingests a set of documents, processes them, and stores them in a vector database.

Source Ingestion Pipeline Load Sink Vector Database Database Chunk / Split Normalize Embedding Generation

From Change Data Capture to AI: Feeding Your RAG with Debezium

RAG: Vector Databases

RAG leverages embeddings and vector databases to represent and retrieve machine-understandable semantic knowledge, which is then used by LLMs to generate context-aware responses.

Source Ingestion Pipeline Load Sink Vector Database Database Chunk / Split Normalize Embedding Generation

From Change Data Capture to AI: Feeding Your RAG with Debezium

RAG: Change data Capture

In modern AI and data architectures, CDC provides a continuous stream of updates that can be transformed into embeddings and indexed in vector databases, ensuring that semantic representations remain synchronized with the latest state of the underlying data.

Data Source Transformation Target Source Source Connector Sink Database Transaction LOG Chunk / Split Normalize Embedding Generation Vector Database

From Change Data Capture to AI: Feeding Your RAG with Debezium

CDC as ingestion pattern for RAG

Key benefits:

  • Low overhead on source systems
  • Near real-time knowledge updates
  • Consistent synchronization between operational data and vector indexes
From Change Data Capture to AI: Feeding Your RAG with Debezium

The different forms of RAG

you can find in the wild different architecture for RAG:

  • Simple RAG
  • Simple RAG with Memory
  • Branched RAG
  • cRAG
  • Agentic RAG
From Change Data Capture to AI: Feeding Your RAG with Debezium

1 Data Integration Patterns: Change Data Capture
2 Machine-to-Machine Semantics: Vector Databases
3 LLM Patterns: Retrieval-augmented generation
4 Debezium
5 Demo
6 Takeaways

From Change Data Capture to AI: Feeding Your RAG with Debezium

4 Debezium

Debezium is an open-source distributed platform for Change Data Capture (CDC). It streams real-time changes from databases into event-driven systems by monitoring database transaction logs.

  • The core component is the Debezium Source Connector.

Source Debezium Reads Changes Writes Changes Database Transaction LOG Source Connector

From Change Data Capture to AI: Feeding Your RAG with Debezium

Kafka Connect: a runtime for Debezium

Kafka Connect provides the runtime framework, handling connector lifecycle management, fault tolerance, offset storage, and scalability, while Debezium supplies source connectors that read database transaction logs.

Source System Kafka Connect Kafka Kafka Connect Target System Postgres MySQL MongoDB Debezium Source Connectors Kafka Topics Debezium Sink Connector Elasticsearch S3 Jdbc

From Change Data Capture to AI: Feeding Your RAG with Debezium

Debezium Server: kafka-less solution

Debezium Server is a lightweight, standalone runtime that enables CDC without requiring Apache Kafka.

Source System Debezium Server Target system Postgres MySQL MongoDB Source Connector Transforms Debezium Server Sink Pub/Sub Kinesis Event Hubs HTTP Endpoint Redis Streams

From Change Data Capture to AI: Feeding Your RAG with Debezium

The Ecosystem

Debezium offers a wide range of solutions to address CDC in different contexts.

Debezium Source Connectors Runtimes Debezium Engine Debezium Server Quarkus Runtime Extensions Debezium AI PyDebeziumAI SMT Deployment Debezium Operator Debezium Platform
From Change Data Capture to AI: Feeding Your RAG with Debezium

Debezium AI

Debezium AI is a set of tools for building data streaming pipelines that support the ingestion phase of RAG architectures using log-based CDC.

  • FieldToEmbedding Single message transformation (SMT)
  • PyDebeziumAI
From Change Data Capture to AI: Feeding Your RAG with Debezium

Debezium AI: SMT

Debezium offers a built-in feature to transform specified text fields into numerical embeddings vectors, and to add the resulting embeddings to the event record.

Source System Embedding Provider Debezium Server Target system data source embedding model Source Connector Transforms Debezium Server Sink Vectore Database

From Change Data Capture to AI: Feeding Your RAG with Debezium

Debezium AI: SMT

The embeddings transformation uses the langchain4j framework.

  • pros:
    • scalable and reliable thanks to Kafka Connect OR Debezium Server
    • compatible with Debezium Platform & Operator
  • cons:
    • limited support for embedding providers
From Change Data Capture to AI: Feeding Your RAG with Debezium

PyDebeziumAI

PyDebeziumAI integrates Debezium CDC streams with LangChain and LangGraph, bringing the capabilities of the Debezium Engine to Python applications via JPype.

PyDebeziumAI capture capture Sink Sink Sink Ingest Transform Sync Adapt Adapt Adapt PostgreSQL MySQL Vector Store Debezium Engine Ingestion Handlers Document Builder Sync Manager Milvus Chroma PGVector

From Change Data Capture to AI: Feeding Your RAG with Debezium

PyDebeziumAI

  • pros:
    • easy to use as python library
    • exposes retriever tools compatible with LangChain agents and LangGraph reactive nodes
  • cons:
    • unmanaged scalability and reliability
    • not compatible with Kafka, Debezium Server, Platform & Operator
From Change Data Capture to AI: Feeding Your RAG with Debezium

1 Data Integration Patterns: Change Data Capture
2 Machine-to-Machine Semantics: Vector Databases
3 LLM Patterns: Retrieval-augmented generation
4 Debezium
5 Demo
6 Takeaways

From Change Data Capture to AI: Feeding Your RAG with Debezium

5 Demo

From Change Data Capture to AI: Feeding Your RAG with Debezium

1 Data Integration Patterns: Change Data Capture
2 Machine-to-Machine Semantics: Vector Databases
3 LLM Patterns: Retrieval-augmented generation
4 Debezium
5 Demo
6 Takeaways

From Change Data Capture to AI: Feeding Your RAG with Debezium

6 Takeaways

From Change Data Capture to AI: Feeding Your RAG with Debezium

Takeaways

  • RAG often includes a preparation phase
  • CDC enables low-overhead pipelines with minimal impact on data sources
  • Debezium provides different ways for preparing data for vector databases
From Change Data Capture to AI: Feeding Your RAG with Debezium

Thanks

From Change Data Capture to AI: Feeding Your RAG with Debezium

Thank you for the JUG Amsterdam to make this happen, I know how much is difficult to organize meetups and events so again thank you. I promised to Geertjan that my talk is light and fast. So I'll go very quick on the next slide

my name is Giovanni software engineer with 10 years of career and the most important part is that I am Debezium Core contributor. A part from that, I am also contributor for the zig opentelementry sdk and Ticino Software Craft organizer. In the photo my natural form, drinking beers. Ok after this embarrassing moment, let's jump to the main topic of today