Introducing Lambdora – A Tool for Time-Traveling Through Your Data

Lambdora is a service that integrates into your existing architecture through a change data capture (CDC) pipeline, for example using Debezium and Kafka. It consumes CDC events emitted by your systems and persistently records all data changes - creates, updates, and deletes - in a structured and queryable form.

Based on these events, Lambdora builds a historical timeline for each data entity. This timeline can be queried across specific time ranges, filtered by operation types, or narrowed down to changes affecting particular fields. The result is a clear and reliable view of how your data evolves over time and what exactly changed at each step.

Background

Lambdora started as a small side project over the Christmas holidays. The original goal was simple - I wanted to experiment with AI tools and models and see how far they could realistically take me when building something from scratch.

What surprised me most was how quickly the idea came together. With AI assisting throughout design discussions, implementation details, refactoring, and even edge-case reasoning, the project moved from a rough concept to a working prototype in just a few days.

As the holidays ended, Lambdora had already grown past a throwaway prototype. The core ideas felt solid, the use cases were real, and the tool was genuinely useful when pointed at actual CDC streams. That's when I decided to keep building - refining the model, adding features, and treating Lambdora more like a potential production tool rather than a one-off experiment.

Core Concepts

Lambdora organizes change events around a few key concepts:

• Source - the system from which change events originated, e.g. a MySQL database
• Collection - a logical grouping for change events within a source, e.g. a table
• Entity - a single data entity within a collection, e.g. a row in your database table
• Change Event - an immutable record indicating that a change occurred on an entity at a specific point in time
• Change History - a chronological sequence of change events that represents the lifecycle of an entity over time
• Entity State - the complete reconstructed state of an entity at a specific point in time

Lambdora supports five operation types: CREATE, UPDATE, DELETE, BACKFILL (for initial bootstrap), and SNAPSHOT (for compaction results).

Architecture

Lambdora is designed to integrate cleanly with existing CDC-based architectures:

1. Source Databases - MySQL, PostgreSQL, or any database that Debezium supports
2. Debezium Kafka Connectors - Monitor source databases and capture row-level changes
3. Kafka CDC Topics - Transport layer providing buffering, ordering, and replayability
4. Lambdora CDC Processor - Consumes events from Kafka and normalizes them
5. Lambdora Core - Persists change events, reconstructs entity state, and exposes APIs
6. Lambdora UI - Interactive interface for exploring entity history and changes

Key Features

Field-Level Change History

Lambdora captures changes at the field level, showing exactly what changed, from what value, and to what value for every event. This makes it easy to understand the intent and impact of each update.

Time-Travel Queries

You can query an entity's state at any point in time. Lambdora reconstructs the full entity state by replaying relevant change events, enabling historical inspection and point-in-time debugging.

Queryable Change Timelines

Change events can be queried by time range, operation type, or affected fields. This allows you to narrow down large histories to only the changes that matter for a specific investigation.

Automated and Manual Compaction

To keep storage and query performance under control, Lambdora supports event compaction - both automated (based on event age or count) and manual (triggered via the UI). Compaction replaces multiple historical events with a snapshot that preserves the effective state while reducing history size.

Entity-Level Statistics

For each entity, Lambdora exposes first and last change event timestamps, total number of events, and a breakdown by operation type. Statistics are available both as queryable statistics (based on retained events) and lifetime statistics (reflecting the full history, even after compaction).

Interactive UI

The Lambdora UI provides a visual way to inspect current and historical entity state, browse change timelines, view statistics, copy entity state as JSON, and trigger compaction actions.

Use Cases

Lambdora makes it easier to:

• Debug production issues by seeing what data changed and when
• Analyze change patterns to understand how your data evolves
• Audit data behavior for compliance and security
• Prove that a specific change occurred with complete historical records
• Restore accidentally modified records by reconstructing past states

Demos

To see Lambdora in action, check out the demo playlist which includes:

• Exploring the current entity state
• Browsing the change timeline of an entity
• Compaction in action
• Exploring entity statistics

Next Steps

Lambdora is still in its early stages, but I'm actively working toward making it easier for others to try it out in real environments. A key focus right now is preparing production-ready Docker images along with deployment templates that simplify running Lambdora locally or in managed infrastructure.

If this post has sparked your interest in Lambdora or you'd like to explore it further, request a demo to see it in action.