Home

Aesop is a keen observer of changes that can also relay change events reliably to interested parties. Provides useful infrastructure for building Eventually Consistent data sources and systems. Aesop is primarily used for reliable data change propagation between a source and number of consumers that optionally use different data stores. It also supports snapshot based change detection. For consumers, it provides a unified interface for change data consumption that is independent of how the data change is sourced. A couple of example uses of Aesop is as follows:

• Inventory updates on a warehousing system reflecting on product pages of an eCommerce portal.
• Applying data changes from an online transactional data store to an archive data store and in the process transforming data from a normalized schema to a denormalized one.

Aesop uses the log mining approach of detecting data changes as described by the LinkedIn Databus project. It also uses the infrastructure components of Databus, mostly for serving change events. The concept of Event Producer, Relay, Event Buffer, Bootstrap, Event Consumer and System Change Number(SCN) are quite appealing and used mostly as-is in Aesop.

Aesop supports change event detection from the following data stores:

• MySQL
• HBase
• Any data store with a high throughput iterator service API