Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

renamed checkpoint by batch samples to by event count #15977

Merged
merged 2 commits into from
Jan 5, 2021
Merged

renamed checkpoint by batch samples to by event count #15977

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
868109d
renamed checkpoint by batch samples to by event count
swathipil Jan 4, 2021
3638523
change link in EH sample readme for event count sample
swathipil Jan 5, 2021
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion sdk/eventhub/azure-eventhub/samples/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -40,7 +40,7 @@ Both [sync version](https://github.com/Azure/azure-sdk-for-python/tree/master/sd
- [recv_with_checkpoint_store.py](https://github.com/Azure/azure-sdk-for-python/tree/master/sdk/eventhub/azure-eventhub/samples/sync_samples/recv_with_checkpoint_store.py) ([async version](https://github.com/Azure/azure-sdk-for-python/tree/master/sdk/eventhub/azure-eventhub/samples/async_samples/recv_with_checkpoint_store_async.py)) - Examples to receive events and do checkpoint using blob checkpoint store:
- Receive events and do checkpoint using blob checkpoint store

- [recv_with_checkpoint_by_batch.py](https://github.com/Azure/azure-sdk-for-python/tree/master/sdk/eventhub/azure-eventhub/samples/sync_samples/recv_with_checkpoint_by_batch.py) ([async version](https://github.com/Azure/azure-sdk-for-python/tree/master/sdk/eventhub/azure-eventhub/samples/async_samples/recv_with_checkpoint_by_batch_async.py)) - Examples to receive events and do checkpoint by batch using blob checkpoint store:
- [recv_with_checkpoint_by_event_count.py](https://github.com/Azure/azure-sdk-for-python/tree/master/sdk/eventhub/azure-eventhub/samples/sync_samples/recv_with_checkpoint_by_event_count.py) ([async version](https://github.com/Azure/azure-sdk-for-python/tree/master/sdk/eventhub/azure-eventhub/samples/async_samples/recv_with_checkpoint_by_event_count_async.py)) - Examples to receive events and do checkpoint by event count using blob checkpoint store:
- Receive events and do checkpoint every fixed amount of events (e.g. checkpoint every 20 events) using blob checkpoint store

- [receive_batch_with_checkpoint.py](https://github.com/Azure/azure-sdk-for-python/tree/master/sdk/eventhub/azure-eventhub/samples/sync_samples/receive_batch_with_checkpoint.py) ([async_version](https://github.com/Azure/azure-sdk-for-python/tree/master/sdk/eventhub/azure-eventhub/samples/async_samples/receive_batch_with_checkpoint_async.py)) - Examples to receive events in batches and do checkpoint by the batch:
Expand Down
7 changes: 4 additions & 3 deletions ...es/recv_with_checkpoint_by_batch_async.py → ...v_with_checkpoint_by_event_count_async.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,8 @@
# --------------------------------------------------------------------------------------------

"""
An example to show receiving events from an Event Hub with checkpoint store doing checkpoint by batch asynchronously.
An example to show receiving events from an Event Hub with checkpoint store doing checkpoint
by every fixed event count asynchronously.
In the `receive` method of `EventHubConsumerClient`:
If no partition id is specified, the checkpoint_store are used for load-balance and checkpoint.
If partition id is specified, the checkpoint_store can only be used for checkpoint.
Expand All @@ -23,15 +24,15 @@
BLOB_CONTAINER_NAME = "your-blob-container-name" # Please make sure the blob container resource exists.

partition_recv_cnt_since_last_checkpoint = defaultdict(int)
checkpoint_batch_event_cnt = 20
checkpoint_event_cnt = 20


async def on_event(partition_context, event):
# Put your code here.
p_id = partition_context.partition_id
print("Received event from partition: {}.".format(p_id))
partition_recv_cnt_since_last_checkpoint[p_id] += 1
if partition_recv_cnt_since_last_checkpoint[p_id] >= checkpoint_batch_event_cnt:
if partition_recv_cnt_since_last_checkpoint[p_id] >= checkpoint_event_cnt:
await partition_context.update_checkpoint(event)
partition_recv_cnt_since_last_checkpoint[p_id] = 0

Expand Down
7 changes: 4 additions & 3 deletions ..._samples/recv_with_checkpoint_by_batch.py → ...es/recv_with_checkpoint_by_event_count.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,8 @@
# --------------------------------------------------------------------------------------------

"""
An example to show receiving events from an Event Hub with checkpoint store doing checkpoint by batch.
An example to show receiving events from an Event Hub with checkpoint store doing checkpoint
by every fixed event count.
In the `receive` method of `EventHubConsumerClient`:
If no partition id is specified, the checkpoint_store are used for load-balance and checkpoint.
If partition id is specified, the checkpoint_store can only be used for checkpoint.
Expand All @@ -22,7 +23,7 @@
BLOB_CONTAINER_NAME = "your-blob-container-name" # Please make sure the blob container resource exists.

partition_recv_cnt_since_last_checkpoint = defaultdict(int)
checkpoint_batch_event_cnt = 20
checkpoint_event_cnt = 20


def on_event(partition_context, event):
Expand All @@ -31,7 +32,7 @@ def on_event(partition_context, event):
p_id = partition_context.partition_id
print("Received event from partition: {}".format(p_id))
partition_recv_cnt_since_last_checkpoint[p_id] += 1
if partition_recv_cnt_since_last_checkpoint[p_id] >= checkpoint_batch_event_cnt:
if partition_recv_cnt_since_last_checkpoint[p_id] >= checkpoint_event_cnt:
partition_context.update_checkpoint(event)
partition_recv_cnt_since_last_checkpoint[p_id] = 0

Expand Down