Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Feature] Add support for Timestamp type for Substream Powered Subgraphs #5637

Closed
1 of 2 tasks
Dirt-Nasty opened this issue Sep 9, 2024 · 2 comments
Closed
1 of 2 tasks
Labels
enhancement New feature or request

Comments

@Dirt-Nasty
Copy link

Description

Enhancement Request: Timeseries Aggregations Support for Substreams

Doing timeseries within a substream can be quite cumbersome, so taking advantage of aggregations would greatly improve both development and performance.

If we are sending over entity changes from a substream, this functionality is currently not possible.

To use Timeseries aggregations, there are a few requirements:

  • The id field must be of type Int8
  • Must have a timestamp field of type Timestamp

The id field seems to be automatically set when the entity type is defined with timeseries: true, so that doesn't seem to be an issue. However, a field type of Timestamp is currently not supported, even after making adjustments to the substreams entity changes library.

I experimented with adding Timestamp to this library, and you can see those changes here: Substreams Sink Entity Changes - Add Timestamp.

With this change, you can now output a type of timeseries from the substream to graph-node:

{
  "name": "timestamp",
  "newValue": {
    "timestamp": "1680880739"
  }
}

However, this still doesn't seem to be enough. Here is the error produced when sending this from my substream:

Error: failed to process trigger: Failed to process substreams block: Entity MessageSent[0]: missing value for non-nullable field `timestamp`

It looks like graph-node doesn’t know how to decode this field type in the protobuf message, so it comes in as a blank field.

I believe graph-node needs to be updated in a similar way to the changes I made in the substreams entity changes library. This might be required somewhere around here:
graph-node substreams codec.proto.

I’d love to hear any feedback on this!

Are you aware of any blockers that must be resolved before implementing this feature? If so, which? Link to any relevant GitHub issues.

No response

Some information to help us out

  • Tick this box if you plan on implementing this feature yourself.
  • I have searched the issue tracker to make sure this issue is not a duplicate.
@Dirt-Nasty Dirt-Nasty added the enhancement New feature or request label Sep 9, 2024
@DenisCarriere
Copy link

More context on Timestamp with Substreams Powered Subgraphs
pinax-network/subgraph-antelope-transactions#2

❌ Errors

Attempted different variances of BigInt/String and none of them are functional.

the value 1528545390 for field timestamp must have type Timestamp! but has type String
the value 2018-06-09T11:56:30Z for field timestamp must have type Timestamp! but has type String
the value 1528545390000000 for field timestamp must have type Timestamp! but has type BigInt
the value 1528545390000000 for field timestamp must have type Timestamp! but has type String

@Dirt-Nasty
Copy link
Author

Hey @DenisCarriere the changes in my fork of substream entity changes https://github.com/Dirt-Nasty/substreams-sink-entity-changes/tree/add_timestamp remove these errors which I thought would have worked. It modifies the types on the entity changes to include Timestamp but on the graph-node (receiving side) it doesn't know how to decode that type. So technically modification is needed on both the substream library and graph-node and I was able to complete the substream side but I'm unable to experiment with graph-node as it would probably take me a good while to get graph-node up and running locally with a modified version.

YaroShkvorets added a commit to YaroShkvorets/graph-node that referenced this issue Sep 10, 2024
@lutter lutter closed this as completed in 51182ca Sep 10, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

2 participants