Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

docs: ADR 056 ORM query layer #11822

Closed
wants to merge 1 commit into from
Closed

docs: ADR 056 ORM query layer #11822

wants to merge 1 commit into from

Conversation

aaronc
Copy link
Member

@aaronc aaronc commented Apr 28, 2022

Description

Ref #11774


Author Checklist

All items are required. Please add a note to the item if the item is not applicable and
please add links to any relevant follow up issues.

I have...

  • included the correct type prefix in the PR title
  • added ! to the type prefix if API or client breaking change
  • targeted the correct branch (see PR Targeting)
  • provided a link to the relevant issue or specification
  • followed the guidelines for building modules
  • included the necessary unit and integration tests
  • added a changelog entry to CHANGELOG.md
  • included comments for documenting Go code
  • updated the relevant documentation or specification
  • reviewed "Files changed" and left comments if necessary
  • confirmed all CI checks have passed

Reviewers Checklist

All items are required. Please add a note if the item is not applicable and please add
your handle next to the items reviewed if you only reviewed selected items.

I have...

  • confirmed the correct type prefix in the PR title
  • confirmed ! in the type prefix if API or client breaking change
  • confirmed all author checklist items have been addressed
  • reviewed state machine logic
  • reviewed API design and naming
  • reviewed documentation is accurate
  • reviewed tests and test coverage
  • manually tested (if applicable)

@github-actions github-actions bot added the T: ADR An issue or PR relating to an architectural decision record label Apr 28, 2022
Comment on lines +57 to +60
The built-in query planner which targets ORM state directly should be as simple as possible without being 100%
naive. The proposed approach is to use a simple rule-based query planner
that chooses a single index to perform queries on based on a few heuristics
without comparing alternate paths:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why not just SQL queries directly? (Assuming we can ensure timeouts on request, and that its a read-only query within the APIs)

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That could be a possible frontend API or part of it but we'd still need to write a query planner to target the ORM data directly

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hrmm, maybe I'm misunderstanding what GORM's SQL Builder guarantees is available (https://gorm.io/docs/sql_builder.html) , I thought the scan lines looked like what would suffice.

I'll try to look through more of the GORM examples to better understand!

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This has nothing to do with GORM. It's about the SDK's ORM. Different thing

@aaronc
Copy link
Member Author

aaronc commented Jun 8, 2022

I'm closing this for now because I'd like to spend more time with the SDK team aligning on the best solution.

Just to give on an update on my current thinking, I'm leaning away from any sort of lightweight query planner in the ORM unless we ever needed to do this inside the state machine (which we don't for now).

Instead, I want to explore direct integration with database query engines. One thing we definitely can and should do is make it easy to index state in SQL. But, there are several database engines which allow support for targeting different data stores with their query engines. For instance in Postgres, there are foreign data wrappers. In Sqlite they're called virtual tables. DuckDB supports something similar. If we could for example expose ORM state as for instance a Postgres FDW, we could have sophisticated queries even against historical state without needing to actually index. The performance would be worse than if the data were indexed, but it's a good option for quick out of the box queries and historical queries.

Anyway, closing for now with the intention to revisit when the SDK team has more time to take this up.

@aaronc aaronc closed this Jun 8, 2022
@tac0turtle tac0turtle deleted the aaronc/orm-grpc-proto2 branch February 13, 2023 14:50
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
T: ADR An issue or PR relating to an architectural decision record
Projects
Archived in project
Development

Successfully merging this pull request may close these issues.

3 participants