Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

graph, store: Avoid using to_jsonb when looking up a single entity #5372

Open
wants to merge 7 commits into
base: master
Choose a base branch
from

Conversation

lutter
Copy link
Collaborator

@lutter lutter commented Apr 23, 2024

Our queries all ultimately get their data by doing something like select to_jsonb(c.*) from ( ... complicated query ... ) c because when these queries were written it was far from obvious how to generate queries with Diesel that select columns whose number and types aren't known at compile time.

The call to to_jsonb forces Postgres to encode all data as JSON, which graph-node then has to deserialize which is pretty wasteful both in terms of memory and CPU.

This commit is focused on the groundwork for getting rid of these JSON conversions and querying data in a more compact and native form with fewer conversions. It only uses it in the fairly simple case of Layout.find, but future changes will expand that use

@mangas
Copy link
Contributor

mangas commented May 28, 2024

@lutter is this ready for review?

@lutter lutter marked this pull request as ready for review June 6, 2024 00:20
@lutter
Copy link
Collaborator Author

lutter commented Jun 6, 2024

It is ready for review in the sense that it does what it claims to do, but it introduces a bunch of machinery whose full value we'd only reap if we used it more widely. But maybe it's enough to do this as a first step in that direction.

@fordN fordN requested a review from mangas June 6, 2024 15:51
Copy link

This pull request hasn't had any activity for the last 90 days. If there's no more activity over the course of the next 14 days, it will automatically be closed.

@github-actions github-actions bot added the Stale label Sep 13, 2024
@fordN fordN requested review from zorancv and removed request for mangas September 16, 2024 15:44
Our queries all ultimately get their data by doing something like `select
to_jsonb(c.*) from ( ... complicated query ... ) c` because when these
queries were written it was far from obvious how to generate queries with
Diesel that select columns whose number and types aren't known at compile
time.

The call to `to_jsonb` forces Postgres to encode all data as JSON, which
graph-node then has to deserialize which is pretty wasteful both in terms
of memory and CPU.

This commit is focused on the groundwork for getting rid of these JSON
conversions and querying data in a more compact and native form with fewer
conversions. It only uses it in the fairly simple case of `Layout.find`,
but future changes will expand that use
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants