Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

enable crossdb column metadata with ra3 nodes #738

Open
wants to merge 2 commits into
base: main
Choose a base branch
from

Conversation

dataders
Copy link
Contributor

@dataders dataders commented Mar 25, 2024

resolves #736

Problem

R3-node users cannot fetch column metadata for tables & views outside of the current database.

Solution

Similar to @jiezhen-chen's approach in #428: use svv_all_columns instead of information_schema."columns".

While we're here, I have a hunch that @aiguofer's dbt-labs/dbt-core#2754 and it's usage of pg_catalog.svv_external_columns can be dropped. Still to be verified.

Questions

  1. Does this fix things for RA3 node cross-db scenarios?
  2. Does it break things for non-RA3 scenarios?
  3. What is the performance impact of such a change?
  4. Is it correct to assume that svv_all_columns includes the results of svv_external_columns?
  5. To ameliorate 2 and 3, should this svv_all_columns only be used if the relation-to-be-fetched's database does not match the one to which dbt is currently connected? (similar: [CT-1122] [CT-1116] [Bug] adapter.get_relation for relations in other databases [RA3] #179 (comment))

Checklist

  • I have read the contributing guide and understand what's expected of me
  • I have run this code in development and it appears to resolve the stated issue
  • This PR includes tests, or tests are not required/relevant for this PR
  • This PR has no interface changes (e.g. macros, cli, logs, json artifacts, config files, adapter interface, etc) or this PR has already received feedback and approval from Product or DX

@VolkerSchiewe
Copy link

  1. Is it correct to assume that svv_all_columns includes the results of svv_external_columns?

I can confirm that svv_all_columns contains column information of external schemas as well.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[Bug] adapter.get_columns_in_relation does not work cross database
2 participants