Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Introduce Injection to add config values from PerTableConfig to TargetClients #306

Open
lmccay opened this issue Jan 7, 2024 · 0 comments

Comments

@lmccay
Copy link
Contributor

lmccay commented Jan 7, 2024

Since #304 split the TargetClient creation and initialization into separate steps with a create and init() method, we not have an opportunity to remove the direct dependency on PerTableConfig from the api module and the init(). Even though #304 added a new interface to the api module for PerTableConfig being added to the init() signature, it has proven difficult to separate table format specifics from the interface. A separate issue has been filed in #297 to help resolve that and improve extensibility.

This issue represents a change that will allow TargetClients to simply implement setters for each of the getters in PerTableConfig that it requires and they will magically be set prior to calling the init(). The init() will then be able to rely on them having already been set and use them as needed to complete initialization.

It will remove the need for the interface layer to be aware of target format config specifics and the need to pass PerTableConfig to the init().

Open Questions:

  1. Should we move each of the targetClients that uses the Hadoop Configuration class to have a setter and remove that from the init() arguments as well?
  2. Should we revisit the public factory method to createTargetClientForName() and only rely on PerTableConfig and the builtin injection or should we keep the flexibility to manually do the setting of those params?
  3. Should we keep the PerTableConfig interface that was added to help determine all the getters for which we will attempt to use setters on each targetClient or should we remove it and add more logic in order to filter out the setters from the getters scraped from the PerTableConifgImpl?
  4. Should we remove the HudiSourceConfig and CatalogConfig interfaces that were previously added so that the implementations wouldn't be pulled into the api module by the PerTableConfig interface?
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant