Dynamically Linked Functions Library in CPP #23634
Conversation
|
|
soumiiow
force-pushed
the
dyllib-files-testing-macos
branch
from
c53ec85
to
ce1a3ee
Compare
There was a problem hiding this comment.
Thanks for the documentation! I suggested some changes to the README to simplify some of the wording and improve readability.
If my suggestions change a sentence's meaning in a way that makes it incorrect, let me know and we can find a better phrasing together.
| @@ -0,0 +1,23 @@ | |||
| # Prestissimo: Dynamically Loading Function Library | |||
|
|
|||
| This library adds the ability to load User Defined Functions (UDFs) without having to fork and build Prestissimo through the use of shared libraries. The Prestissimo worker is to access said code. The dynamic functions are to be loaded upon running an instance of the presto server. In the presto server instance, it will search for any .so or .dylib files and load them using this library. | |||
There was a problem hiding this comment.
| This library adds the ability to load User Defined Functions (UDFs) without having to fork and build Prestissimo through the use of shared libraries. The Prestissimo worker is to access said code. The dynamic functions are to be loaded upon running an instance of the presto server. In the presto server instance, it will search for any .so or .dylib files and load them using this library. | |
| This library adds the ability to load User Defined Functions (UDFs) without having to fork and build Prestissimo, through the use of shared libraries that a Prestissimo worker can access. The dynamic functions are loaded on launch of the Presto server. The Presto server searches for any .so or .dylib files and loads them using this library. |
Suggested changes for readability.
|
|
||
| ## Getting started | ||
|
|
||
| The Process can be roughly broken down into the steps below: |
There was a problem hiding this comment.
| The Process can be roughly broken down into the steps below: |
|
|
||
| The Process can be roughly broken down into the steps below: | ||
| ### 1. Create a cpp file for your dynamic library | ||
| For dynamically loaded function registration, the format followed is mirrored of that of built-in function registration with some noted differences. For instance, the below example function uses the extern "C" keyword to protect against name mangling. Additionally, a registry() function call is also necessary here. |
There was a problem hiding this comment.
| For dynamically loaded function registration, the format followed is mirrored of that of built-in function registration with some noted differences. For instance, the below example function uses the extern "C" keyword to protect against name mangling. Additionally, a registry() function call is also necessary here. | |
| For dynamically loaded function registration, the format is the same as for built-in function registration with some differences. For instance, the below example function uses the extern "C" keyword to protect against name mangling. A registry() function call is also necessary. |
There was a problem hiding this comment.
@steveburnett I was thinking here, for clarity, I could copy paste the test function in MyDynamicTestFunction.cpp so it clearly demonstrates the format of the function. do you think that would be a useful addition or repetitive as the reader can find the test function for themselves too
There was a problem hiding this comment.
I think it might be repetitive, but copy-pasting the test function in MyDynamicTestFunction.cpp here could also be valuable. It would depend how you write it. I could see it useful if you want to emphasize certain details, and if you keep the copied portion to the minimum number of lines needed so it doesn't overwhelm your steps here in the README.
I'd suggest that you try it and decide for yourself if you think it works or not.
If the part you need to copypaste is large, maybe discuss the test function here and include a link to the file MyDynamicTestFunction.cpp for readers.
| target_link_libraries(name_of_dynamic_fn PRIVATE xsimd fmt::fmt velox_expression) | ||
| ``` | ||
|
|
||
| ### 3. In the prestissimo worker's config.properties file, set the plugin.dir property |
There was a problem hiding this comment.
| ### 3. In the prestissimo worker's config.properties file, set the plugin.dir property | |
| ### 3. In the Prestissimo worker's config.properties file, set the plugin.dir property |
| ``` | ||
| plugin.dir="User\Test\Path\plugin" | ||
| ``` | ||
| ### 4. Upon start of the Presto Server, this directory will be scanned and all shared libraries will be attempted to be dynamically loaded when the worker or the sidecar process starts. |
There was a problem hiding this comment.
| ### 4. Upon start of the Presto Server, this directory will be scanned and all shared libraries will be attempted to be dynamically loaded when the worker or the sidecar process starts. | |
| ### 4. When the worker or the sidecar process starts, it scans this directory and attempts to dynamically load all shared libraries. |
Description
RFC for the changes prestodb/rfcs#24
These changes will allow users to dynamically load functions in prestissimo using cpp
Motivation and Context
Having these changes will enable users to register custom functions dynamically without requiring a fork of Prestissimo.
Impact
Test Plan
currently the unit test and container tests are still being debugged. and are being actively worked on
Contributor checklist
Release Notes
Please follow release notes guidelines and fill in the release notes below.
If release note is NOT required, use: