Skip to content

Latest commit

 

History

History
130 lines (116 loc) · 3.21 KB

dbk_libraries.md

File metadata and controls

130 lines (116 loc) · 3.21 KB
Raw

dbk_libraries

dbk_libraries(name, cluster_name, configure, dbfs, maven_info, maven_package_exclusion)

For example, if the BUILD file contains:

dbk_libraries(
    name = "lib"
    cluster_name = "test",
    configure = ":cfg",
    dbfs = ":src.cp",
    maven_info = {
        "https://repo.maven.apache.org/maven2/" : [
          "GroupId:ArtifactId:Version"
        ]
    },
    maven_package_exclusion = {
        "GroupId:ArtifactId:Version" : [
          "foo:bar"
        ]
    },
)
Attributes
name Name, required

A unique name for this rule.
cluster_name String, required

The name of the databricks cluster where operations can be performed.

cluster_name = "FOO"

Subject to "Make" variable substitution.
configure Label, required

Label of dbk_configure target.

Specify the databricks cluster settings.

configure = ":cfg"
dbfs Label, optional

File to add to the Databricks File Store (DBFS).

The file path in DBFS follows the pattern: dbfs:/FileStore/bazel/{target}
maven_info String list Dict, optional

The keys are the URLs of the maven repository and the list is coordinated by the maven in the form of GroupId:ArtifactId:Version

maven_info = { "https://repo.maven.apache.org/maven2/": ["GroupId:ArtifactId:Version"] }

maven_package_exclusion String list Dict, optional

The keys are the GroupId:ArtifactId:Version coordinated of the maven and the list with the dependencies to be excluded.

maven_info = { "GroupId:ArtifactId:Version": ["foo:bar"] }

Usage

The dbk_libraries rules expose a collection of actions. We will follow the :lib target from the example above.

Status

Users can list the state of the libraries in the cluster by running:

bazel run :lib

Install

Users can install libraries by running:

bazel run :lib.install

Uninstall

Users can uninstall libraries by running:

bazel run :lib.uninstall

It is notable that when uninstalling a library, a message to restart the cluster is displayed. This action must be performed manually by the Databricks administrator.