Arcaflow Elasticsearch

A plugin for loading data into an Elasticsearch instance.

Development

During the development of this plugin it is useful to start a local Elasticsearch via:

docker-compose -f docker-compose-dev.yml up -d

and stop it again via:

docker-compose -f docker-compose-dev.yml down -v

Testing

The tests of this plugin are split up into unit and integration tests located in

• ./tests/integration/
• ./tests/unit/

Unit Tests

Run all unit tests via:

# Run all unit tests
python -m unittest tests.unit.test_es_plugin

Integration Tests

Running all integration tests can be run either

• using a running a local Elasticsearch as described in Development and then execute the tests via

# Run all integration tests
python -m unittest tests.integration.test_es_plugin

• using the docker-compose-integration.yml and run

# the --abort-on-container-exit ensures a docker-compose down after the tests have run
docker-compose -f docker-compose-integration.yml up --abort-on-container-exit

Note: Make sure to docker-compose down and remove the volume after one run as there is currently no cleanup done.

Autogenerated Input/Output Documentation by Arcaflow-Docsgen Below

Elasticsearch (elasticsearch)

Load data into elasticsearch instance

Input

Type:	scope
Root object:	StoreDocumentRequest
Properties	data (map[string,any]) Name: data Description: Data to upload to your Elasticsearch index. Required: Yes Type: map[string,any] Key type Type: string Value type Type: any index (string) Name: index Description: Name of the Elasticsearch index that will receive the data. Required: Yes Type: string Minimum length: 1 password (string) Name: password Description: Name of the environment variable containing the password for the given user. Required: Yes Type: string url (string) Name: url Description: Name of the environment variable containing the URL for the Elasticsearch instance. Required: Yes Type: string username (string) Name: username Description: Name of the environment variable containing an authorized user for the given Elasticsearch instance. Required: Yes Type: string Minimum length: 1
Objects StoreDocumentRequest (object) Type: object Properties data (map[string,any]) Name: data Description: Data to upload to your Elasticsearch index. Required: Yes Type: map[string,any] Key type Type: string Value type Type: any index (string) Name: index Description: Name of the Elasticsearch index that will receive the data. Required: Yes Type: string Minimum length: 1 password (string) Name: password Description: Name of the environment variable containing the password for the given user. Required: Yes Type: string url (string) Name: url Description: Name of the environment variable containing the URL for the Elasticsearch instance. Required: Yes Type: string username (string) Name: username Description: Name of the environment variable containing an authorized user for the given Elasticsearch instance. Required: Yes Type: string Minimum length: 1

Outputs

error

Type:	scope
Root object:	ErrorOutput
Properties	error (string) Required: Yes Type: string
Objects ErrorOutput (object) Type: object Properties error (string) Required: Yes Type: string

success

Type:	scope
Root object:	SuccessOutput
Properties	message (string) Required: Yes Type: string
Objects SuccessOutput (object) Type: object Properties message (string) Required: Yes Type: string