add docker compose for testing spark connector

[penghuo]

Signed-off-by: Peng Huo penghuo@gmail.com

Description

add docker compose for testing.

customized spark image

The customized Spark image include opensearch-spark-30_2.12-3.0.0-SNAPSHOT.jar
docker build --pull --rm -f "spark/Dockerfile" -t maximus/spark:v3.2.1 "spark"

docker compose

docker compose includes

1. 1 OpenSearch 2.3 node.
2. 1 Spark master node.
3. 2 Spark worker node.

How to use

• launch spark-shell on master node.

./bin/spark-shell --master spark://spark-master:7077

• execute

scala> import org.apache.spark.sql.SQLContext
import org.apache.spark.sql.SQLContext

scala> import org.apache.spark.sql.SQLContext._
import org.apache.spark.sql.SQLContext._

scala> import org.opensearch.spark.sql._
import org.opensearch.spark.sql._

scala> val sql = new SQLContext(sc)

scala> val accounts = sql.read.json("/root/maximus/data/accounts.json")

scala> val options = Map("pushdown" -> "true", "opensearch.nodes" -> "maximus-os")

scala> accounts.saveToOpenSearch("accounts", options)

Check List

• New functionality includes testing.
  • All tests pass, including unit test, integration test and doctest
• New functionality has been documented.
  • New functionality has javadoc added
  • New functionality has user manual doc added
• Commits are signed per the DCO using --signoff

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

[codecov-commenter]

Codecov Report

Merging #1311 (b8524bd) into feature/spark-integration (45fc371) will not change coverage.
The diff coverage is n/a.

📣 This organization is not using Codecov’s GitHub App Integration. We recommend you install it so Codecov can continue to function properly for your repositories. Learn more

@@                     Coverage Diff                      @@
##             feature/spark-integration    #1311   +/-   ##
============================================================
  Coverage                        98.35%   98.35%           
  Complexity                        3627     3627           
============================================================
  Files                              343      343           
  Lines                             8951     8951           
  Branches                           578      578           
============================================================
  Hits                              8804     8804           
  Misses                             142      142           
  Partials                             5        5           

Flag	Coverage Δ
sql-engine	98.35% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

[Yury-Fridlyand]

Do you mind updating PR description? Something like

... for testing Spark connector

It will be reflected in released notes.