Persisting runtime env info into Flint metadata

[dai-chen]

Description

Persist env info in Flint metadata, such as EMR-S job info. We persist them for CREATE statement regardless of index options, such as auto_refresh.

TODO

1. Refactor Flint metadata to de-dup serialization code in Refactoring Flint metadata and build index API #24
2. Remove hardcoding env name in [FEATURE] Add Flint config for env to persist in metadata #59

Example

For EMR-S Spark:

# Skipping index test
CREATE SKIPPING INDEX ON myglue.default.http_logs (clientip VALUE_SET);

GET flint_myglue_default_http_logs_skipping_index/_mapping
{
    "mappings": {
      "_meta": {
        "kind": "skipping",
        "indexedColumns": 
        ...
        "source": "myglue.default.http_logs",
        "version": "0.1.0",
        "properties": {
          "env": {
            "SERVERLESS_EMR_VIRTUAL_CLUSTER_ID": "XXXXXX",
            "SERVERLESS_EMR_JOB_ID": "YYYYYY"
          }
        }
      },
      ...

# Covering index test
CREATE INDEX clientip_and_request ON default.http_logs_streaming (clientip, request);

GET flint_myglue_default_http_logs_streaming_clientip_and_request_index/_mapping
{
    "mappings": {
      "_meta": {
        "kind": "covering",
        "indexedColumns":
        ...
        "name": "clientip_and_request",
        "options": {},
        "source": "myglue.default.http_logs_streaming",
        "properties": {
          "env": {
            "SERVERLESS_EMR_VIRTUAL_CLUSTER_ID": "XXXXXX",
            "SERVERLESS_EMR_JOB_ID": "YYYYYY"
          }
        }
      },
      ...

For Non-EMR-S Spark, the properties should be empty without any env info:

spark-sql> CREATE SKIPPING INDEX ON myglue.stream.lineitem_tiny (l_shipdate VALUE_SET);

GET flint_myglue_stream_lineitem_tiny_skipping_index/_mapping
{
    "mappings": {
      "_meta": {
        "kind": "skipping",
        "indexedColumns":
        ...
        "source": "myglue.stream.lineitem_tiny",
        "version": "0.1.0",
        "properties": {}
      },
      ...

Issues Resolved

#57

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.