[FEATURE] Support more options in create index statement

[dai-chen]

Is your feature request related to a problem?

Currently Flint streaming job uses Foreach Batch sink which creates temporary checkpoint folder by default. The checkpoint will be lost after restart.

What solution would you like?

Add another option in create statement to allow user configure where they want to store the checkpoint data. Without the location provided, we can assume user doesn't concern about this and stick with temporary checkpoint as before.

For example:

CREATE SKIPPING INDEX ON alb_logs
(
  client_ip VALUE_SET
)
WITH (
  auto_refresh = true,
  checkpoint_location = 's3://...'
)

What alternatives have you considered?

Use some default location of current database. For example, because we map data source concept to SparkSQL catalog, we may use the root folder of a S3 data source. But this needs user awareness and write permission.

Do you have any additional context?

N/A

[dai-chen]

Similarly, we want to support other options such as refresh interval, index setting etc.

CREATE SKIPPING/INDEX/MV ...
WITH (
  auto_refresh = [true | false],
  auto_start = [true | false],
  refresh_interval = '1s',
  checkpoint_location = 's3://...',
  index_settings = '{ "key": value ... }'
);

[dai-chen]

auto_start is low priority and will be tracked separately.