make si mode two separate flags

[LenaAn]

Will split into commits that build later.
Also need to think about compatibility

Fixes: #4499

[LenaAn]

This change breaks forward compatibility.

Also I think it's better to base it on top of @mmaslankaprv 's change on Serde

[LenaAn]

Currently rpk topic create doesn't work since it passes shadow_indexing: {nullopt}

[Lazin]

nit: comment about version -4?

[Lazin]

on the other hand, the names are self descriptive

[Lazin]

that was genuinely terrible, good riddance

[Lazin]

LGTM, the only reservation that I have is that we might soon transition to serde so the serialisation of this commands might be done much simpler. In this form it breaks forward compatibility. We won't be able to roll back redpanda to previous version if at least one topic is created.

[jcsp]

It's not obvious to me why the on-disk encoding needs to change: it was already storing two bits of information (packed into shadow_indexing_mode), so surely it can store the same thing as two boolean attributes stores? Couldn't we just do some logic in the encode/decode path that changes converts from "two bools" into "value of shadow_indexing_mode"?

[LenaAn]

It's not obvious to me why the on-disk encoding needs to change: it was already storing two bits of information (packed into shadow_indexing_mode), so surely it can store the same thing as two boolean attributes stores? Couldn't we just do some logic in the encode/decode path that changes converts from "two bools" into "value of shadow_indexing_mode"?

@jcsp you mean not to change topic_properties, but to change adl<cluster::topic_configuration>::to( iobuf& out, cluster::topic_configuration&& t) and cluster::topic_configuration adl<cluster::topic_configuration>::from(iobuf_parser& in)?

What about incremental_topic_update? Currently the cause of the bug is this function assumes that only either read or write can be set/removed. https://github.com/redpanda-data/redpanda/blob/dev/src/v/cluster/topic_table.cc#L295

We actually have {read_enabled, read_disabled} * {write_enabled, write_disabled} = 4 states and {set_read, drop_read} * {set_write, drop_write} = 4 operation and try to encode that in 7 values of shadow_indexing_mode enum.