Using the flag delete_retention_ms gives error with redpanda cluster configuration.

[singhsneetendra]

Version & Environment

Redpanda version: v21.10.2 (rev bcc20e8)

Using this as base file to deploy 3 node cluster.
https://github.com/vectorizedio/redpanda/blob/dev/src/go/k8s/config/samples/one_node_cluster.yaml

My redpanda yaml

apiVersion: redpanda.vectorized.io/v1alpha1
kind: Cluster
metadata:
labels:
instance: testing
name: redpanda
namespace: redpanda-cluster
spec:
additionalConfiguration:
redpanda.auto_create_topics_enabled: "true"
redpanda.delete_retention_ms: "604800000"
redpanda.enable_sasl: "false"
redpanda.log_cleanup_policy: delete
configuration:
adminApi:
- port: 9644
developerMode: true
kafkaApi:
- port: 9092
pandaproxyApi:
- port: 8082
rpcServer:
port: 33145
image: vectorized/redpanda
replicas: 3
resources:
limits:
cpu: 1
memory: 1.2Gi
requests:
cpu: 1
memory: 1.2Gi
storage:
capacity: 10Gi
version: latest

value for redpanda.delete_retention_ms: "604800000" fails with error
INFO 2021-11-28 16:12:28,489 [shard 0] redpanda::main - application.cc:324 - Use rpk config set <cfg> <value> to change values below:
INFO 2021-11-28 16:12:28,490 [shard 0] redpanda::main - application.cc:200 - Failure during startup: YAML::TypedBadConversion<std::__1::chrono::duration<long long, std::__1::ratio<1l, 1000l> > > (yaml-cpp: error at line 42, column 24: bad conversion)

any value above 950400 throw above error.

[jcsp]

Trying this with redpanda 21.11.1 (Neetendra mentioned in slack they were also using this version), redpanda itself doesn't seem to have a problem with a config like:

redpanda:
  delete_retention_ms: 604800000

I can see the value getting loaded:

INFO  2021-12-09 13:32:41,084 [shard 0] redpanda::main - application.cc:332 - redpanda.delete_retention_ms:604800000	- delete segments older than this - default 1 week

Seems like something could be going wrong in the operator, although I'm not sure what, as apparently smaller values do get passed through correctly.

[alenkacz]

@singhsneetendra can you please paste the part of redpanda logs where it shows the config loaded?

I'll also try to reproduce. Thanks for filing the issue

[singhsneetendra]

cloud_storage_bucket: nimbus-testing-celerway-com-redpanda-data
cloud_storage_reconciliation_interval_ms: 900
cloud_storage_max_connections: 5
cloud_storage_trust_file: /etc/ssl/certs/ca-certificates.crt
enable_sasl: false
log_segment_size: 536870912
auto_create_topics_enabled: true
delete_retention_ms: 6.048e+08
enable_idempotence: true
enable_transactions: true
log_cleanup_policy: delete

value for delete_retention_ms read as exponential pattern (edited)
redpanda_party
eyes
dancing_chicken

[jcsp]

Thanks for that @singhsneetendra

Looks like this is an issue on the path through our k8s operator, where the integer setting is getting passed through as a float (and thereby scientific notation) rather than an integer.

Intermediate yaml representations don't distinguish between floats and integers, but redpanda does at the point it loads values.

[singhsneetendra]

That means there is code fix required in redpanda operator CRD ?
or there is way to specify it in some other way in custom yaml ?

[singhsneetendra]

@jcsp you view on above comments ?

[jcsp]

@singhsneetendra it sounds like a code fix is needed in operator, I don't know if there is a way to workaround this and inject a config by hand.

[alenkacz]

@singhsneetendra so I spent some time on this, can you share which version of the operator were you running? I verified with 21.10.2 and also the latest and this is no longer an issue - we already fixed this in the past.

[singhsneetendra]

@alenkacz , Yes I was running 21.7.* something and the updated version for operator fixed the issue and is working fine.
Thanks for help