ActiveMQ scaler not respecting HTTPS protocol setting

[M-Adam]

Report

I have KEDA running on my local Docker Desktop cluster (Windows).
I am using ActiveMQ scaler
https://keda.sh/docs/2.7/scalers/activemq/
Here is my scaledObject setup:

apiVersion: v1
kind: Secret
metadata:
  name: activemq-secret
  namespace: my-namespace
type: Opaque
data:
  activemq-password: ACTIVEMQ_PASSWORD
  activemq-username: ACTIVEMQ_USERNAME
---
apiVersion: keda.sh/v1alpha1
kind: TriggerAuthentication
metadata:
  name: my-auth
  namespace: my-namespace
spec:
  secretTargetRef:
  - parameter: username
    name: activemq-secret
    key: activemq-username
  - parameter: password
    name: activemq-secret
    key: activemq-password
---
apiVersion: keda.sh/v1alpha1
kind: ScaledObject
metadata:
  name: my-name
  namespace: my-namespace
spec:
  scaleTargetRef:
    name: my-deployment
  pollingInterval: 30
  cooldownPeriod: 300
  minReplicaCount: 1
  maxReplicaCount: 3
  fallback:
    failureThreshold: 3
    replicas: 1
  triggers:
  - type: activemq
    metadata:
      targetQueueSize: "10"
      restAPITemplate: "https://myBrokerHost:8162/api/jolokia/read/org.apache.activemq:type=Broker,brokerName=myBrokerName,destinationType=Queue,destinationName=keda-test/QueueSize"
    authenticationRef:
      name: my-auth

Please note that I've used restAPITemplate that starts with HTTPS. However, I can see the following in the logs of keda-metrics-apiserver-xxx pod:

2022-06-18T17:13:38.207Z | I0618 17:13:38.207003 1 fallback.go:109] keda_metrics_adapter/provider "msg"="Suppressing error error inspecting ActiveMQ queue size: Get \"http://myBrokerHost:8162/api/jolokia/read/org.apache.activemq:type=Broker,brokerName=myBrokerName,destinationType=Queue,destinationName=keda-test/QueueSize\": EOF, falling back to 1 replicas"

Note the HTTP protocol in front of the URL, while it should be HTTPS. I believe it should work because the docs clearly say that you can overwrite the default URL template:

I am not able to use HTTP URL, because the broker I am connecting to only has HTTPS port exposed (8162).
I have tried using all available parameters of the scaler (managementEndpoint, corsHeader) hoping that one of them will fix the problem, but no luck...

Expected Behavior

I expect to be able to choose the protocol that Keda will use to get metrics from ActiveMQ REST API.

Actual Behavior

The protocol is hardcoded to be HTTP (?) or it doesn't respect the full API template passed as a parameter of the ScaledObject. Hence, the Scaler fails to get the metric from AMQ.

Steps to Reproduce the Problem

1. Have an ActiveMQ broker instance running somewhere, with HTTPS endpoint exposed for the REST API access.
2. Deploy the ScaledObject I posted above to your K8S cluster with your own HTTPS URL and required configuration.
3. Check the Keda Operator logs.

Logs from KEDA operator

2022-06-18T18:08:56.300Z | 1.6555757363001995e+09 ERROR activeMQ_scaler Unable to access activeMQ management endpoint {"managementEndpoint": "myBrokerHost:8162", "error": "Get \"http://myBrokerHost:8162/api/jolokia/read/org.apache.activemq:type=Broker,brokerName=myBrokerName,destinationType=Queue,destinationName=keda-test/QueueSize\": EOF"}
2022-06-18T18:08:56.300Z | github.com/kedacore/keda/v2/pkg/scalers.(*activeMQScaler).IsActive
2022-06-18T18:08:56.300Z | /workspace/pkg/scalers/activemq_scaler.go:163
2022-06-18T18:08:56.300Z | github.com/kedacore/keda/v2/pkg/scaling/cache.(*ScalersCache).IsScaledObjectActive
2022-06-18T18:08:56.300Z | /workspace/pkg/scaling/cache/scalers_cache.go:93
2022-06-18T18:08:56.300Z | github.com/kedacore/keda/v2/pkg/scaling.(*scaleHandler).checkScalers
2022-06-18T18:08:56.300Z | /workspace/pkg/scaling/scale_handler.go:278
2022-06-18T18:08:56.300Z | github.com/kedacore/keda/v2/pkg/scaling.(*scaleHandler).startScaleLoop
2022-06-18T18:08:56.300Z | /workspace/pkg/scaling/scale_handler.go:149
2022-06-18T18:08:56.300Z | 1.6555757363002703e+09 ERROR scalehandler Error getting scale decision {"scaledobject.Name": "my-name", "scaledObject.Namespace": "my-namespace", "scaleTarget.Name": "my-deployment", "error": "Get \"http://myBrokerHost:8162/api/jolokia/read/org.apache.activemq:type=Broker,brokerName=myBrokerName,destinationType=Queue,destinationName=keda-test/QueueSize\": EOF"}

KEDA Version

2.7.1

Kubernetes Version

1.23

Platform

Other

Scaler Details

ActiveMQ

Anything else?

Looking at the scaler code, this is where the restAPITemplate is parsed:
https://github.com/kedacore/keda/blob/main/pkg/scalers/activemq_scaler.go#L79
And this is where the protocol from the parameter seems to get lost:
https://github.com/kedacore/keda/blob/main/pkg/scalers/activemq_scaler.go#L175

I think it is a relatively easy and quick to fix, but I don't know Go and its APIs 😢

[JorTurFer]

hey @M-Adam
Thanks for reporting it, I have checked the code and the error seems that's here
https://github.com/kedacore/keda/blob/main/pkg/scalers/activemq_scaler.go#L203
Even KEDA parses the template (including the protocol), when it gets the endpoint to do the query against to, it still uses the default template, ignoring the user provide value

[JorTurFer]

@M-Adam ,
We have merged a fix 1 hour ago. if you could use main tag to validate that the problem is solved would be nice

[M-Adam]

Hi @JorTurFer,
Thank you for the fix! I'd love to test this, but I don't know how.
Is it possible to get the new keda-xxx.yaml file from anywhere? So that I can install it like it's mentioned in the docs?

[JorTurFer]

There is not any yaml to download because main tag is not "production ready" because it's unstable (every commit updates that tag), remember this if you try it to undo the change after checking it or to use any other mechanism to pin it.
The way to do use main is just editing the deployment and changing the image tag from 2.7.1 to main, ie image: ghcr.io/kedacore/keda:2.7.1 -> image: ghcr.io/kedacore/keda:main

[M-Adam]

Finally, I had a moment to test this. All is working fine, can't see any errors in KEDA logs. My app gets scaled correctly. Looking forward to the production release :)