Upgraded Azure Pipelines to support demands

[Eldarrin]

Signed-off-by: mortx mortxbox@live.com

Old PR: #2707

You can now supply a master template to support demands. NOTE: You must have a master supporting agent even if it is offline!

Checklist

• Commits are signed with Developer Certificate of Origin (DCO - learn more)
• Tests have been added
• A PR is opened to update our Helm chart (repo) (if applicable, ie. when deployment manifests are modified)
• A PR is opened to update the documentation on (repo) (if applicable)
• Changelog has been updated and is aligned with our changelog requirements

Relates to kedacore/keda-docs#685
Relates to #2328

[tomkerkhove]

@JorTurFer would you mind reviewing this one when you have time? It's been open for a while

[JorTurFer]

I'm waiting your opinions is docs

[tomkerkhove]

@Eldarrin Any thought if we can close this PR soon? We are planning to release in 2 weeks.

[Eldarrin]

@tomkerkhove Its all fine and ready to be merged I believe. I'm actually running it in production at my client's organisation.

[tomkerkhove]

Nice to hear! But we should probably move to the approach proposed by @JorTurFer if that's OK for you?

[Eldarrin]

Ok, let me have a look. Want me to take a new fork and start from there?

[tomkerkhove]

I don't have a problem if you want to update this PR, but it's up to what you prefer. We can have both approaches in parallel to compare as well if you want.

[Eldarrin]

Now support demands and parent metadata, tested it on my clusters as working; added docs to support

[tomkerkhove]

Thank you @JorTurFer Do you want to have a look?

[JorTurFer]

/run-e2e pipeline
Update: You can check the progress here

mine

[JorTurFer]

I understand the "parent" approach, but requiring an already deployed deployment only for registering the job looks like overkill. Nothing to add to the code itself, but I think that we need to solve the approach for that in the docs repo.

[JorTurFer]

@Eldarrin
We have plan to do a release aiming to next week, could you solve the conflicts in pkg/scalers/azure_pipelines_scaler.go to merge this PR?

[JorTurFer]

There is a code linting issue, could you solve it please?

[JorTurFer]

/run-e2e pipelines*
Update: You can check the progress here

[zroubalik]

@JorTurFer @v-shenoy @tomkerkhove what is the status of this?

[JorTurFer]

This PR is ready but we are waiting till some things are solved in docs