fix(utils.mjs) use the old stats.jenkins.io website (temporarily?)

[dduportal]

The plugin site generation fails since jenkins-infra/helpdesk#4265 was applied earlier today.

This PR is a short term fix to use the old stats website, still available through http://old.stats.jenkins.io (and still updated).

These files will have to be served in the new website

cc @lemeurherve @lemeurherveCB @krisstern @MarkEWaite for info

[dduportal]

Now it's failing on the Update Center plugins list, most probably due to jenkins-infra/helpdesk#2649

20:22:35  [@jenkins-cd/jenkins-plugin-site]: success Fetching 'https://updates.jenkins.io/plugin-versions.json' - 0.292s
20:22:35  [@jenkins-cd/jenkins-plugin-site]: error gatsby-source-jenkinsplugin: Failed to parse API call -  TypeError: Cannot convert undefined or null to object
20:22:35  [@jenkins-cd/jenkins-plugin-site]:     at Function.values (<anonymous>)
20:22:35  [@jenkins-cd/jenkins-plugin-site]:     at fetchPluginVersions (file:///home/jenkins/agent/workspace/Infra_plugin-site_PR-1890/plugins/gatsby-source-jenkinsplugins/utils.mjs:477:41)
20:22:35  [@jenkins-cd/jenkins-plugin-site]:     at processTicksAndRejections (node:internal/process/task_queues:95:5)
20:22:35  [@jenkins-cd/jenkins-plugin-site]:     at async Promise.all (index 4)
20:22:35  [@jenkins-cd/jenkins-plugin-site]:     at Module.sourceNodes (file:///home/jenkins/agent/workspace/Infra_plugin-site_PR-1890/plugins/gatsby-source-jenkinsplugins/gatsby-node.mjs:19:9)
20:22:35  [@jenkins-cd/jenkins-plugin-site]:     at runAPI (/home/jenkins/agent/workspace/Infra_plugin-site_PR-1890/node_modules/gatsby/src/utils/api-runner-node.js:509:16)

[dduportal]

plugin-site/plugins/gatsby-source-jenkinsplugins/utils.mjs

Line 33 in dbf1ff1

if (results.status !== 200) {

=> looks like the Axios HTTP client does not follow redirects 🤦

[dduportal]

Ping @timja @halkeye if you can help on this one

[timja]

I'm looking

[halkeye]

in theory you can do const results = await axios.get(url, { maxRedirects: 10 }); or something to handle the redirects. i can't really look till after work

[dduportal]

in theory you can do const results = await axios.get(url, { maxRedirects: 10 }); or something to handle the redirects. i can't really look till after work

One last try for me: 5489791

If it does not work, I don't mind anyone pushing on this branch (or doing another PR) to help as it starts to be late here

[dduportal]

in theory you can do const results = await axios.get(url, { maxRedirects: 10 }); or something to handle the redirects. i can't really look till after work

One last try for me: 5489791

If it does not work, I don't mind anyone pushing on this branch (or doing another PR) to help as it starts to be late here

does not work: I might have misunderstood the error. But the file in the new UC is strictly the same as the old UC. Only the redirections are differents.

I'm letting you folks have a look. Many thanks ❤️

[timja]

I'm unsure if its redirects axios should handle that automatically, that last change didn't help either, this is easy to run locally

[timja]

In node.js repl this is what I get which does not look right:

const axios = require('axios')
const url = "https://updates.jenkins.io/plugin-versions.json"
const results = await axios.get(url);
JSON.stringify(results.data)

result:

{
  "FileInfo": {
    "Path": "/current/plugin-versions.json",
    "Size": 18595440,
    "ModTime": "2024-09-11T19:53:20.9436167Z",
    "Sha1": "432462f8feffc3db439253c2c1ebe618b7346113",
    "Sha256": "9d482e0f77066d7bff52854df41a4ff73fb5967523ddf88fece1020b5bbb1101",
    "Md5": "208824be0422f1009c701562c739b003"
  },
  "IP": "my-ip",
  "ClientInfo": {
    "CountryCode": "GB",
    "ContinentCode": "EU",
    "City": "Brighton",
    "Country": "United Kingdom",
    "Latitude": <redacted>,
    "Longitude": <redacted>,
    "ASName": "My ISP",
    "ASNum": 5089
  },
  "MirrorList": [
    {
      "ID": 0,
      "Name": "fallback0",
      "HttpURL": "https://eastamerica.cloudflare.jenkins.io/",
      "RsyncURL": "",
      "FtpURL": "",
      "SponsorName": "",
      "SponsorURL": "",
      "SponsorLogoURL": "",
      "AdminName": "",
      "AdminEmail": "",
      "CustomData": "",
      "ContinentOnly": false,
      "CountryOnly": false,
      "ASOnly": false,
      "Score": 0,
      "Latitude": 0,
      "Longitude": 0,
      "ContinentCode": "NA",
      "CountryCodes": "US",
      "ExcludedCountryCodes": "",
      "Asnum": 0,
      "Comment": "",
      "Enabled": false,
      "StateSince": "0001-01-01T00:00:00Z",
      "Distance": 0,
      "ComputedScore": 0,
      "LastSync": "0001-01-01T00:00:00Z",
      "LastSuccessfulSync": "0001-01-01T00:00:00Z",
      "LastModTime": "0001-01-01T00:00:00Z"
    }
  ],
  "ExcludedList": [
    {
      "ID": 1,
      "Name": "westeurope",
      "HttpURL": "https://westeurope.cloudflare.jenkins.io/",
      "RsyncURL": "rsync://updates-jenkins-io-rsyncd.updates-jenkins-io.svc.cluster.local/jenkins/",
      "FtpURL": "",
      "SponsorName": "westeurope.cloudflare.jenkins.io",
      "SponsorURL": "",
      "SponsorLogoURL": "",
      "AdminName": "jenkins-infra",
      "AdminEmail": "jenkins-infra@googlegroups.com",
      "CustomData": "",
      "ContinentOnly": false,
      "CountryOnly": false,
      "ASOnly": false,
      "Score": 0,
      "Latitude": 48.8691,
      "Longitude": 2.324464,
      "ContinentCode": "EU",
      "CountryCodes": "FR",
      "ExcludedCountryCodes": "",
      "Asnum": 13335,
      "Comment": "",
      "Enabled": true,
      "ExcludeReason": "File size mismatch",
      "StateSince": "2024-09-11T19:27:57Z",
      "Distance": 280.6335,
      "ComputedScore": 0,
      "LastSync": "2024-09-11T19:56:56Z",
      "LastSuccessfulSync": "2024-09-11T19:56:56Z",
      "LastModTime": "2024-09-11T19:53:05Z"
    },
    {
      "ID": 2,
      "Name": "eastamerica",
      "HttpURL": "https://eastamerica.cloudflare.jenkins.io/",
      "RsyncURL": "rsync://updates-jenkins-io-rsyncd.updates-jenkins-io.svc.cluster.local/jenkins/",
      "FtpURL": "",
      "SponsorName": "eastamerica.cloudflare.jenkins.io",
      "SponsorURL": "",
      "SponsorLogoURL": "",
      "AdminName": "jenkins-infra",
      "AdminEmail": "jenkins-infra@googlegroups.com",
      "CustomData": "",
      "ContinentOnly": false,
      "CountryOnly": false,
      "ASOnly": false,
      "Score": 0,
      "Latitude": 35.221386,
      "Longitude": -80.83601,
      "ContinentCode": "NA",
      "CountryCodes": "US",
      "ExcludedCountryCodes": "",
      "Asnum": 13335,
      "Comment": "",
      "Enabled": true,
      "ExcludeReason": "File size mismatch",
      "StateSince": "2024-09-07T08:18:52Z",
      "Distance": 6445.7246,
      "ComputedScore": 0,
      "LastSync": "2024-09-11T19:57:10Z",
      "LastSuccessfulSync": "2024-09-11T19:57:10Z",
      "LastModTime": "2024-09-11T19:53:05Z"
    }
  ],
  "Fallback": true,
  "LocalJSPath": ""
}

[timja]

Reproducible in curl by passing the same headers (roughly, I had 2 drop a couple of compression algorithms as my curl couldn't handle whatever was being sent):

curl --compressed -H "User-Agent: axios/1.7.4" -H "Accept: application/json, text/plain, */*" -H "Accept-Encoding: gzip, deflate" -H "Host: mirrors.updates.jenkins.io" -H "Connection: keep-alive"  -L "https://updates.jenkins.io/plugin-versions.json"

[timja]

Minimised it down to an accept header, if you add an accept header its broken:

curl -H "Accept: application/json"  -L "https://updates.jenkins.io/plugin-versions.json"

---

I'll leave to you now @dduportal 😄

[dduportal]

Many thanks @timja !

Got the error, it's in mirrorbits. Incoming fix and I'll report in the Update Center helpdesk issue

[dduportal]

Reported in jenkins-infra/helpdesk#2649 (comment)