[Instagram] API URL redirects to login page even on home servers

[lorenzos]

Looks like Instagram started redirecting the API URL used by the bridge to its login page, even on household internet connections with "clean" IP addresses. Currently, I'm experince the following on my Raspberry and my laptop at home, and my desktop at office, not only for RSS-Bridge, but on browsers too:

$ curl -I https://www.instagram.com/web/search/topsearch/?query=makenoisemusic
HTTP/1.1 302 Found
Content-Type: text/html; charset=utf-8
Location: https://www.instagram.com/accounts/login/

This is the same as #1617, but looks like the "workaround" doesn't work anymore.

Is it game over?

[peter279k]

Try to use this following command:

curl -H "User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/89.0.4389.82 Safari/537.36" -I https://www.instagram.com/web/search/topsearch/?query=makenoisemusic

And it will get following response headers:

HTTP/2 200
content-type: application/json; charset=utf-8
vary: Accept-Language, Cookie, Accept-Encoding
content-language: en
date: Wed, 17 Mar 2021 19:21:49 GMT
strict-transport-security: max-age=31536000
cache-control: private, no-cache, no-store, must-revalidate
pragma: no-cache
expires: Sat, 01 Jan 2000 00:00:00 GMT
x-frame-options: SAMEORIGIN
......

Why? The current reason that the Instagram will check the User-Agent header.

It can use the correct user agent header via the chrome://version/ URL on Google Chrome.

[lorenzos]

@peter279k Nope, it's the same, I get the same 302. In fact, the redirect occurs also when using the browser directly to open the URL...

[peter279k]

It's weird. I get the different result about with/without User-Agent header.

[peter279k]

I guess Instagram will check some stuffs when some clients send request via this URL.

I believe the one of stuffs is about setting User-Agent header and I'm not sure others.

Oh..I found that you're located on the home server, and Instagram will check the remote IP address.

If your home server IP address is not public (such as home server has private IP address), Instagram will block this request then being forced to redirect to the login page with 302 HTTP status code.

[lorenzos]

@peter279k Yeah, I think it started blocking also private IP addresses in addition to VPS and VPN. In fact, when I tried the request before opening the issue, it succeeded just a couple times, then started 302ing also when switching client (cURL and various browsers). I'll try restarting the router many times and see if I'll get an unblocked IP address... Anyway is clear the situation is getting worse 😞

[lorenzos]

Got a new home IP address, with which I got a couple successful requests (200). I won't fire more for now, and will wait for RSS-Bridge to try to recover old missed posts.

[em92]

Just read, that bibliogram has the same problem with rate limiting. https://proxy.vulpes.one/gemini/cadence.moe/gemlog/2020-12-17-future-of-bibliogram.bliz

[em92]

If you have instagram account, you can try this patch.
#1891 (comment)

Also suggesting to increase cache timeout for this bridge to higher value. For example:

const CACHE_TIMEOUT = 43200; // 12 hours

In any case closing as duplicate of #1891

[lorenzos]

Thanks @em92, I'll try to borrow an account from family 😄 Hopefully it'll work.

Also suggesting to increase cache timeout for this bridge to higher value

Is it enough to add const CACHE_TIMEOUT = 43200 in code? Does this override any smaller value in the feed URL? For example:

?action=display&bridge=Instagram&u=profile&media_type=all&_cache_timeout=900&format=Atom

[em92]

Does this override any smaller value in the feed URL?

No, it does not. You need to change in feed URL instead or disable custom_timeout in config.php

I'll try to borrow an account from family

FYI, I tried with doing the same idea with my account on facebook. It lead me to account suspention. See #2047 (comment). So, I suggest to use your own account instead.

[lorenzos]

@em92 Thanks for the warning. I don't have an account, that's why I was thinking borrowing one. I guess I won't, I'll consider making my own.