requests.exceptions.HTTPError: 403 Client Error

[deadpixel134]

Today I got this message.

E:\fantiadl-master>fantiadl.py https://fantia.jp/posts/1731261
Fantia session cookie (_session_id or cookies.txt path): E:\fantiadl-master\cookies.txt
Downloading post 1731261...
Traceback (most recent call last):
  File "E:\fantiadl-master\fantiadl.py", line 111, in <module>
    downloader.download_post(url_groups[1])
  File "E:\fantiadl-master\models.py", line 415, in download_post
    response.raise_for_status()
  File "C:\Python310\lib\site-packages\requests\models.py", line 960, in raise_for_status
    raise HTTPError(http_error_msg, response=self)
requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://fantia.jp/api/v1/posts/1731261

Checked my cookies.txt, updated to latest one but no luck.

[Zellos234]

it looks like the API now need an additional "X-CSRF-Token" header or it will response 403.
this token seems to change everytime, some workaround is needed.

[zyzmrx]

Agree. Same issue is received with multiple session IDs/cookies/posts.

[ytoaa]

Is there a way to solve this?

[bitbybyte]

From what I can tell this is only required on API calls to the /posts endpoint currently. A new CSRF token will be generated on each page load, as seen in the post page source:

<meta name="csrf-token" content="GUvDwTBBc1Q4luaNH4ukLbbSzAoZwm1-Sc07qU8dYGtsvB80QKGf9mgNWCJQW8tbfodX2-WmOBPJchDhxKTLEw" /

This token is present on every page but I don't see any other places it's actively being used. We will probably want to eventually implement this for every API request by first hitting the actual page everywhere, but I believe what needs to happen for now is:

1. Request the post page, parse HTML for token
2. Add as a X-CSRF-Token header to /api/v1/posts/{} requests
3. Perform this same step for every post

[bitbybyte]

The referenced commit should fix this. Would appreciate tests to see if there are any other requests that seem to return 403.

[deadpixel134]

It's seems to be working to me.
I think we can close this issue but maybe need more people confirm that it's working.

[bitbybyte]

Got enough reports this is working so closing out, new release soon.