Skip to content
This repository has been archived by the owner on Mar 12, 2024. It is now read-only.

关于user-agent #45

Closed
puppylpg opened this issue Nov 17, 2020 · 3 comments
Closed

关于user-agent #45

puppylpg opened this issue Nov 17, 2020 · 3 comments
Labels
suggestion Suggestion for improvement

Comments

@puppylpg
Copy link
Owner

建议描述

一开始图方便，把user-agent写死了，确实不太合适，考虑修改一下。两种想法：

  1. 让用户自己配置，就像cookie一样；
  2. 继续对用户隐藏，不过往上多找一些user-agent，放入单独的文件，每次爬取从里面随机挑一个user-agent；

对于防止封禁有一定好处。
@puppylpg puppylpg added the suggestion Suggestion for improvement label Nov 17, 2020
@puppylpg puppylpg mentioned this issue Dec 11, 2020
Merged
4 tasks
@zzwk
Copy link
Collaborator

zzwk commented Dec 13, 2020

大佬好，我有点不明白，普通用户即使使用不同浏览器也不会是同一个cookie吧，每次请求都随机一个UA却是同一个cookie访问不会触发网易的反爬机制吗？感觉就跟同一个账户频繁更换IP一样危险…

@puppylpg puppylpg mentioned this issue Dec 14, 2020
Merged
3 tasks
@puppylpg
Copy link
Owner Author

大佬好，我有点不明白，普通用户即使使用不同浏览器也不会是同一个cookie吧，每次请求都随机一个UA却是同一个cookie访问不会触发网易的反爬机制吗？感觉就跟同一个账户频繁更换IP一样危险…

嗯，也有道理。我现在改成配置里贴自己的ua，如果不贴，系统随机一个ua出来，本次爬取所有的请求都使用这个ua。这样总比一直用一个写死在代码里的ua好: #55

@puppylpg puppylpg mentioned this issue Dec 14, 2020
Closed
@z1y1m1
Copy link

z1y1m1 commented Aug 30, 2023

有个fake_useragent包可以考虑，专门干这个用的

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Assignees
No one assigned
Labels
suggestion Suggestion for improvement
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@puppylpg @z1y1m1 @zzwk