Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Copy known zyte_api_session-prefixed meta into session initialization requests #205

Merged
merged 5 commits into from
Jun 25, 2024

Conversation

Gallaecio
Copy link
Contributor

@Gallaecio Gallaecio commented Jun 21, 2024

@VMRuiz reported that, to implement a check method that takes into account meta-defined locations, the init request also needs to have that meta key set. I went a bit further and made it so that any zyte_api_session_-prefixed meta key is copied into the init request, for extra flexibility.

To do:

  • Docs about the meta key copy.
  • Docs about the need, in these scenarios, to create separate pools per location. Or maybe we should provide a default implementation that does so? Maybe even the ability to set a custom pool through meta? (separate PR)

@Gallaecio Gallaecio requested review from kmike and wRAR June 21, 2024 19:29
Copy link

codecov bot commented Jun 21, 2024

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 97.56%. Comparing base (7151e73) to head (7780645).

Additional details and impacted files
@@           Coverage Diff           @@
##             main     #205   +/-   ##
=======================================
  Coverage   97.56%   97.56%           
=======================================
  Files          14       14           
  Lines        1476     1476           
  Branches      309      310    +1     
=======================================
  Hits         1440     1440           
  Misses         15       15           
  Partials       21       21           
Files Coverage Δ
scrapy_zyte_api/_session.py 100.00% <ø> (ø)

@Gallaecio
Copy link
Contributor Author

I’ll keep this PR simple and easy to merge, just copy the 2 known meta keys, and open a separate PR with additional ideas brought up with working on this.

@Gallaecio Gallaecio marked this pull request as ready for review June 25, 2024 09:38
@Gallaecio Gallaecio changed the title Copy zyte_api_session-prefixed meta into session initialization requests Copy known zyte_api_session-prefixed meta into session initialization requests Jun 25, 2024
@Gallaecio Gallaecio merged commit 748be07 into scrapy-plugins:main Jun 25, 2024
19 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants