v2.4.0

Removed official support for Python 3.4, 3.5, 3.6, 3.7 and 3.8. Added official Python 3.13 support.

Backward-compatible X-Crawlera-prefixed headers are no longer translated into their matching Zyte API proxy mode headers, Zyte API now handles their translation on the server side.

Added a new ZYTE_SMARTPROXY_KEEP_HEADERS setting that allows disabling header dropping and translation.

Full Changelog: https://scrapy-zyte-smartproxy.readthedocs.io/en/latest/news.html

v2.3.5

Ban and throttling responses from Zyte API proxy mode are now handled in line with matching responses from Zyte Smart Proxy Manager.

Full Changelog: https://scrapy-zyte-smartproxy.readthedocs.io/en/latest/news.html

v2.3.4

Use a different stat prefix for Zyte API proxy mode.

Full Changelog: https://scrapy-zyte-smartproxy.readthedocs.io/en/latest/news.html

2.3.3

Fix response handling for Zyte API proxy mode.

Full Changelog: https://scrapy-zyte-smartproxy.readthedocs.io/en/latest/news.html

2.3.2

Detect scenarios where the proxy Request.meta key has probably been accidentally copied from an earlier response, warn about it, and fix the value.

The Zyte-Client header is again sent when using Zyte API proxy mode, now that Zyte API supports it.

Full Changelog: https://scrapy-zyte-smartproxy.readthedocs.io/en/latest/news.html

2.3.1

Fixed support for the proxy mode of Zyte API.

Full Changelog: https://scrapy-zyte-smartproxy.readthedocs.io/en/latest/news.html

v2.3.0

Support for the upcoming proxy API of Zyte API.

Full Changelog: https://scrapy-zyte-smartproxy.readthedocs.io/en/latest/news.html

v2.2.0

Support Scrapy 2.6.2. Scrapy 1.4+ is now required.

v2.1.0

Use a custom logger instead of the root one

v2.0.0

Following the upstream rebranding of Crawlera as Zyte Smart Proxy Manager, scrapy-crawlera has been renamed as scrapy-zyte-smartproxy, with the following backward-incompatible changes:

• The repository name and Python Package Index (PyPI) name are now scrapy-zyte-smartproxy.
• Setting prefixes have switched from CRAWLERA_ to ZYTE_SMARTPROXY_.
• Spider attribute prefixes and request meta key prefixes have switched from crawlera_ to zyte_smartproxy_.
• scrapy_crawlera is now scrapy_zyte_smartproxy.
• CrawleraMiddleware is now ZyteSmartProxyMiddleware, and its default url is now http://proxy.zyte.com:8011.
• Stat prefixes have switched from crawlera/ to zyte_smartproxy/.
• The online documentation is moving to https://scrapy-zyte-smartproxy.readthedocs.io/

Note: Zyte Smart Proxy Manager headers continue to use the X-Crawlera- prefix.

• In addition to that, the X-Crawlera-Client header is now automatically included in all requests.