From b1e2c8fd5cb5dfa46bc440a12eafaf56cd844b1c Mon Sep 17 00:00:00 2001 From: Philipp Tanlak Date: Mon, 24 Nov 2025 20:54:57 +0100 Subject: Docs --- public/docs/configuration/retry/index.html | 576 +++++++++++++++++++++++++++++ 1 file changed, 576 insertions(+) create mode 100644 public/docs/configuration/retry/index.html (limited to 'public/docs/configuration/retry/index.html') diff --git a/public/docs/configuration/retry/index.html b/public/docs/configuration/retry/index.html new file mode 100644 index 0000000..3da9826 --- /dev/null +++ b/public/docs/configuration/retry/index.html @@ -0,0 +1,576 @@ + + + + + + + + + + + + + + + Retry – Flyscrape + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
+
+ + +
+ +
+ + + + + + +
+
+
+
+ Documentation +
+
+ Configuration +
Retry
+
+ +
+

Retry

+

The retry feature allows the scraper to automatically retry failed requests. This is particularly useful when dealing with unstable networks or servers that occasionally return error status codes.

+

The retry feature is automatically enabled and will retry requests that return the following HTTP status codes:

+
    +
  • 403 Forbidden
    • +
  • 408 Request Timeout
    • +
  • 425 Too Early
    • +
  • 429 Too Many Requests
    • +
  • 500 Internal Server Error
    • +
  • 502 Bad Gateway
    • +
  • 503 Service Unavailable
    • +
  • 504 Gateway Timeout
    • +
+

Retry Delays +

After a failed request, the scraper will wait for a certain amount of time before retrying the request. The delay increases with each consecutive failed attempt, according to the following schedule:

+
    +
  • 1st retry: 1 second delay
    • +
  • 2nd retry: 2 seconds delay
    • +
  • 3rd retry: 5 seconds delay
    • +
  • 4th retry: 10 seconds delay
    • +
+ +
+
+
Rate LimitingCaching
+ +
+
+
+ + + + + + + + + + -- cgit v1.2.3