From 0daefa86b400efe08245f4f2a386f7341b76b24e Mon Sep 17 00:00:00 2001 From: Philipp Tanlak Date: Thu, 19 Oct 2023 17:54:18 +0200 Subject: docs: Add documentation --- docs/configuration/rate-limiting.md | 14 ++++++++++++++ 1 file changed, 14 insertions(+) create mode 100644 docs/configuration/rate-limiting.md (limited to 'docs/configuration/rate-limiting.md') diff --git a/docs/configuration/rate-limiting.md b/docs/configuration/rate-limiting.md new file mode 100644 index 0000000..c3014d1 --- /dev/null +++ b/docs/configuration/rate-limiting.md @@ -0,0 +1,14 @@ +# Rate Limiting + +The `rate` config option allows you to specify at which rate the scraper should send out requests. The rate is measured in _Requests per Second_ (RPS) and can be set as a whole or decimal number to account for shorter and longer request intervals. + +When no `rate` is specified, rate limiting is disabled and the scraper will send out requests as fast as it can. + +Example: + +```javascript +export const options = { + url: "http://example.com/", + rate: 50, +}; +``` -- cgit v1.2.3