diff options
| author | Philipp Tanlak <philipp.tanlak@gmail.com> | 2023-10-19 17:54:18 +0200 |
|---|---|---|
| committer | Philipp Tanlak <philipp.tanlak@gmail.com> | 2023-10-19 18:21:58 +0200 |
| commit | 0daefa86b400efe08245f4f2a386f7341b76b24e (patch) | |
| tree | 60d743bb9734a9d7f46701a5ac9026650a330d49 /docs/configuration/depth.md | |
| parent | 03b3be0c3bbc70584e8988e1810dc28eacf4521f (diff) | |
docs: Add documentationv0.3.0
Diffstat (limited to 'docs/configuration/depth.md')
| -rw-r--r-- | docs/configuration/depth.md | 23 |
1 files changed, 23 insertions, 0 deletions
diff --git a/docs/configuration/depth.md b/docs/configuration/depth.md new file mode 100644 index 0000000..cabb0fa --- /dev/null +++ b/docs/configuration/depth.md @@ -0,0 +1,23 @@ +# Depth + +The `depth` config option allows you to specify how deep the scraping process should follow links from the initial URL. + +When no value is provided or `depth` is set to `0` link following is disabled and it will only scrape the initial URL. + +Example: + +```javascript +export const config = { + url: "http://example.com/", + depth: 2, + // ... +}; +``` + +With the config provided in the example the scraper would follow links like this: + +``` +http://example.com/ (depth = 0, initial URL) +↳ http://example.com/deeply (depth = 1) + ↳ http://example.com/deeply/nested (depth = 2) +``` |