diff options
| author | Philipp Tanlak <philipp.tanlak@gmail.com> | 2023-10-19 17:54:18 +0200 |
|---|---|---|
| committer | Philipp Tanlak <philipp.tanlak@gmail.com> | 2023-10-19 18:21:58 +0200 |
| commit | 0daefa86b400efe08245f4f2a386f7341b76b24e (patch) | |
| tree | 60d743bb9734a9d7f46701a5ac9026650a330d49 /docs/configuration/caching.md | |
| parent | 03b3be0c3bbc70584e8988e1810dc28eacf4521f (diff) | |
docs: Add documentationv0.3.0
Diffstat (limited to 'docs/configuration/caching.md')
| -rw-r--r-- | docs/configuration/caching.md | 37 |
1 files changed, 37 insertions, 0 deletions
diff --git a/docs/configuration/caching.md b/docs/configuration/caching.md new file mode 100644 index 0000000..4a06435 --- /dev/null +++ b/docs/configuration/caching.md @@ -0,0 +1,37 @@ +# Caching + +The `cache` config option allows you to enable file-based request caching. When enabled every request cached with its raw response. When the cache is populated and you re-run the scraper, requests will be served directly from cache. + +This also allows you to modify your scraping script afterwards and collect new results immediately. + +Example: + +```javascript +export const config = { + url: "http://example.com/", + cache: "file", + // ... +}; +``` + +### Cache File + +When caching is enabled using the `cache: "file"` option, a `.cache` file will be created with the name of your scraping script. + +Example: + +```bash +$ flyscrape run hackernews.js # Will populate: hackernews.cache +``` + +### Shared cache + +In case you want to share a cache between different scraping scripts, you can specify where to store the cache file. + +```javascript +export const config = { + url: "http://example.com/", + cache: "file:/some/path/shared.cache", + // ... +}; +``` |