blob: 4a06435ccecaafd6f83c70595d7ab72952b0fdcd (
plain) (
blame)
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
|
# Caching
The `cache` config option allows you to enable file-based request caching. When enabled every request cached with its raw response. When the cache is populated and you re-run the scraper, requests will be served directly from cache.
This also allows you to modify your scraping script afterwards and collect new results immediately.
Example:
```javascript
export const config = {
url: "http://example.com/",
cache: "file",
// ...
};
```
### Cache File
When caching is enabled using the `cache: "file"` option, a `.cache` file will be created with the name of your scraping script.
Example:
```bash
$ flyscrape run hackernews.js # Will populate: hackernews.cache
```
### Shared cache
In case you want to share a cache between different scraping scripts, you can specify where to store the cache file.
```javascript
export const config = {
url: "http://example.com/",
cache: "file:/some/path/shared.cache",
// ...
};
```
|