blob: 6522ce8b9114c0874f7e8598e1c36707af984ae0 (
plain) (
blame)
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
|
# Link Following
The `follow` config option allows you to specify a list of CSS selectors that determine which links the scraper should follow.
When no value is provided the scraper will follow all links found with the `a[href]` selector.
Example:
```javascript
export const config = {
url: "http://example.com/",
follow: [".pagination > a[href]", ".nav a[href]"],
// ...
};
```
### Following non `href` attributes
For special cases where the link is not to be found in the `href`, you specify a selector with a different ending attribute.
Example:
```javascript
export const config = {
url: "http://example.com/",
follow: [".articles > div[data-url]"],
// ...
};
```
|