blob: b9755f71f9e4a653414ae8a2b845ff74eef6e557 (
plain) (
blame)
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
|
---
title: 'Link Following'
weight: 5
---
The `follow` config option allows you to specify a list of CSS selectors that determine which links the scraper should follow.
When no value is provided the scraper will follow all links found with the `a[href]` selector.
```javascript {filename="Configuration"}
export const config = {
url: "http://example.com/",
follow: [
".pagination > a[href]",
".nav a[href]",
],
// ...
};
```
## Following non `href` attributes
For special cases where the link is not to be found in the `href`, you specify a selector with a different ending attribute.
```javascript {filename="Configuration"}
export const config = {
url: "http://example.com/",
follow: [
".articles > div[data-url]",
],
// ...
};
```
|