git @ Cat's Eye Technologies yastasoti / 812f066
Some more notes. Chris Pressey 6 years ago
1 changed file(s) with 3 addition(s) and 2 deletion(s). Raw diff Collapse all Expand all
44
55 Yet another script to archive stuff off teh internets.
66
7 It's not a spider that automatically crawls previously undiscovered pages — it's intended
8 to be run by a human to make backups of pages they have already read and recorded.
7 It's not a spider that automatically crawls previously undiscovered webpages — it's intended
8 to be run by a human to make backups of resources they have already seen and recorded the URLs of.
99
1010 It was split off from [Feedmark][], which doesn't itself need to support this function.
1111
7979 * Archive youtube links with youtube-dl.
8080 * Handle failures (redirects, etc) better (detect 503 / "connection refused" better.)
8181 * Allow use of an external tool like `wget` or `curl` to do fetching.
82 * Multiple `--article-roots`.
8283
8384 [Feedmark]: http://catseye.tc/node/Feedmark