git @ Cat's Eye Technologies yastasoti / 6d7b00a
Call the option --archive-to, as "links" is a bit redundant here Chris Pressey 1 year, 10 months ago
2 changed file(s) with 16 addition(s) and 10 deletion(s). Raw diff Collapse all Expand all
1818
1919 * archive youtube links with youtube-dl.
2020 * logging
21 * ignore certain URLs
2221 * Handle failures (redirects, etc) better. Fall back to external tool like `wget` or `curl`.
2322
2423 ### Examples ###
3231
3332 Archive stuff off teh internets:
3433
35 yastasoti --archive-links-to=downloads links.json
34 cat >links.json << EOF
35 [
36 {
37 "url": "http://catseye.tc/"
38 }
39 ]
40 EOF
41 yastasoti --archive-to=downloads links.json
3642
37 If it is only desired that the links be checked, `--check-links` will
38 make `HEAD` requests and will not save any of the responses.
43 ### Requirements ###
3944
40 Requirements: requests
45 Python 2.7 for sure, Python 3.x not sure, will need to run some tests.
4146
42 Optional dependencies: tqdm
47 Requires `requests` Python library and/or `wget` external utility to make
48 network requests.
4349
44 TODO: Update this documentation and make it make sense
50 If `tqdm` Python library is installed, will display a nice progress bar.
173173 help='JSON files containing the links to archive'
174174 )
175175
176 argparser.add_argument('--archive-links-to', metavar='DIRNAME', type=str, default=None,
176 argparser.add_argument('--archive-to', metavar='DIRNAME', type=str, default=None,
177177 help='Download a copy of all web objects linked to from the entries'
178178 )
179179 argparser.add_argument('--archive-missing-only', action='store_true',
210210 fragile=options.fragile,
211211 )
212212
213 if options.archive_links_to:
213 if options.archive_to:
214214 traverser = LinkArchiver(
215 links, options.archive_links_to, missing_only=options.archive_missing_only,
215 links, options.archive_to, missing_only=options.archive_missing_only,
216216 **common_kwargs
217217 )
218218 elif True: