Sometimes it is necessary to download an archival copy of a complete website, luckily https://web.archive.org crawls the interwebs regularly. Unluckily there is no direct way to download a whole website directly.

One tool that facilitates this is the aptley named Wayback Machine Downloader:

• https://github.com/hartator/wayback-machine-downloader

Usage

wayback_machine_downloader https://example.com

I have been running into some issues with the original version:

Downloading https://www.example.com to websites/www.example.com/ 
from Wayback Machine archives.

Getting snapshot pages../Users/abc/.rbenv/versions/3.1.1/lib/
ruby/3.1.0/open-uri.rb:364: in `open_http': 400 BAD REQUEST
(OpenURI::HTTPError)
...

This fork is working:

• https://github.com/StrawberryMaster/wayback-machine-downloader

wayback_machine_downloader https://example.com

Downloading https://example.com to websites/example.com/
from Wayback Machine archives.
Getting snapshot pages from Wayback Machine API...
.. found 6350 snapshots.
Saved snapshot list to websites/example.com/.cdx.json
...

If you want to download only to a certain timestamp, use the --to switch:

wayback_machine_downloader --to 20241231235959 https://example.com

Update 2025

youtube-dl throws an error in 2025, use yt-dlp instead.

https://github.com/rg3/youtube-dl/blob/master/README.md#readme

Installation

brew install youtube-dl

Downloading a Video

youtube-dl 'https://www.youtube.com/watch?v=1234'

The link needs to be in inverted commas:

OK: youtube-dl 'https://www.youtube.com/watch?v=1234'
NG: youtube-dl https://www.youtube.com/watch?v=1234

Downloading a Playlist

youtube-dl -o '%(playlist)s/%(playlist_index)s - %(title)s.%(ext)s' playlist_url

How to select video quality from youtube-dl?

youtube-dl -F playlist_url
[youtube] Setting language
[youtube] playlist_url: Downloading webpage
[youtube] playlist_url: Downloading video info webpage
[youtube] playlist_url: Extracting video information
[info] Available formats for playlist_url:
format code extension resolution  note 
17          3gp       176x144     
36          3gp       320x240     
5           flv       400x240     
43          webm      640x360     
18          mp4       640x360     
22          mp4       1280x720    (best)

Select download quality youtube-dl -f 22 playlist_url