Wget & curl
Wget
Command syntax
wget [option] [URL]
Download sequence of files using Bash range syntax
wget http://example.com/file_{1..4}.webp
Useful options
Option | Description |
---|---|
--continue (-c for short) | Resume a previously interrupted download. |
--debug | Display request and response headers. |
--header="name: value" | Add or update a request header. |
-l depth | Maximum depth level for recursion, inf can be used for unlimited. |
-m | Turns on recursion and time-stamping, sets infinite recursion depth and keeps FTP directory listings. |
--no-parent | Do not ascend to the parent directory when retrieving recursively. Only the files below a certain hierarchy will be downloaded. |
--no-proxy | Don't use proxies, even if the appropriate *_proxy environment variable is defined. |
-O fileName | Save target as fileName. If - is used as argument, documents will be printed to standard output. |
-r | Turn on recursive retrieving. The default maximum depth is 5. |
--show-progress | Display the progress bar (in any verbosity). |
--user=usr --password=pwd | Specify the username and password to be used for both FTP and HTTP authentication. |
curl
Basic usage of the command is the same as wget
.
Useful options
Option | Description |
---|---|
-A 'Mozilla/5.0' | Set user-agent |
-x proxy:port | Use HTTP proxy |
-H 'Authorization: bearer-token' | Use custom header |
-u username:password | HTTP Basic Authentication |
-L | Follow any redirects until final destination is reached |
-X PUT | Set request method; if not passed GET is used by default |
-X POST -d 'key1=val1&key2=val2' | Specify key-value pairs in request body |
-X POST -d 'key1=val1' -d 'key2=val2' | Alternative way of sending multiple keys and values |
-X POST -d '{"k1":v1}' -H 'Content-Type: application/json' | Use JSON data |
-o output.txt | Save response to a file |
-O | Download a file and save it as the same name |
-O -C | Continue a partial download |
-O --limit-rate 1M | Transfer rate limited to 1MB/s |