Unix & Linux: Why would curl and wget result in a 403 forbidden? (5 Solutions!!)
👉 https://amzn.to/4aLHbLD 👈 You’re literally one click away from a better setup — grab it now! 🚀👑
As an Amazon Associate I earn from qualifying purchases. Unix & Linux: Why would curl and wget result in a 403 forbidden?
The Question: I try to download a file with wget and curl and it is rejected with a 403 error
(forbidden).
I can view the file using the web browser on the same machine.
I try again with my browser's user agent, obtained by http://
www.whatsmyuseragent.com. I do this:
wget -U 'Mozilla/5.0 (X11; Linux x86_64; rv:30.0) Gecko/20100101 Firefox/30.0'
http://...
and
curl -A 'Mozilla/5.0 (X11; Linux x86_64; rv:30.0) Gecko/20100101 Firefox/30.0'
http://...
but it is still forbidden. What other reasons might there be for the 403, and
what ways can I alter the wget and curl commands to overcome them?
(this is not about being able to get the file - I know I can just save it from
my browser; it's about understanding why the command-line tools work
differently)
update
Thanks to all the excellent answers given to this question. The specific
problem I had encountered was that the server was checking the referrer. By
adding this to the command-line I could get the file using curl and wget.
The server that checked the referrer bounced through a 302 to another location
that performed no checks at all, so a curl or wget of that site worked cleanly.
If anyone is interested, this came about because I was reading http://css-
tricks.com/forums/topic/font-face-in-base64-is-cross-browser-compatible page to
learn about embedded CSS and was trying to look at the site's css for an
example. The actual URL I was getting trouble with was this and the curl I
ended up with is
curl L -H 'Referer: http://css-tricks.com/forums/topic/fo...
cross-browser-compatible/' http://cloud.typography.com/610186/69...
fonts.css
and the wget is
wget -referer='http://css-tricks.com/forums/topic/fo...
cross-browser-compatible/' http://cloud.typography.com/610186/69...
fonts.css
Very interesting.
Solutions: Please watch the whole video to see all solutions, in order of how many people found them helpful
With thanks & praise to God, and with thanks to the many people who have made this project possible! | Content (except music & images) licensed under cc by-sa 3.0 | Music: https://www.bensound.com/royalty-free... | Images: https://stocksnap.io/license & others | With thanks to user user3707737 (https://unix.stackexchange.com/users/..., user starfry (https://unix.stackexchange.com/users/..., user solde9 (https://unix.stackexchange.com/users/..., user Richard (https://unix.stackexchange.com/users/..., user Lekensteyn (https://unix.stackexchange.com/users/..., user garethTheRed (https://unix.stackexchange.com/users/..., user Bruce Ediger (https://unix.stackexchange.com/users/..., and the Stack Exchange Network (http://unix.stackexchange.com/questio.... Trademarks are property of their respective owners. Disclaimer: All information is provided "AS IS" without warranty of any kind. You are responsible for your own actions. Please contact me if anything is amiss at Roel D.OT VandePaar A.T gmail.com.