When working previously in Opera Developer Relations Team, Patrick and/or Bruce had given me the nickname: "curl dubost". My bug reports contained a lot of curl requests detailing the HTTP transactions with Web sites. Slowly, I have transitionned to use more and more httpie. For analyzing HTTP transactions, httpie is a very good lightweight tool. It is easy to install and use. It has syntax coloring. It has also some caveats.
There are a few things I use on a regular basis worth sharing.
Request View
When testing HTTP requests, it is important to know what you are really sending to the server. It sometimes create surprises. http -v will display the request headers in addition to the response headers. Let's try, for example on the command line the following:
→ http -v HEAD http://www.yahoo.com/ User-Agent:'Mozilla/5.0 (Mobile; rv:18.0) Gecko/18.0 Firefox/18.0'
The HTTP request will
HEAD / HTTP/1.1
Accept: */*
Accept-Encoding: gzip, deflate, compress
Host: www.yahoo.com
User-Agent: Mozilla/5.0 (Mobile; rv:18.0) Gecko/18.0 Firefox/18.0
And the HTTP response from Yahoo!
HTTP/1.1 200 OK
Age: 0
Cache-Control: private
Connection: keep-alive
Content-Encoding: gzip
Content-Type: text/html; charset=utf-8
Date: Mon, 05 Aug 2013 18:57:25 GMT
P3P: policyref="http://info.yahoo.com/w3c/p3p.xml", CP="CAO DSP COR CUR ADM DEV TAI PSA PSD IVAi IVDi CONi TELo OTPi OUR DELi SAMi OTRi UNRi PUBi IND PHY ONL UNI PUR FIN COM NAV INT DEM CNT STA POL HEA PRE LOC GOV"
Server: YTS/1.20.13
Set-Cookie: B=5r85hah8vvtcl&b=3&s=1d; expires=Thu, 06-Aug-2015 18:57:25 GMT; path=/; domain=.yahoo.com
Vary: Accept-Encoding
Nothing specifically weird. OK let's see and let's change the Accept header.
→ http -v HEAD http://www.yahoo.com/ User-Agent:'Mozilla/5.0 (Mobile; rv:18.0) Gecko/18.0 Firefox/18.0' Accept:"text/html"
Same request except that now, we are sending an Accept which has text/html instead of */*
HEAD / HTTP/1.1
Accept: text/html
Accept-Encoding: gzip, deflate, compress
Host: www.yahoo.com
User-Agent: Mozilla/5.0 (Mobile; rv:18.0) Gecko/18.0 Firefox/18.0
And the result is this time… a redirection
HTTP/1.1 302 Found
Age: 0
Cache-Control: private
Connection: keep-alive
Content-Type: text/html; charset=utf-8
Date: Mon, 05 Aug 2013 18:59:51 GMT
Location: http://ca.yahoo.com/?p=us
P3P: policyref="http://info.yahoo.com/w3c/p3p.xml", CP="CAO DSP COR CUR ADM DEV TAI PSA PSD IVAi IVDi CONi TELo OTPi OUR DELi SAMi OTRi UNRi PUBi IND PHY ONL UNI PUR FIN COM NAV INT DEM CNT STA POL HEA PRE LOC GOV"
Server: YTS/1.20.13
Set-Cookie: B=3o52fvt8vvth7&b=3&s=md; expires=Thu, 06-Aug-2015 18:59:51 GMT; path=/; domain=.yahoo.com
Set-Cookie: fpc=d=XsBGi89d2ejkYP5lI.jm4bLsP03sUgXuA9SXuzVmW5LjDNIlr2bZLo1.uVvXQDC2crEvSHsRTFLpv4VjhUe31RFioEX3I3i1XAyvAhdalVrOzw62Q3NCN1DSnan0Qo3G6JIlnO.Mtq1ixNPiKP6KuehAJZYiRZ409AoxUKQjaOA8.Q3DSCGE4tNSgqg4q2T8Gw9PoCc-&v=2; expires=Tue, 05-Aug-2014 18:59:51 GMT; path=/; domain=www.yahoo.com
Vary: Accept-Encoding
X-Frame-Options: SAMEORIGIN
Another practical things from httpie is the possibility to see the requests and responses headers without the body when doing an HTTP GET. It's useful because there are a lot of Web sites which send something quite different in between the GET and the HEAD. So when the HEAD is not enough
→ http HEAD http://ajw.asahi.com/
give it a GET without the body.
→ http --print Hh GET http://ajw.asahi.com/
I will certainly update this list of tips little by little.
Otsukare!