Understanding HTTP using Perl
What is HTTP?
HTTP is a client-server protocol by which two machines can communicate over a tcp/ip
connection. An HTTP server is a program that sits listening on a machine's port for HTTP
requests. An HTTP client (we will be using the terms HTTP client and web client interchangeably)
opens a tcp/ip connection to the server via a socket, transmits a request for a document, then
waits for a reply from the server. Once the request-reply sequence is completed, the socket is
closed. So the HTTP protocol is a transactional one. The lifetime of a connection corresponds to
a single request-reply sequence. (a transaction)
HTTP is the protocol used for document exchange in the World-Wide-Web. Everything that
happens on the web, happens over HTTP transactions. TCP/IP networking and HTTP are the two
essential components that make the web work. In order to write software that accesses the web
(like a web browser, or a custom web client) you need a basic understanding of both. In this
article we will cover HTTP, how it works and how to use it for simple transactions. We plan to
include in this site some more articles which will cover basic network programming issues relating
to TCP/IP and HTTP.
The client side: HTTP requests
So basically what happens when we open a URL with the browser, is that the browser figures out
from the url, what the HTTP server's host machine and port are, as well as the document path for
the document we request from the server. For example,
http://www.perlfect.com/articles/index.shtml suggests the document /articles/index.shtml on the
server at www.perlfect.com and port 80. (no port is specified in the url, so the default, 80, is used)
Subsequently, an HTTP request will be recited for that document and the appropriate connection
via TCP/IP will be made with the server. Then the client (the browser, that is) will send the
request, and wait for the server to respond with an HTTP response and, hopefully, the requested
document. If all goes fine, the browser will arrange for displaying the document on our desktop
window. (by rendering the HTML code into visual layout and making additional request for any
images or other files that are embedded in the HTML document)
Now, let's have a look under the hood to see what those HTTP requests lok like. Suppose you
type the URL of the previous example, http://www.perlfect.com/articles/index.shtml on your
netscape's location text box. Here's what the request will look like. (for the sake of clarity, the
following request contains just as many headers as needed to demonstrate the HTTP request's
general form and functionality - Netscape will surely make up a more complicated request, but
the essential part of it are what is shown below)
GET /articles/index.shtml HTTP/1.0
User-Agent: Mozilla 4.0 (X; I; Linux-2.0.35i586)
Accept: image/gif, image/jpeg, */*
The first line contains three important pieces of information: The request method (GET), the
requested document (/articles/index.shtml) and the HTTP protocol version that the client uses.
(1.0) You might wonder what the request method is, but you really don't need to be worried about
it at this point. There are a few different request methods the omst common ones being:
There are others, too that are much less frequently used, and we won't discuss them. The general
structure of a request applies to all methods, so we will stick to GET for now, to demonstrate how
request work in general.
Following that, there are a number of lines called request headers. They are all of the form:
Header-name: Header Value and they specify information and parameters that will help the
server provide a suitable response. In this example the parameters indicate the client software
name and version, the server hostname for which the request is meant (this is because
sometimes, a single HTTP server might serve documents under different names, and each name
corresponds to a different directory tree - so the server needs to be told what name to look up
the document for) and the MIME types that the client is willing to accept.
- GET asks to retrieve a document
- POST passes form data to the server for use as input to some CGI program
- HEAD asks to retrieve only the HTTP response header for a document but not the
The server side: HTTP responses
Now, looking on to the server's response:
HTTP/1.0 200 OK
Date: Thus, 08 Oct 1998 16:17:52 GMT
Last-modified: Mon, 05 Oct 1998 01:23:50 GMT
The first line contains the version of HTTP used in the response, and the response status in both
numerical code (200) and human-readable string (OK). There are a number of such resonse
codes. To give two common examples : 200 OK means that the document has been found and
that it follows the response headers and 404 NOT FOUND means that the document path does
Similarly to request headers, we also have response headers, which are used to pass information
about the document in transit and the status of the server and the request. In the example above
the headers provide information about the server software and version, the date and time the
response was issued and finally the MIME type, length and last modification date of the document
A blank line marks the end of the head of the resonse, and then the document follows. After the
browser's finished receiving the HTML document in question, and the TCP/IP connection has
been dropped, it will go on to request any additional embedded documents (in-line images for
example) and render the page's layout on screen. Clicking on a link will cause the browser to
issue a new request for the page pointed to by the link, and so on.
As mentioned earlier in our discussion, the examples shown here, while perfectly correct and
working, are merely indicative of the HTTP protocol. The reader is encourged to play around and
experiment with the HTTP requests and responses by real clients and servers. For example, if
you do a simple telnet to the port 80 of a host with an active web server and type in a simple
request like the example we gave, you can have fun watching the server's response come
streaming live. Try non-existent documents, images, or whatever to see real examples of
responses. On the other end check if your web server provides diagnostic facilities to let you
inspect the incoming requests from web browsers. As with anything in computing, there's a lot to
learn from such playing around.
Posted at 10:26pm on Sunday, March 4th, 2007
A good document for starters ...
Devendra Singh Rathore
Posted at 7:58pm on Sunday, April 15th, 2007
Posted at 11:29pm on Saturday, May 5th, 2007
Posted at 3:03pm on Wednesday, May 9th, 2007
Hope this helps me.
Posted at 4:40am on Tuesday, May 29th, 2007
Posted at 5:30am on Wednesday, June 27th, 2007
That information helped me finish the last details on my http based log server! Thanx!
Posted at 3:05am on Monday, October 15th, 2007
really nice .need some more detailed information .
Posted at 1:41pm on Monday, January 21st, 2008
Not really a great beginners start! will use a beginners guide i think as above seems an alien language!!!
Posted at 2:44pm on Saturday, May 3rd, 2008
protip: its you thats broke not the guide.
Posted at 11:52pm on Sunday, June 1st, 2008
IT'S Too Gud
Posted at 3:25pm on Sunday, August 17th, 2008
Posted at 6:22am on Friday, October 3rd, 2008
Really a very good article for the beginners to understand about HTTP Request and response.
Posted at 12:48pm on Monday, November 17th, 2008
A TCP socket has client IP, client TCP port, server IP, and server TCP port. Q: what is the client TCP port in a typical web page download?
Posted at 3:13am on Tuesday, March 17th, 2009
thanks its working
Posted at 3:08am on Friday, April 10th, 2009
Audie C. Gates
Posted at 4:27pm on Sunday, April 19th, 2009
What is it if something was mailed to you & you received it, read it, and the next day or so,you go to look at it again, and you get the "404 error" instead!? Isn't that infringment? Or a violation of my privacy?
Posted at 12:51am on Sunday, September 13th, 2009
the document is very informative..thanks to the author
Posted at 5:03pm on Friday, November 27th, 2009
When I visited this site on 2009.11.28, the second link under
"Online Documentation tutorials" had an incorrect URL. The correct URL ended "streaming.shtml" instead of "http.shtml"
Except for that error, this page is a good, brief introduction.
Posted at 12:14pm on Saturday, January 23rd, 2010
Nice but need some more details.........
Posted at 12:14pm on Saturday, January 23rd, 2010
Nice but need some more details.........
Posted at 5:47am on Monday, April 12th, 2010
Good. But if you could have given one example it will be appreciated more.
Posted at 12:39pm on Tuesday, January 25th, 2011
nice work , i like it
Posted at 9:00pm on Monday, June 27th, 2011
YMMD with that anewsr! TX
Posted at 1:14am on Tuesday, June 28th, 2011
Posted at 1:51am on Friday, July 1st, 2011
Posted at 2:48pm on Saturday, October 8th, 2011
Don't have enough money to buy a car? Worry not, because that's achievable to get the home loans to resolve such kind of problems. Thence get a college loan to buy everything you need.
Posted at 2:25pm on Wednesday, February 15th, 2012
Posted at 2:09am on Tuesday, August 21st, 2012
If you do what? Your website is a honitsg company for Minecraft servers If those slow down, then my internet is better than yours . And connections don't slow down over time. Also they don't get faster after a week. If a server puts a strain on a network it's instant or a matter of minutes/hours So what are you trying to sell?
Posted at 9:15pm on Tuesday, August 21st, 2012
Posted at 3:49am on Friday, August 24th, 2012
Posted at 12:28am on Sunday, April 7th, 2013
Продаю реганные аккаунты: Vk.com SMS Photo , Qip.ru (Pochta.ru) , 000Webhost.com , Livejournal.com , Facebook.com EN , Hotmail Second Hand , Редиректы 0Webhost , Mail.ru Second Hand . и многое другое в наличии и на заказ.
- очень большой ассортимент + различные аккаунты активированные по смс
По всем вопросам пишите сразу в аську ICQ 231538
продам аккаунт одноклассники , а%2
Posted at 8:08am on Monday, September 2nd, 2013
Comments to date: 32.
Perl and LWP
is an excellent book to get you started with using sockets
and HTTP to write your own web clients in perl. It covers many issues relevant to web clients and while it does
not go into much depth in some of them, by the time you have absorbed the techniques described in it, you will
no longer need a book to walk you through more complex problems.
Advanced Perl Programming
among various other very interesting subjects, dedicates a chapter to socket
programming, not in the context of web clients, but still in a very clear and to-the-point manner. It is also a good book
to have if you're seriously interested about perl programming, in my opinion.