Understanding HTTP using Perl
What is HTTP?HTTP is a client-server protocol by which two machines can communicate over a tcp/ip connection. An HTTP server is a program that sits listening on a machine's port for HTTP requests. An HTTP client (we will be using the terms HTTP client and web client interchangeably) opens a tcp/ip connection to the server via a socket, transmits a request for a document, then waits for a reply from the server. Once the request-reply sequence is completed, the socket is closed. So the HTTP protocol is a transactional one. The lifetime of a connection corresponds to a single request-reply sequence. (a transaction) HTTP is the protocol used for document exchange in the World-Wide-Web. Everything that happens on the web, happens over HTTP transactions. TCP/IP networking and HTTP are the two essential components that make the web work. In order to write software that accesses the web (like a web browser, or a custom web client) you need a basic understanding of both. In this article we will cover HTTP, how it works and how to use it for simple transactions. We plan to include in this site some more articles which will cover basic network programming issues relating to TCP/IP and HTTP.
The client side: HTTP requestsSo basically what happens when we open a URL with the browser, is that the browser figures out from the url, what the HTTP server's host machine and port are, as well as the document path for the document we request from the server. For example, http://www.perlfect.com/articles/index.shtml suggests the document /articles/index.shtml on the server at www.perlfect.com and port 80. (no port is specified in the url, so the default, 80, is used) Subsequently, an HTTP request will be recited for that document and the appropriate connection via TCP/IP will be made with the server. Then the client (the browser, that is) will send the request, and wait for the server to respond with an HTTP response and, hopefully, the requested document. If all goes fine, the browser will arrange for displaying the document on our desktop window. (by rendering the HTML code into visual layout and making additional request for any images or other files that are embedded in the HTML document) Now, let's have a look under the hood to see what those HTTP requests lok like. Suppose you type the URL of the previous example, http://www.perlfect.com/articles/index.shtml on your netscape's location text box. Here's what the request will look like. (for the sake of clarity, the following request contains just as many headers as needed to demonstrate the HTTP request's general form and functionality - Netscape will surely make up a more complicated request, but the essential part of it are what is shown below)
The server side: HTTP responsesNow, looking on to the server's response:
Playing aroundAs mentioned earlier in our discussion, the examples shown here, while perfectly correct and working, are merely indicative of the HTTP protocol. The reader is encourged to play around and experiment with the HTTP requests and responses by real clients and servers. For example, if you do a simple telnet to the port 80 of a host with an active web server and type in a simple request like the example we gave, you can have fun watching the server's response come streaming live. Try non-existent documents, images, or whatever to see real examples of responses. On the other end check if your web server provides diagnostic facilities to let you inspect the incoming requests from web browsers. As with anything in computing, there's a lot to learn from such playing around.
Warning: mysql_connect() [function.mysql-connect]: Can't connect to MySQL server on 'mysql303.secureserver.net' (4) in /home/content/42/6167842/html/comments/comments_include.php on line 6
Connection Error: Can't connect to MySQL server on 'mysql303.secureserver.net' (4)
Like it? Share it!
Perl and LWP is an excellent book to get you started with using sockets and HTTP to write your own web clients in perl. It covers many issues relevant to web clients and while it does not go into much depth in some of them, by the time you have absorbed the techniques described in it, you will no longer need a book to walk you through more complex problems.
Advanced Perl Programming among various other very interesting subjects, dedicates a chapter to socket programming, not in the context of web clients, but still in a very clear and to-the-point manner. It is also a good book to have if you're seriously interested about perl programming, in my opinion.