Perlfect Solutions

Writing a web client for MP3 streaming in perl.

What is streaming?

By streaming we refer to the decoding of a file consurrently with its download, i.e. rendering the contents of the file as they become available, without having to stand by for the entire file to be downloaded. Streaming is very useful in applications that involve live content (such as readio transmissions, or newscasts) or media files that tend to be very large, that is in applications where the user cannot afford waiting for the download to be completed before the file can be decoded. Not all file formats can be streamed. Many common media fromats/compressions that we use everyday, such as GIF or WAV require the entire file to be available before the decoding can take place. Formats such as mpeg-3 or RealAudio on the other hand, can be decoded as the data comes in. Such formats are called collectively streaming formats.

Taking advantage of the streaming properties of a file.

If you start downloading an mp3 file and stop the download before it's finshed, you'll then be able to hear the song that you have downloaded perfectly ok, up to the point where the transfer was interrupted. Going a step further, if you start the download and also ask your mp3 player to play the file which is being written to right now, you'll notice that the decoder plays the file just fine, and if your network connection can maintain a sufficient throughput, you could listen to the entire song as it's being downloaded without a single jitter. Because network connections rarely are too stable, you will find that it is good practice to let the download do some progress first and then start the decoder, so that even if your network throughput loses pace for a while, there will be some data read ahead for the decoder to munch as the network catches up.. otherwise you might experience jerks and pauses. This is a common technique, similar to the lookahead buffer that car stero cd's use to ensure sound continuity even when you hit a bump on the road.

Automating the process...

While it was fun to experiment with streaming manually, it would be even better fun if we made a program to do all of it for us: download the file from the internet and redirect the incoming data stream to the decoder. Ideally, we would just provide the url of the remote mp3 file and soon we would be listening to the song hot off the network. All we need is to make a program that can open a socket to a remote host, use HTTP to request the file from the server, and than just feed the incoming data from the socket into the standard input of the decoder/player to which we will open an input pipe.

Introducing webamp...

And here's our program, webamp. Have a look at the code, which is pretty basic, and see for yourself how it works.. I will be explaining the details shortly.

0 #!/usr/bin/perl 1 use Socket; 2 3 my $handler = 'splay'; 4 5 my $url = shift @ARGV; 6 $url=~m/http\:\/\/([^\:^\/]*)(?:\:(\d+))?\/(.*)/; 7 my $host = $1; 8 my $port = $2; 9 $port = 80 unless($port); 10 my $file = '/'.$3; 11 12 my $proto = getprotobyname('tcp'); 13 socket(SOCK, PF_INET, SOCK_STREAM, $proto); 14 print "Looking up $host..\n"; 15 my $sin = sockaddr_in($port, inet_aton($host)); 16 print "Connecting to $host:$port..\n"; 17 connect(SOCK, $sin) || die "Connect failed: $!\n"; 18 19 my $old_fh = select(SOCK); 20 $|=1; 21 select($old_fh); 22 23 print "Requesting $file..\n"; 24 print SOCK "GET $file HTTP/1.0\n"; 25 print SOCK "Accept: */*\n"; 26 print SOCK "User-Agent: webamp $version\n\n"; 27 print "Waiting for reply..\n"; 28 my $header = <SOCK>; 29 print "$header\n"; 30 exit unless($header=~m/200|OK/); 31 32 while($header = <SOCK>) { 33 chomp; 34 last unless(m/\S/); 35 } 36 37 my $content; 38 open(HANDLER, "|$handler") or die "Cannot pipe input to $handler: $!\n"; 39 print "Redirecting HTTP filestream to $handler..\n"; 40 while(read(SOCK, $content, 512)) 41 { 42 print HANDLER $content; 43 } 44 close SOCK;

How it works.

Line 3

This is just the name of the program that we will use as a decoder. It is important to use a decoder that is capable of accepting standard input as we are going to be piping the data stream to it.

Lines 5-10

Here we parse the URL that was passed as an argument from the command line. The URL consists of three components of interest to us. The hostname, the port and the filepath relatiev to the server's root. A simple regular expression (line 6) helps us identify each and store them in variables $host, $port and $file. Note that if no port is specified we assume port 80, the default HTTP server port.

Lines 12-17

Here we create a socket and try to connect to the remote server/port. If all goes well, we wnd up with a filhandle SOCK bound to the socket. We will be using this filhandle to communicate with the remote program. (the web server in that case) Printing to the filhandle sends data and reading from the filhandle blocks waiting to receive until incoming data is available to satisfy the read request.

Lines 19-21

In order for this to work we must turn off the buffering of the stream. Line 20 turns off buffering of the selected stream. We first select (in line 19) the socket filhandle, then switch off the bufferring, and then restore the previously selected filhandle (good housekeeping rules dictate leaving things as you found them, unless there's good reason for not doing so) which we had saved on our first select().

Lines 23-30

Now that our communication channel is set up, we may send our HTTP request (lines 24-26) to the remote web server. Then we must wait for the server to respond. We read the first line from the response which contains the status code and message. If it's 200 OK, this means that our request can be satisfied and that headers and content follow... otherwise we abort. (This is a far too simplistic approach at interpreting server responses, but our main concern here is to provide a demonstration rather than a sophisticated web client that can deal with redirects, caching etc)

Lines 32-35

Here we just loop over the headers ignoring them to get to the actual content. Again this is probably not an appropriate tactic for a serious web client, but we just want to provide a working demonstration at the moment.

Lines 37-44

Finally we ger to the fun part. In line 38 we open a pipe to the inpu of the handler program, and right after that, we start reading through the incoming data stream and redirecting all data to the handler which will decode and play the music.

Further Improvement

There is a lot of room for improvement in this program. First of all one desirable feature would be to also store the incoming data into a file as well as passing it along to the handler. So if at the end of the day we like the song, we have it stored somewhere to play it again later. Another obsious enhancement would be to change the last part of the program to buffer the incoming stream and keep a lookahead of a few Kbytes so that we reduce the chance of jitters in the decoding. Finally, note that this is a generic streamer application after all.. we can use any program as a handler, so with appropriate handler programs for various formats we could implement streaming of any other file type. We could even provide the ability to choose the handler from the command line, so the same program can be applied to all streaming uses we wish.

Online Documentation/Tutorials

Comments

Your name:
Your comments:

Security check *

 

Mike   

Posted at 10:12pm on Saturday, April 28th, 2007

Nice simple tutorial.

Submitted in queue @ tweako

( http://www.tweako.com )

Pete   

Posted at 9:08am on Friday, May 23rd, 2008

Yup, very nice, simple example. Thanks!

Md.Ruhul   

Posted at 6:54am on Friday, February 6th, 2009

My Web Site mp3 Data Input Help

Shaq   

Posted at 12:05am on Monday, May 9th, 2011

How about the scripts for client?

Keylogger   

Posted at 4:57am on Wednesday, May 18th, 2011

Thanks for describing this concept in detail. I would like to look forward for another informative post from you.

Download Keylogger
www.keylogger.in

Backlink Checker   

Posted at 2:20am on Tuesday, August 23rd, 2011

Thanks for the nice example information.

Backlink Checker
http://www.backlinkschecker.ws

data recovery software   

Posted at 12:29am on Wednesday, September 21st, 2011

Your blog is nice. Thanks for providing us this information.

data recovery
http://www.sanmaxi.org

Zip Repair   

Posted at 1:53am on Tuesday, October 4th, 2011

This is amazing zip recovery software.

Zip Recovery   

Posted at 9:23pm on Monday, October 10th, 2011

I am impressed your suggestion.

Repair Zip

http://www.zip-repair.org

DataRecovery   

Posted at 5:02am on Wednesday, October 12th, 2011

data recovery software

This is a great inspiring article.

Recover NTFS Files
http://www.datarecoverytrial.com

Download Free Keylogger Software   

Posted at 12:27am on Thursday, March 22nd, 2012

Thanks to you so much for sharing this wonderful piece of work with us.

Keylogger
Keylogger

keylogger download
http://www.keystrokecapture.ws

Marko   

Posted at 6:42am on Tuesday, August 21st, 2012

2011 年 11 月 12 日 - 下午 2:43 6F United Kingdom Internet Explorer 8.0 Windows 7I used to be recommended this web site by means of my ciousn. I am now not certain whether or not this publish is written by means of him as no one else understand such distinctive approximately my trouble. You're incredible! Thank you!

zyccdglkbh   

Posted at 9:34pm on Tuesday, August 21st, 2012

Ax9HiU ctgadynpkipp

gjnzlufz   

Posted at 4:05am on Friday, August 24th, 2012

rvw157 wpdwhzqjgitd

Can Le   

Posted at 11:45am on Wednesday, November 13th, 2013

Sir,

I tested your script with error below after OK on first part.
Please show me how to correct problem on Redirecting from line 39.

LC
HTTP/1.1 200 OK

'splay' is not recognized as an interna
operable program or batch file.
Redirecting HTTP filestream to splay..

zewprhyxul   

Posted at 1:54pm on Monday, February 10th, 2014

alqbiqfsmgfdu, http://www.gcdayegysx.com/ wpzlfgleap

Comments to date: 16.

Like it? Share it!

Suggested Reading

Order your copy of Perl and LWP now! Perl and LWP is an excellent book to get you started with using sockets and HTTP to write your own web clients in perl. It covers many issues relevant to web clients and while it does not go into much depth in some of them, by the time you have absorbed the techniques described in it, you will no longer need a book to walk you through more complex problems.
Order your copy of Advanced Perl Programming now! Advanced Perl Programming among various other very interesting subjects, dedicates a chapter to socket programming, not in the context of web clients, but still in a very clear and to-the-point manner. It is also a good book to have if you're seriously interested about perl programming, in my opinion.