Using libcurl: follow redirect and get header only

Let’s look at a sample code first(I use the WWW::Curl perl module here, the idea is the same for other language).

#!/usr/bin/perl
use warnings;
use strict;

use WWW::Curl::Easy;

my $url = 'http://search.cpan.org/CPAN/authors/id/L/LO/LORN/LWP-Curl-0.09.tar.gz';
my $resp_body;

#Get file length via HTTP HEAD  request
my $length;
my $curl = WWW::Curl::Easy->new();
$curl->setopt(CURLOPT_URL, $url);
#follow redirect
$curl->setopt(CURLOPT_FOLLOWLOCATION, 1);
#inlcude header in response
$curl->setopt(CURLOPT_HEADER, 1);
#do not include body in response
$curl->setopt(CURLOPT_NOBODY, 1);
$curl->setopt(CURLOPT_WRITEDATA,\$resp_body);
my $retcode = $curl->perform();
if($retcode == 0){
        print "header:$resp_body\n";
        print "*" x 80,"\n";
        $length = $curl->getinfo(CURLINFO_CONTENT_LENGTH_DOWNLOAD);
        if($length == -1 ){
                print "content length not available\n";
        }
        else {
                print "length: $length\n";
        }
}else{
        print "error happened:$retcode " . $curl->strerror($retcode) ." | "
                . $curl->errbuf ."\n";
        exit 1;
}

There’s nothing complicated, the points are:

  • CURLOPT_FOLLOWLOCATION: set to 1 to follow HTTP redirect response
  • CURLOPT_HEADER: set to 1 to include HTTP Header in response
  • CURLOPT_NOBODY: set to 1 to exclude body content from response, so that we get header only
  • CURLOPT_WRITEDATA: you get HTTP headers here, more about this later

You may think that we should use CURLOPT_WRITEHEADER to collect the HTTP headers, but it doesn’t work,we get only the first response header in this way, not the final headers after several redirection.

check http://curl.haxx.se/libcurl/ for more about libcurl

This entry was posted in Perl, Programming and tagged , , . Bookmark the permalink.

Leave a Reply