UserAgent and Referer Blank When Plugin Downloads a File

  • Resolved golddave

    (@golddave)


    18 years, 2 months ago

    Another question related to my XML parsing plugin. The plugin copies an XML file from a remote server to my server every hour and then parses it from the local copy. I’ve been in close contact with the owner of the site I’m getting the XML from (I want to be sure he approves of whatever I’m doing with his work) he says that the UserAgent and Referer fields in his logs are coming in as blanks when I download the XML. Here’s a sample (see the last two fields with the values “-“):
    xxx.xxx.xxx.xxx [11/Sep/2006:00:00:34 -0700] “GET /mlb/standings.xml HTTP/1.0” 200 51482 “-” “-“

    He’d like, if possible, for me to put something in those fields. How would I do that?
    I’m using PHPs copy command to download the file. I’m open to other methods if it will help the site owner get what he wants for UserAgent and Referer.

    Thanks.

Viewing 2 replies - 1 through 2 (of 2 total)

  • I noticed you marked this [Resolved], but, can you let everyone else know what you did to “Resolve” it? This might help others that are having the same problem as you had.. =) ??

    spencerp

    Thread Starter golddave

    (@golddave)

    Sorry it’s taken me so long to get back to this. I don’t normally go back to my resolved issues but accidentally clicked and saw a comment. So here’s my solution.

    I used CURL. CURL is a PHP library that “allows you to connect and communicate to many different types of servers with many different types of protocols”. With CURL I wrote the function below.

    function download($sourcefile, $filename) {
    $useragent=”user agent string”;
    $refferer=”refferer string”;
    // create a new curl resource
    $ch = curl_init();

    // set URL and other appropriate options
    curl_setopt($ch, CURLOPT_URL, $sourcefile);
    curl_setopt($ch, CURLOPT_USERAGENT, $useragent);
    curl_setopt($ch, CURLOPT_REFERER, $refferer);
    curl_setopt($ch, CURLOPT_HEADER, false);
    curl_setopt($ch, CURLOPT_BINARYTRANSFER, true);
    curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);

    set_time_limit(300); # 5 minutes for PHP
    curl_setopt($ch, CURLOPT_TIMEOUT, 300); # and also for CURL

    $outfile = fopen($filename, ‘wb’);
    curl_setopt($ch, CURLOPT_FILE, $outfile);

    // grab file from URL
    curl_exec($ch);
    fclose($outfile);

    // close CURL resource, and free up system resources
    curl_close($ch);
    sleep(10);
    return true;
    }

Viewing 2 replies - 1 through 2 (of 2 total)
  • The topic ‘UserAgent and Referer Blank When Plugin Downloads a File’ is closed to new replies.