• There are several high quality WPHacks which allow you to display one or more RSS feeds on your blog. However what I wanted to was to use WP as an RSS aggregator, periodically polling a number of RSS feeds, and inserting each new item as a new WP post. There might be an existing implementation of this idea, but I didn’t find it, so I whipped up something quick this morning.
    Features:

    • tries to be smart about mapping DublinCore fields to WP fields
    • supports automatically categorizing new posts based on their originating feed
    • uses postmeta to keep track of previously seen posts

    Limitations:

    • Ignores mod_content, only works with RSS 1.0
    • Only supports inserts, not updates
    • Built against nightly build 2004-7-14, ymmv

    Download at: https://laughingmeme.org/code/wp-rss-aggregate.php.txt
    This is just a quick hack, but I thought I’d share. Among other things, there might be a couple of functions in there that would be useful to other hack authors.

Viewing 15 replies - 1 through 15 (of 21 total)
  • That would be a neat hack for WP…if one doesn’t exist already. Drupal has some of the infrastructure already in place for parts of what you’d like to do. It has a built in aggregator and php cron file…and it lists all the news items with a special link icon to “blog it”…thus filling out most of the form with what you need to start a post–including the quoted excerpt. It would be very cool to see something like this built into WP.

    Thread Starter kellan

    (@kellan)

    Hi Alan,
    The Drupal aggregator sounds more like a “reBlogging” system, where you’re picking and choosing from a number of incomimg feeds.
    Removing the manual intervention was definitely key to what I was wanting to do.
    Thanks for the feedback.

    I had thought about something like this as an extension to CG-FeedRead (or part of the PowerPack), but wasn’t sure there would be enough users to make the time investment working. At the moment, the CG-Amazon stuff is keeping me busy, as is my site, but I’ll keep this idea on the backburner, see if I can find an interesting way to do this.
    -d

    Kellan,
    Nice tool. Do you have a howto for setting this up?
    -Vohiyaar

    Thread Starter kellan

    (@kellan)

    Hi Vohiyaar,
    Here is a quick howto. The code leverages the WP libraries, but is essentially a stand alone application in that it isn’t designed to be used as a WP hack, or WP plugin.
    While you don’t have to place wp-rss-aggregate.php in your WordPress directory (and I don’t) that is probably the simplest way to set it up (or you could place in in wp-admin)
    Open up wp-rss-aggregate.php in a text editor and you’ll see a define, and 4 require_once’s at the top of the file. Edit the define and first two requires to point at where you have MagpieRSS installed on your system. Edit the second to requires for where you have WordPress installed. (e.g. if you place wp-rss-aggregate.php in your WP folder you’d want the require to read: require_once ‘./wp-config.php’;)
    Now run it from your web browser, make sure it works, (no error messages), and that should aggregate any feeds you’ve configured in the feeds array. (e.g. https://example.org/wordpress/wp-rss-aggregate.php)
    Presumably you want the aggregation to happen automatically, so the next step is to addd a cron job that runs periodically requesting this page. To do this you’ll need cron, understanding of crontabs, and a command line web browser (lynx, links, curl, wget, GET would all work). FeedOnFeeds has an example of this.

    Hello! I tried using the wp-rss-aggregate-script, but I keep getting SQL errors. The table name is missing from all the statements, so I guess the $wpdb->users calls are not working (does not generate any output). I know very little PHP, can anyone point me to what I am doing wrong?

    Thread Starter kellan

    (@kellan)

    manne, you have to use WP 1.3, grab a nightly build

    That did it! Thanks a bunch. ??

    I love this script. I use it now to get my links from de.icio.us and republish them in a separate wordpress blog. Works great. I changed the script slightly as so to hack up all the space separated tags from del.icio.us using explode() on the dc:subject-tag before they are inserted as categories in WordPress, works like a charm.
    One question though: I just tried to add a link to del.icio.us and then run the script again. Expected result would be that one new linkpost would be published, but nothing happened. A total of zero posts were added. How come?

    Thread Starter kellan

    (@kellan)

    Could be that Magpie still has a cached version of the del.icio.us RSS feed, and therefore didn’t re-fetch. Default caching time is 1 hour.
    Ditto I wouldn’t be surprised is Joshua has a lag between when you add an item to del.icio.us and when they show up in your RSS feed.

    klintron

    (@klintron)

    Could anyone tell me what to change in this script to keep it from adding new categories based on del.icio.us tags? I’d like to just dump all my del.icio.us bookmarks into one category.

    klintron

    (@klintron)

    A more major problem: how do I get the del.icio.us links to actually work? They end up in a “custom field” but not in the actual posts anywhere. What’s up with that? My blog: https://www.klintron.com/brain

    Anyone know who the most recent “Anonymous – Unregistered” poster is? This post!

    I want to know how this all works for him/her. If there is any action on the forum anywhere else please post. I’ve got this thread RSS’d.

    Is there any further documentation for this script – particularly explanations of wp-agg.ini (how to use/config) and the exact place to add the various RSS feeds to pull in as posts in the blog.

    Thank you.

    I tried this script, but after a few days I gave up. I can’t figure it out. Can’t find the wp-agg.ini file anywhere. Will give it another try in the next week or so.

Viewing 15 replies - 1 through 15 (of 21 total)
  • The topic ‘RSS Aggregator’ is closed to new replies.