Disastrous wrong turn?

  • I asked a while back how WordPress pages are indexed considering that they are only created on the fly (i.e., are not static HTML files sitting out there) and was advised (by esmi) that crawlers “run” (my word) a site, creating pages on the fly as would any visitor.

    But is this a problem?: in order to control layout placement both within the front page and any interior page, we pull content using our own SQL. Its different versions sit, for any crawler to read, out in the open, although in functions.php, called from the likes of home.php, etc.

    Our worry is whether crawlers make assumptions about how WordPress is constructed, therefore never find custom SQL, therefore don’t create our pages and therefore fail to index our site, in which case we’ve created a mess. Does anyone have any knowledge of this?

Viewing 5 replies - 1 through 5 (of 5 total)

  • It seems to me that they would be able to crawl anything that you can see in the page source when you view the page.

    Actually there may be one consideration. I know that in the “old days” of Google and SEO, it *used to be* (may not still be) that Google would not crawl any URLs with more than two wildcard characters in them, like the “&” in:

    whatever.com/&store_name=MyStore&category=stuffedanimals&item=bear

    So, if you have that going on it may be something to consider. A long time ago I was the genius who figured out how to have shorter URLs in Miva Merchant to avoid this SEO problem. It doesn’t seem to be a problem with WordPress because wildcard characters are not used in the URLs.

    there are endless variations of dynamic webpages and websites out there (for instance all mail order sites), and they all get crawled and indexed and found by the search engines.
    crawlers have as little access to the sql and database as any other visitor – they get the rendered end result html page presented.

    imho, crawlers are too busy to make assumptions.

    if you like, ask these questions at the major search engine help forums – these guys are probably better prepared to give you a satisfying answer…

    Thread Starter stevewilson

    (@stevewilson)

    Thanks to you both. Sounds like we’re safe. No “&”s used. But I’ll certainly take up your suggestion to ask again at engine forums.

    Thread Starter stevewilson

    (@stevewilson)

    You wouldn’t happen to have a URL for such a forum would you? For Google, Bing, whatever? I went looking and kept falling into SEO sites and others pretending to be Google. Just a standard help-each-other forum or two like this one?

Viewing 5 replies - 1 through 5 (of 5 total)
  • The topic ‘Disastrous wrong turn?’ is closed to new replies.