It can’t create crawler sitemap again

  • Resolved babydu

    (@babydu)


    3 weeks, 1 day ago

    This is the code:SWNLSGMT

    I had test there code in the phpmyadmin:

    They always didn’t work.

    CREATE TABLE IF NOT EXISTS wp_litespeed_crawler ( id bigint(20) unsigned NOT NULL AUTO_INCREMENT,
    url varchar(1000) NOT NULL DEFAULT ”,
    res varchar(255) NOT NULL DEFAULT ” COMMENT ‘-=not crawl, H=hit, M=miss, B=blacklist’,
    reason text NOT NULL COMMENT ‘response code, comma separated’,
    mtime timestamp NOT NULL DEFAULT current_timestamp() ON UPDATE current_timestamp(),
    PRIMARY KEY (id),
    KEY url (url(191)),
    KEY res (res)
    )

    CREATE TABLE IF NOT EXISTS wp_litespeed_crawler_blacklist ( id bigint(20) unsigned NOT NULL AUTO_INCREMENT,
    url varchar(1000) NOT NULL DEFAULT ”,
    res varchar(255) NOT NULL DEFAULT ” COMMENT ‘-=Not Blacklist, B=blacklist’,
    reason text NOT NULL COMMENT ‘Reason for blacklist, comma separated’,
    mtime timestamp NOT NULL DEFAULT current_timestamp() ON UPDATE current_timestamp(),
    PRIMARY KEY (id),
    KEY url (url(191)),
    KEY res (res)
    )
    ENGINE=InnoDB DEFAULT CHARSET=utf8 ROW_FORMAT=DYNAMIC;

    The page I need help with: [log in to see the link]

Viewing 11 replies - 1 through 11 (of 11 total)
  • Plugin Support qtwrk

    (@qtwrk)

    what did it return when you run in phpmyadmin ?

    and what’s database version ?

    Thread Starter babydu

    (@babydu)

    At first, it returns nothing.

    But just now I test it again, and when I click the blank parts, it returns

    发生致命JavaScript错误。是否发送错误报告？

    { “pma_version”: “5.1.3”, “browser_name”: “FIREFOX”, “browser_version”: “128.0”, “user_os”: “Linux”, “server_software”: “LiteSpeed”, “user_agent_string”: “Mozilla/5.0 (X11; Linux x86_64; rv:128.0) Gecko/20100101 Firefox/128.0”, “locale”: “zh_CN”, “configuration_storage”: “disabled”, “php_version”: “8.0.30”, “script_name”: “index.php”, “exception_type”: “js”, “exception”: { “name”: “”, “message”: “Script error.”, “mode”: “onerror”, “stack”: [ { “line”: “0”, “column”: “0”, “func”: “?”, “context”: “”, “uri”: “index.php?”, “scriptname”: “index.php” } ], “uri”: “index.php?route=%2F” } }

    database version is

    服务器版本： 10.3.38-MariaDB-0ubuntu0.20.04.1 – Ubuntu 20.04

    Plugin Support qtwrk

    (@qtwrk)

    return nothing should mean it created tables successfully , did you see new table shows up ?

    Thread Starter babydu

    (@babydu)

    Yes,

    I checked in the dashboard, and in the SQL table.

    both of them are null.

    Plugin Support qtwrk

    (@qtwrk)

    if you see table exists , then please try crawl again , see what it returns

    Thread Starter babydu

    (@babydu)

    I had tried aggin and again. It’s still not worked.

    and return nothing.

    It sent a message:没有为爬虫解析出有效的网站地图。

    Plugin Support qtwrk

    (@qtwrk)

    as log says , the sitemap is not valid

    please share the sitemap link , let me verify a bit

    Thread Starter babydu

    (@babydu)

    I had checked, the sitemap is valid.

    It’s created by rank math.

    This is the Link:https://littleprince.site/sitemap_index.xml

    Plugin Support qtwrk

    (@qtwrk)

    <?php
    require( './wp-load.php' );

    $response = wp_remote_get( 'https://littleprince.site/sitemap_index.xml' );

    echo '<pre>';
    var_dump($response);
    echo '</pre>';

    please try this code , and see if wordpress can get it

    you have some kind of WAF it seems, I feel the firewall has blocked reuqest from wp

    Thread Starter babydu

    (@babydu)

    I tried this code in the code snippets plugin.

    It return a error and didn’t work.

    Then I turn off the CDN (there is a WAF too) and turn on. The crawler worked.

    Plugin Support qtwrk

    (@qtwrk)

    then it looks like your CDN is blocking the request from worpdress to fetch sitemap.

Viewing 11 replies - 1 through 11 (of 11 total)
  • You must be logged in to reply to this topic.