Setting up the LiteSpeed Cache crawler

Sometimes when you visit your site, you may notice that the initial load takes significantly longer than normal, even though you're already using a caching plugin.

The cache is designed in a way so that the first user that hits an un-cached page will generate a fresh copy that will then be served to all subsequent visitors. During quiet periods however, there may be situations where a page is only visited frequently enough that each user encounters an expired cache and has to generate a fresh copy when they load the page, increasing the load times.

To get around this, we can utilise the LiteSpeed crawler to hit each page and generate a cached copy and then re-crawl the site every few hours replacing the cache.

Setting up the LiteSpeed Crawler

Please Note: The LiteSpeed Crawler is only available on our Business hosting plans (Emerald and above).
  1. Log in to your WordPress Dashboard
  2. Navigate to the LiteSpeed Cache plugin

  1. Select the "Crawler" option

  1. Navigate to the "Sitemap Settings" tab and enter your sitemap URL.

Tip: This will normally be "https://yourdomainname.tld/wp-sitemap.xml", however, this can change depending on the plugins you're using:

All in one SEOhttps://yourdomainname.tld/sitemap.xml
RankMath SEOhttps://yourdomainname.tld/sitemap_index.xml

  1. Once this has been added, you can navigate to "General settings" and set the crawler to "ON".

  1. Then navigate to the "Map" tab and click the "Refresh Crawler Map" button.

  1. Lastly, head to the "Summary" tab and select "Manually Run" to start the first crawl straight away.

  1. Done!

The LiteSpeed crawler will initially do a first pass of every page on the site and ensure that this is cached. The first pass will normally take upwards of a few hours so you won't see the benefit immediately.

Once the first pass is complete, any further passes will only crawl pages that are un-cached or due to expire to ensure that every single page has a pre warmed cache to be served to human visitors.

How did we do?

Powered by HelpDocs (opens in a new tab)
© Krystal Hosting Ltd 2002–