Enable or Disable URL Checking

HREFLang Builder has integrated a number of error checks into the process of building your HREFLang XML site maps that can enhance your experience and increase the number of validated URL’s in the XML site maps.

Since the tool must visit each URL imported this feature is set to OFF (Skip URL Checking) by default.

When to turn on error checking

There are two main reasons to enable error checking.

Extracting Unique Codes for Mapping URLs

If you have set Hreflang Builder to extract unique codes from your pages for mapping URLs, we must visit the URLs to extract them, so error checking MUST be enabled.

Valid URL Issues

You may consider using URL error checking if you are not confident of the quality of the URLs from your CMS XML sitemaps or if you have appended URLs that may become invalid in the future.  If you are importing your URLs from an SEO Diagnostic tool’s API solutions, there would be no need to use this feature. 

Enabling URL Checking

If you decide to use this feature, the following actions are necessary:

1. Before activating, you must add HREFLang Builder to your server or cloud host whitelist using these instructions to whitelist HREFLang Builder.
2. Understand that using this feature will do the requests as HREFLang Builder Bot and without the whitelisting may increase 403 errors blocking the bot from the site.
3. Notify your DevOps and Web Security Teams that you will be using Hreflang Builder to ping the URLs.  Please let us know if they require specific request limits and times of the day.

When to turn off error checking

If you no longer need error checking, simply check the box in the edit screen and we will not ping any of the URL’s on the site. 

Only turn off URL checking when you have confidence in your source files and/or you cannot whitelist our servers. Turning off error checking will make it possible to add URLs that have an error to the XML site map output. Excessive errors in your XML may result in Search Engines decreasing your crawl budget and/or the search engines ignoring your files, thereby eliminating the purpose of the HREFLang XML.

Note, if your URL source is XML site maps we will need to ping the site to request that source of URL’s. Some additional reasons to turn off error checking:

1. If you have complex bot security from an application like Cloud Flare and you have not added our IP to the whitelist
2. When setting up a new site and the URL’s are on a site on staging server
3. if you have a lot of errors that you are still working on them and don’t need them checked yet
4. If you are confident in the URL’s you are loading don’t have any errors.
5. The system will automatically turn off error checking if we get 20 or more header status codes of 403 Forbidden during any testing sessions.  We will notify your subscribing contact of this action via email.