Step 1: Make sure your content is visible to search engines.
Right click on your page and view source to see how search engines read your content. Once there, use CTRL + F to see if your headings, body content, and metadata is crawlable.
First of all, make sure that your page has the robots meta tag that tells search engines to follow
Remember that the text following a <h1 or <h2 tag should be based on relevant keywords and isn’t too long. Make sure that the text in your navigation bar doesn’t use heading tags (anything <h )
To read your content the way search engines read it, look for the following tags in your source.
<h1
<h2
<p
<meta name=”description”
Social media platforms will also crawl your site to generate social media cards using open graph meta data when a page is shared. Make sure that the following tags are on your content.
“og:url”
“og:title”
“og:description”
“twitter:description”
“Og:image”
“twitter:card”
“twitter:title”
“twitter:image”
Step 2: Check your robots.txt file
Navigate to [yourdomain.com]/robots.txt and make sure the content is not blocked. If it is blocked the page will read as follows.
User-agent: *
Disallow: /
You should also make sure that your sitemaps is listed on the page. See the example below.
Every website’s robots.txt file will look slightly different, but here is an example of one that allows search engines to crawl the right content.
User-agent: *
Disallow: /facebook/
Disallow: /languageselectorpartial/
Disallow: /twitter/
Disallow: /en/search/
Disallow: /es/search/
Disallow: /fr/search/
Disallow: /ar/search/
Sitemap: https://www.amnesty.org/sitemap.xml
Step 3: Set up important redirects
Make sure that page with “https://” in the URL is redirected to a page with “https://.” The same should apply to URLs without “www” in the address.
Make sure HTTP redirects to HTTPS, ie
https://www.amnesty.org/en/ to https://www.amnesty.org/en/
Make sure www redirects to non www or vice versa, ie
https://www.amnesty.org/en/ to or from https://amnesty.org/en/
Step 4: Check your href lang metadata.
It is important for search engines to be able to quickly identify your site’s language. This can be done using the href lang tag. View your page source (right click on the page and click view source) and CTRL + F “hreflang”. If your site is available in multiple languages, the various languages should be listed here.
Example:
<link rel=”alternate” hreflang=”en” href=”https://www.amnesty.org/en/” />
<link rel=”alternate” hreflang=”fr” href=”https://www.amnesty.org/fr/” />
<link rel=”alternate” hreflang=”es” href=”https://www.amnesty.org/es/” />
<link rel=”alternate” hreflang=”ar” href=”https://www.amnesty.org/ar/” />
Step 5: Check that your pages return the right status codes
You can use this using the tool below.
Your good pages should return a 200
Some pages or files may return a 301 or 302 which is a redirect
Broken or nonexistent pages will return 400s or 500s (which they should)
For example:
https://www.amnesty.org/en/ should be 200
https://www.amnesty.org/ should be a 302
https://www.amnesty.org/en/not-a-page-dumbo should return a 404
Step 6: Check your page speed
The faster your content loads, the easier it is for search engines to crawl it. If your page speed is greater than 2 seconds, it would be a good idea to investigate which elements are slowing it down.
It is advised to regularly check the page speed of your homepage to evaluate how quickly your site is loading more generally.
Page speed
https://developers.google.com/speed/pagespeed/insights/
Step 7: Set up canonical links
Make sure that search engines understand how your different pages relate to one another. For example, if you have two versions of the same page, you will want your website to automatically set up a canonical link between the two. Doing this will prioritize crawling one version of the page over the other.
https://www.amnesty.org/en/?=not-a-real-page
Should have a canonical of <link rel=”canonical” href=”https://www.amnesty.org/en/”/>
<link rel=”next” href=”https://www.amnesty.org/en/search/?q=london&p=2“/>
Step 8: Make sure your CMS allows you to add important metadata
As a user uploading content to your website, you should be able to enter specific metadata that informs search engines that your content is relevant to certain search queries. For example, adding alt text (metadata that describes what’s happening in the image) to images is an important way to tell search engines what your page is about.
The same should apply to Open Graph and Twitter page meta data, which tells social media platforms how to visualize your page when someone shares a link. This text should be different for every page on your site.
Open Graph metadata should look something like this.
The Page Title and meta description tags tell search engines what to show on search engine result pages when your page is listed.