Google’s Insights On Robots.txt, Semantic HTML, & Rel-Canonical
![Google’s Insights On Robots.txt, Semantic HTML, & Rel-Canonical](https://altwhed.com/wp-content/uploads/2023/02/1675349825_Googles-Insights-On-Robotstxt-Semantic-HTML-Rel-Canonical-780x470.png)
Google Analyst Gary Illyes provides guidance on large robots.txt files, the SEO impact of website redesigns, and the correct use of rel-canonical tags.
Illyes takes questions sent to him via a LinkedIn direct message and answers them publicly, offering valuable insights to those in the SEO community.
It’s already newsworthy for a Googler to share his SEO tips. This is especially so considering Ellis, who isn’t as active on social media as fellow Search Advocate John Mueller and Developer Advocate Martin Splitt.
Over the past week, Illyes has shared tips and provided guidance on the following topics:
- Large robots.txt files
- The impact of search engine optimization on website redesigns
- Correct use of rel-canonical tags
Given the buzz his posts are getting, there will likely be more to come. Here’s a rundown of what you’re missing out on if you don’t follow him on LinkedIn.
Keep Robots.txt files under 500 KB
Regarding a previously published survey about the size of robots.txt files, the Illyes PSA shares for those with a file size greater than 500KB.
Illyes recommends paying attention to the size of your site’s robots.txt file, especially if it’s larger than 500KB.
Google’s crawlers only process the first 500KB of a file, so it’s crucial to make sure the most important information appears first.
Doing so can help ensure that Google crawls and indexes your website correctly.
Website redesigns can cause ratings to go ‘nuts’
When redesigning a website, it is important to remember that its search engine rankings may be affected.
As Illyes explains, this is because search engines use the HTML of your pages to understand and rank the content on your site.
If you make changes to the structure of the HTML, such as splitting paragraphs, using CSS styles instead of H-tags, or adding unnecessary break tags, this can cause HTML parsers to produce different results.
This can greatly affect your site’s rankings in search engines. Or, as Ellis puts it, it could cause ratings to get “crazy”:
![Google Insights on Robots.txt and Semantic HTML & # 038; Rel-Canonical](https://altwhed.com/wp-content/uploads/2023/02/1675349824_569_Googles-Insights-On-Robotstxt-Semantic-HTML-Rel-Canonical.png)
Illyes recommends using semantically similar HTML when redesigning your site and avoiding adding unnecessary tags to reduce your SEO impact.
This will allow HTML parsers to better understand your site’s content, which helps maintain search rankings.
Do not use relative paths in your relativity
Don’t take shortcuts when implementing rel-canonical tags. Illyes strongly recommends that you type the entire URL path:
![Google Insights on Robots.txt and Semantic HTML & # 038; Rel-Canonical](https://altwhed.com/wp-content/uploads/2023/02/1675349824_234_Googles-Insights-On-Robotstxt-Semantic-HTML-Rel-Canonical.png)
Saving a few bytes with a relative path in the rel-canonical tag is not worth the potential problems it can cause.
Using relative paths may cause search engines to treat it as a different URL, which can confuse search engines.
Clarifying the full URL path removes potential ambiguity and ensures that search engines identify the correct URL as the preferred version.
In summary
By answering questions sent to him via direct messages and offering his expertise, Illyes is giving back to the community and providing valuable insights into various SEO-related topics.
This is a testament to Illyes’ dedication to helping people understand how Google works. Send him a direct message, and your question might be answered in a future LinkedIn post.
source: linkedin
Featured image: SNEHIT PHOTO / Shutterstock