Screaming Frog wasted no time in integrating Google’s new URL Inspection API which allows access to existing indexing data. The new API allows Screaming Frog to include seven brand new filters that identify website issues that need attention.
In addition to the expanded usefulness of Screaming Frog, the new update, code named Romeo, also patches the log4j vulnerability, adds new keyboard shortcuts and fixes some bugs to make it more stable.
Seven other ways to fix websites
Google’s new URL Inspection API allows software tools like Screaming Frog to access Google Search Console and import data into the tool.
API stands for Application Programming Interface. It acts as a bridge between two programs allowing both of them to communicate to share data and achieve some level of integration between the two programs.
Screaming Frog’s URL Inspection API integration allows popular site auditing software to provide the following new troubleshooting filters.
Seven new audit filters:
- “The URL is not on Google
The URL is not indexed by Google and will not appear in search results. This filter can include non-indexable URLs (such as those with “noindex”) as well as indexable URLs that can be indexed. It’s a catch-all filter for anything that isn’t on Google as per the API.
- Indexable URL is not indexed
Crawlable indexable URLs are found that are not indexed by Google and will not appear in search results. This can include URLs that are unknown to Google, those that are detected but not indexed, and more.
- The URL is on Google, but it has issues
The URL is indexed and can appear in Google search results, but there are some issues with mobile usability, AMP pages, or rich results that may mean it doesn’t display optimally.
- Canonical authorized by the user is not specified
Google chose to index a different URL than the one declared by the user in the HTML.
- The page is not mobile friendly
The page has issues on mobile devices.
- Invalid AMP URL
AMP contains an error that prevents it from being indexed.
- Invalid formatted result
The URL contains an error in one or more rich result optimizations that will prevent the rich result from appearing in Google search results.
The value of the URL Inspection API’s integration is that you can now overlay Search Console indexing data on Screaming Frog’s crawl data, allowing you to see all data points on a page extracted by Screaming Frog along with their indexing status.
Screaming Frog now gains a Rich Results Inspection tool that allows bulk export of data from the Bulk Export tab in the top menu.
Additional updates
Other updates, in addition to patching the aforementioned log4j vulnerabilities, are adding keyboard shortcuts (ctrl+W to close crawl projects), they’ve made the Word Cloud Visualization tool language-aware in order to improve word filtering, and they fixed a Page Speed Insights bug, an issue with Crawling javascript that might cause a hang, extracting a static CSS path to make it support forward and back slashes and many other fixes that make Screaming Frog run smoother.
Screaming Frog URL Inspection API Integration Part 1?
The update announcement made a statement that indicated that there was still more integration work to be done to make it even better.
Because of practical limitations in how data can fit into the Screaming Frog interface, not all URL scan data can fit in it. But they promised to listen to user feedback to include additional data if users feel it is important.
The Screaming Frog ad noted:
“Through usage and user feedback, the way we integrate data to gain additional insights will also be improved.
If you spot any issues, or would like to see additional data or reports from the URL Inspection API – just let us know via support.
We already have some items planned, but we wanted to quickly get the core feature including the most important items available for users to enjoy.”
So it looks like more goodies are coming our way from Screaming Frog.
the quote
Read the official Screaming Frog announcement
