Skip to main content ↓
A person sitting at a table with laptops in a modern room with a blue and yellow color scheme, featuring a large stylized sun on the wall, suspended circles from the ceiling, and framed pictures and diagrams in matching colors.
  • Home
  • Blog
  • SEO How To Use Screaming Frog for SEO (+ Key Features)

How To Use Screaming Frog for SEO (+ Key Features)

With so many search engine optimization (SEO) tools out there, knowing the benefits, features, and operations of each one can be challenging. That’s why finding and learning one tool can be the most helpful choice. Enter our Screaming Frog Guide.

This page will break down how to use Screaming Frog for SEO through the following sections:

Read on to learn more about Screaming Frog for your SEO campaign. If you want to stay on top of industry updates as well, sign up for our email newsletter below!

Don’t miss our Marketing Manager Insider emails!

Join 200,000 smart marketers and get the month’s hottest marketing news and insights delivered straight to your inbox!

Enter your email below:

Inline Subscription Form – CTA 72

*” indicates required fields

Hidden
Hidden
Hidden
Hidden
Hidden
Hidden
Hidden
This field is for validation purposes and should be left unchanged.

(Don’t worry, we’ll never share your information!)

cta44 img

What is Screaming Frog?

Screaming Frog is a UK-based company that specializes in SEO for businesses of all sizes. The Screaming Frog SEO spider is a desktop tool you can use to crawl websites and gather data for SEO purposes.

This quick, advanced SEO tool makes it easy for companies of all sizes to learn information about the web while building their site.

Screaming Frog pricing

Screaming Frog has two options: free and paid. The free version is essentially a basic version of the paid plan, with some changes and limited features. But, if you want to test out the tool before buying or have a limited budget, the free version gets the job done.

The free plan lets you:

  • Find broken links, errors, and redirects
  • Analyze page titles and metadata
  • Review meta robots and directives
  • Find duplicate pages
  • Create XML sitemaps
  • Complete site visualizations
  • Crawl up to 500 URLs

The paid version lets you do all of this and more, with no limit on the number of URLs you can crawl. A year license is $259, and it includes free technical support, multiple integrations, and scheduling options.

How to set up Screaming Frog

To set up Screaming Frog, you must download the desktop tool. On their website, you can download the version suited to your computer:

Screenshot of Screaming Frog download page.

Once you download the app successfully, you’ll be prompted to sign in to sync your account. Whether you have a free or paid plan will determine the features you can access.

7 valuable Screaming Frog features for SEO

Now that you’ve got the basics for using the Screaming Frog spider down, let’s go over how you can use Screaming Frog for various SEO purposes:

  1. Crawling a website
  2. Finding broken links
  3. Creating an XML sitemap
  4. Scoring links
  5. Checking for crawlability issues
  6. Performing keyword research
  7. Finding embedded PDFs, videos, and images

With the tasks below, you can greatly improve your website and start ranking in search engines. Read on to learn more!

1. Crawling a website

The Screaming Frog Spider will automatically crawl the URLs you enter into the app.

To crawl a single website, follow these steps:

  • Copy the homepage of the website you want to crawl

Entering a URL into Screaming Frog

  • Enter the link into the “Enter URL to spider” bar and click “Start”

Starting a crawl in Screaming Frog

  • View the crawl in real time by checking the bottom of the application window

Checking crawl progress in Screaming Frog

To crawl a list of URLs, click “Mode > List” to upload your list of URLs.

2. Finding broken links

You can also use Screaming Frog to find broken internal links that could be hurting your SEO. To find broken internal links, follow these steps:

  • Click the “Internal” tab
  • Sort the results by “Response Codes”
  • View the 404, 301, or related status codes to find broken links

Finding broken links in Screaming Frog

If you want to export this data into a report, you can:

  • Go to “Bulk Export” > “Response Codes” > “Internal” > “Redirection (3xx) In Links” or “Client Error (4xx) In Links” or “Server Error (5xx) In Links”

Checking the report pathway in Screaming Frog

  • Export the data as a CSV

Saving a CSV report in Screaming Frog

3. Creating an XML sitemap

Once the spider has finished crawling your site, you can proceed to making an XML sitemap. Here’s how:

  • Go to the “Sitemaps” menu and select “XML” sitemap

Creating an XML sitemap in Screaming Frog

  • Make any configurations to your sitemap using the settings
  • Click “Ok,” and your sitemap will download to your device

Now that you have a sitemap, you can upload your XML file to Google to help with indexing.

4. Scoring links

Link Score is a ranking system that qualifies links based on the internal link architecture. Basically, it tells you how important a link is depending on its position on the site’s linking structure. This step happens at the end of a crawl as part of the analysis.

To find out a link score, follow these steps:

  • Once a crawl is complete, click “Crawl Analysis > Start” from the top menu

Scoring links in a Screaming Frog crawl

  • The numbers should populate in a list once the analysis is complete

5. Checking for crawlability issues

If some of your pages aren’t being indexed, you can use Screaming Frog to figure out why. First, you should make sure that these pages don’t have any noindex tags, which will prevent them from being found.

Then, you can follow these steps to troubleshoot:

  • Look through the URL list on the “Internal” tab to see what pages are missing
  • Check for internal links to those pages, and look for patterns

If you don’t link to a page internally, the spider won’t be able to find it. So, if you find that a whole product category is missing, you might not have linked to it from your category page. This is just one way that the SEO report can help you find missing links.

6. Performing keyword research

The SEO spider can help you with keyword research by telling you what other sites are using for anchor text. To do so, follow these steps:

  • Click “Bulk Export” > “All Anchor Text”
  • Export a list of all the anchor text on the site, including where it is and what it links to

This will tell you what phrases they are trying to rank for and how they are using them on their site.

7. Finding embedded PDFs, videos, and images

Finally, you can use Screaming Frog to find embedded media, like PDFs, videos, and images.

To find PDFs, simply click the “PDF” filter in the “Internal” tab and filter all the crawl results.

Viewing a PDF report in Screaming Frog

To find pages that have embedded video or audio content, you must create a custom filter for a snippet of the embed code for YouTube or other media players. Go to “Internal” > “Filter” > “Custom” and enter the snippet.

To find images and their alt text, click “Check Images” in the Spider Configuration menu. Then, go to “Images” and filter by “Missing alt text” or “Alt text over 100 characters” to sort out images that need alt text improvements.

Crawling images in Screaming Frog

Independent research from Clutch has named WebFX the

top SEO company in the United States.

Clutch has personally interviewed more than 250 WebFX clients to discuss their experience partnering with us.

Read More Clutch Reviews arrow right
cta6 img

Want to learn more about using Screaming Frog for SEO?

If you want to leverage Screaming Frog in your SEO campaign, knowing how to use it is just the beginning. WebFX will help you outline a plan and use Screaming Frog — or any other SEO tool — to maximize your revenue.

Our SEO services have generated $10 billion in revenue and 24,859,684 leads for our clients. If you want to earn more from your SEO campaigns, we will help you accomplish that.

Want to learn more? Contact us online today!

Try our new free SEO checker at SEO.com

Boost your site’s search performance with our free SEO Checker. Analyze your website for optimization tips on titles, headers, content, speed, and more.

Get your Free SEO report
TO TOP