How To Use Screaming Frog for SEO (+ Key Features)

With so many search engine optimization (SEO) tools out there, knowing the benefits, features, and operations of each one can be challenging. That’s why finding and learning one tool can be the most helpful choice. Enter our Screaming Frog Guide.

This page will break down how to use Screaming Frog for SEO through the following sections:

• What is Screaming Frog?
• Screaming Frog pricing
• How to set up Screaming Frog
• 7 valuable Screaming Frog features for SEO

Read on to learn more about Screaming Frog for your SEO campaign. If you want to stay on top of industry updates as well, sign up for our email newsletter below!

Don’t miss our Marketing Manager Insider emails!

Join 200,000 smart marketers and get the month’s hottest marketing news and insights delivered straight to your inbox!

Enter your email below:

Inline Subscription Form – CTA 72

“*” indicates required fields

Hidden

cf_country

Hidden

ajs_anonymous_id

Hidden

mcfxlp

Hidden

mcfxsource

Hidden

mcfxmedium

Hidden

mcfxterm

Hidden

mcfxreferrer

CAPTCHA

Email*

Email

This field is for validation purposes and should be left unchanged.

(Don’t worry, we’ll never share your information!)

What is Screaming Frog?

Screaming Frog is a UK-based company that specializes in SEO for businesses of all sizes. The Screaming Frog SEO spider is a desktop tool you can use to crawl websites and gather data for SEO purposes.

This quick, advanced SEO tool makes it easy for companies of all sizes to learn information about the web while building their site.

Screaming Frog pricing

Screaming Frog has two options: free and paid. The free version is essentially a basic version of the paid plan, with some changes and limited features. But, if you want to test out the tool before buying or have a limited budget, the free version gets the job done.

The free plan lets you:

• Find broken links, errors, and redirects
• Analyze page titles and metadata
• Review meta robots and directives
• Find duplicate pages
• Create XML sitemaps
• Complete site visualizations
• Crawl up to 500 URLs

The paid version lets you do all of this and more, with no limit on the number of URLs you can crawl. A year license is $259, and it includes free technical support, multiple integrations, and scheduling options.

How to set up Screaming Frog

To set up Screaming Frog, you must download the desktop tool. On their website, you can download the version suited to your computer:

Once you download the app successfully, you’ll be prompted to sign in to sync your account. Whether you have a free or paid plan will determine the features you can access.

7 valuable Screaming Frog features for SEO

Now that you’ve got the basics for using the Screaming Frog spider down, let’s go over how you can use Screaming Frog for various SEO purposes:

1. Crawling a website
2. Finding broken links
3. Creating an XML sitemap
4. Scoring links
5. Checking for crawlability issues
6. Performing keyword research
7. Finding embedded PDFs, videos, and images

With the tasks below, you can greatly improve your website and start ranking in search engines. Read on to learn more!

1. Crawling a website

The Screaming Frog Spider will automatically crawl the URLs you enter into the app.

To crawl a single website, follow these steps:

• Copy the homepage of the website you want to crawl

• Enter the link into the “Enter URL to spider” bar and click “Start”

• View the crawl in real time by checking the bottom of the application window

To crawl a list of URLs, click “Mode > List” to upload your list of URLs.

2. Finding broken links

You can also use Screaming Frog to find broken internal links that could be hurting your SEO. To find broken internal links, follow these steps:

• Click the “Internal” tab
• Sort the results by “Response Codes”
• View the 404, 301, or related status codes to find broken links

If you want to export this data into a report, you can:

• Go to “Bulk Export” > “Response Codes” > “Internal” > “Redirection (3xx) In Links” or “Client Error (4xx) In Links” or “Server Error (5xx) In Links”

• Export the data as a CSV

3. Creating an XML sitemap

Once the spider has finished crawling your site, you can proceed to making an XML sitemap. Here’s how:

• Go to the “Sitemaps” menu and select “XML” sitemap

• Make any configurations to your sitemap using the settings
• Click “Ok,” and your sitemap will download to your device

Now that you have a sitemap, you can upload your XML file to Google to help with indexing.

4. Scoring links

Link Score is a ranking system that qualifies links based on the internal link architecture. Basically, it tells you how important a link is depending on its position on the site’s linking structure. This step happens at the end of a crawl as part of the analysis.

To find out a link score, follow these steps:

• Once a crawl is complete, click “Crawl Analysis > Start” from the top menu

• The numbers should populate in a list once the analysis is complete

5. Checking for crawlability issues

If some of your pages aren’t being indexed, you can use Screaming Frog to figure out why. First, you should make sure that these pages don’t have any noindex tags, which will prevent them from being found.

Then, you can follow these steps to troubleshoot:

• Look through the URL list on the “Internal” tab to see what pages are missing
• Check for internal links to those pages, and look for patterns

If you don’t link to a page internally, the spider won’t be able to find it. So, if you find that a whole product category is missing, you might not have linked to it from your category page. This is just one way that the SEO report can help you find missing links.

6. Performing keyword research

The SEO spider can help you with keyword research by telling you what other sites are using for anchor text. To do so, follow these steps:

• Click “Bulk Export” > “All Anchor Text”
• Export a list of all the anchor text on the site, including where it is and what it links to

This will tell you what phrases they are trying to rank for and how they are using them on their site.

7. Finding embedded PDFs, videos, and images

Finally, you can use Screaming Frog to find embedded media, like PDFs, videos, and images.

To find PDFs, simply click the “PDF” filter in the “Internal” tab and filter all the crawl results.

To find pages that have embedded video or audio content, you must create a custom filter for a snippet of the embed code for YouTube or other media players. Go to “Internal” > “Filter” > “Custom” and enter the snippet.

To find images and their alt text, click “Check Images” in the Spider Configuration menu. Then, go to “Images” and filter by “Missing alt text” or “Alt text over 100 characters” to sort out images that need alt text improvements.

Independent research from Clutch has named WebFX the

top SEO company in the United States.

Clutch has personally interviewed more than 250 WebFX clients to discuss their experience partnering with us.

Read More Clutch Reviews

Want to learn more about using Screaming Frog for SEO?

If you want to leverage Screaming Frog in your SEO campaign, knowing how to use it is just the beginning. WebFX will help you outline a plan and use Screaming Frog — or any other SEO tool — to maximize your revenue.

Our SEO services have generated $10 billion in revenue and 24,859,684 leads for our clients. If you want to earn more from your SEO campaigns, we will help you accomplish that.

Want to learn more? Contact us online today!