- 12 min. read
-
Sarah BerryLead Web Marketing Consultant
- Sarah Berry is a Lead Web Marketing Consultant at WebFX. With more than 10,000 hours of experience, she offers practical insights and strategies you can use to grow your digital revenue. When she isn’t polishing her Time Magazine Person of the Year Award, she’s spending time with her flock of ducks.
Did you know that more than half of website traffic comes from organic search engine results? Search engines, especially Google, play an enormous role in how people find content online. If your website isn’t indexed on Google, you’ll miss out on a massive portion of your potential traffic.
In this post, we’ll go over how to index your website on Google so that your content can show up in search results, bringing more traffic to your site, which include:
- Create An XML Sitemap
- Use Internal Links
- Create Unique, Valuable Content
- Get Rid Of Low-Quality Pages
- Build Backlinks
- Check Your Robots.Txt File
- Check Your Noindex Tags
- Check Your Canonical Tags
Want to quickly check your website’s SEO? Try our free SEO checker tool and get an SEO report in less than 60 seconds.
What is indexing?
Indexing is the process by which Google adds pages to its database so it can show them in search results.
In its index, Google keeps track of pages and information about those pages, which it uses for ranking.
How Google indexing works
So that it can show users the most relevant search results for their queries, Google crawls the web to discover content. It then indexes it. When someone conducts a search, Google ranks the content based on how well it meets the searcher’s needs.
Google uses bots called spiders or web crawlers to crawl the web looking for content. These spiders discover pages by following links. When a spider finds a page, it gathers information about that page that Google uses to understand and assess it.
It stores that information in its index so that it’s easily accessible when a user performs a search. Google’s index contains hundreds of billions of pages. Google periodically recrawls pages, which allows it to gather information about updates made to them.
When a user enters a search query on Google, Google pulls the most relevant pages from its index. Google’s search algorithms then rank the content so that the best content appears at the top of the results. This process takes place in a fraction of a second.
Learn How to Use a Web Cache Viewer
How to check if your site is indexed
One of the best ways to check if your website is indexed is to use Google Search Console.
If you’re signed up for Google Search Console and have access to your website’s account, you can go to the “Coverage” report under “Index.” In this report, you’ll see several categories and the number of pages on your website in each category. These categories are:
- Valid: Valid pages have been successfully indexed with no errors.
- Valid with warnings: These pages have been indexed, but there are issues with the pages.
- Error: These pages have not been indexed due to an error.
- Excluded: These are pages that were intentionally not indexed.
You can click through to various pages to see why they weren’t indexed and potentially fix any issues. If you search a specific URL in the URL inspection tool search bar at the top of the page, you can see whether that page has been indexed. Another way to check whether your site is indexed is to conduct a site search on Google.
Simply search “site:” plus your website’s URL on Google. You’ll then see how many pages on your website are in Google’s index. You can use the same method to check whether a specific URL is indexed.
You can also use our free SEO checker to get an SEO report on your website. The report includes a crawl and indexation analysis that will help you identify any errors and ensure your website gets crawled and indexed properly.
SEO checker provides data on key metrics to give you:Looking for an all-in-one SEO audit tool? You’ve found it
How to index your website on Google
So, how can you go about getting Google to index your site?
Google will eventually do it on its own unless there’s an error. There are steps you can take, though, to help Google find your site and potentially speed up the process. Here’s how to index your website on Google.
Ways to help Google find, crawl, and index your site
Here are some of the best ways to help Google find, crawl, and index your website:
- Create An XML Sitemap
- Use Internal Links
- Create Unique, Valuable Content
- Get Rid Of Low-Quality Pages
- Build Backlinks
- Check Your Robots.Txt File
- Check Your Noindex Tags
- Check Your Canonical Tags
1.Create an XML sitemap
An XML sitemap is a list of URLs on your website with information about those pages. A sitemap helps Google navigate your website and find the pages you want it to index.
You can create an XML sitemap manually or generate a sitemap that automatically updates using tools such as plugins. You can also create an image sitemap to help Google understand the images across your site.
Here’s an example of a sitemap with a single URL from sitemaps.org.
2. Use internal links
Google crawls the web by following links, so linking between pages on your website is an excellent way to help Google find your pages. Make sure your pages are linked together, and always add links to new content after publishing. If you want a new page to be crawled especially fast, link to it from one of your top-performing pages since Google recrawls these types of pages more often.
Ensure your internal links don’t have the rel=“nofollow” tag since Google won’t crawl nofollow links.
3. Create unique, valuable content
Google frequently says that publishers should focus on creating unique, high-quality content. Making sure your content fits this description may help with getting Google to index your site. Create pages that provide value to users, perhaps through offering actionable advice or a comprehensive answer to a question.
Avoid creating pages that have little useful content or have the same content as other pages on your site.
4. Get rid of low-quality pages
The more pages your website has, the longer it will take Google to crawl them all. If you remove low-quality pages from your site, you prevent those pages from wasting your “crawl budget,” and Google will get to your most important pages sooner. This tip is especially useful for larger sites with more than a few thousand URLs.
Most sites can benefit from removing low-quality content, though. In addition to helping you take full advantage of your crawl budget, getting rid of this content helps boost your site’s overall quality.
5. Build backlinks
Another tip for how to index your website on Google is to build backlinks — links from other websites to yours. Just as internal links help Google’s spiders find your website, so do backlinks. If you create useful, high-quality content, you should earn backlinks naturally as people discover your content.
Other publishers will want to link to your content if doing so provides value to their pages. You can also build backlinks by reaching out to publishers and asking them to link to one of your pages. Look for pages where a link to your site could add value.
For example, you could offer a link to one of your pages as a replacement for an outdated, broken link.
Get started by learning how to check your backlinks!
Clutch has personally interviewed more than 250 WebFX clients to discuss their experience partnering with us.Independent research from Clutch has named WebFX the
top SEO company in the United States.
How to make sure you’re not accidentally blocking Google’s crawlers
There are a number of technical issues that may cause Google not to crawl and index your website or individual pages on your website. Preventing and correcting these errors is another important aspect of how to index your website on Google. To check for these problems, look at Google Search Console’s “Coverage” report or search a URL in Google Search Console’s URL inspection tool.
Here are some of these issues and how to prevent and fix them.
6. Check your robots.txt file
If your website’s robots.txt file isn’t correctly configured, it could be preventing Google’s bots from crawling your website. What is a robots.txt file? It’s a plain text file that lives in your site’s root directory and tells bots such as search engine crawlers which pages to crawl and which to avoid.
If your robots.txt file isn’t set up correctly, you may accidentally be “disallowing” Google’s bots from crawling your site, portions of your site, or particular pages on your site that you want Google to index. To fix these issues, delete the relevant “disallow” directives from the file. Here’s an example of a simple robots.txt file from Google.
This robots.txt file would prevent Googlebot from crawling the folder. It would allow all other crawlers to access the whole site.
7. Check your noindex tags
Another way to block a page from being indexed is to include a noindex tag in the <head> section of the page’s HTML file. A noindex tag looks something like this:
- <meta name=“robots” content=“noindex”>
If you accidentally included one of these tags on a page, that page won’t be indexed. To fix this issue, delete the tag.
8. Check your canonical tags
Canonical tags tell Google which version of a page is the main page and should be indexed. This tag is useful if you have duplicate or similar content under various URLs. Canonical tags are placed in the <head> section of an HTML document and look like this:
- <link rel=“canonical” href=“https://example.com/sample-page/” />
If the canonical tag on a page you want indexed points to another page or even a non-existent page, the page won’t be indexed. To fix this issue, change the tag and point it to the correct URL, even if that’s the current URL. A canonical tag that references the current URL is called a self-referencing canonical tag.
How to request indexing from Google
If you follow the tips above, Google should eventually crawl your site on its own.
However, you can speed up the process by requesting a crawl of a new site or page or a recrawl of existing pages. Here’s how to index your website on Google by requesting a crawl directly from Google. To submit many URLs to Google at once, you can submit your sitemap via Google Search Console.
Under “Crawl” in the left-hand menu, click “Sitemaps.” From this page, you can submit sitemaps to Google and see the sitemaps you previously submitted. Another way to ask Google to crawl a sitemap is to send an HTTP GET request using the “ping” functionality. Type a request using the following formula into a browser or command line.
- https://developers.google.com/search/docs/advanced/sitemaps/build-sitemap;
If you want to submit one URL or a few URLs, use the URL Inspection tool in Google Search Console. First, inspect the URL by typing it into the search bar. Then, click “Request indexing” to submit the URL to Google.
The tool will then run a few checks to make sure the page is free of major issues. If the page checks out, the tool will queue it for indexing. Note that you must be an owner or full user of the Search Console property to submit a URL to Google for indexing using the URL Inspection tool.
Another option is to use the Google Indexing API to notify Google about new pages. However, the tool is designed for sites with lots of short-lived pages, and you can only use it on pages that host job postings or video livestreams.
Next steps after getting Google to index your site
For people to find your website on Google search, Google needs to be able to index it. With the above tips, you can make sure your pages are indexable and speed up the indexing process.
Don’t forget to check for crawl errors frequently in Google Search Console as well. You can also resubmit your sitemaps when you update them or request indexing for new URLs. Want help getting Google to index your site?
At WebFX, we have more than 28 years of experience in digital marketing and a team of award-winning SEO experts. We have the skills and knowledge to get your site indexed — and get it ranking too! Request a free quote online or call us at 888-601-5359 to learn more.
-
Sarah Berry is a Lead Web Marketing Consultant at WebFX. With more than 10,000 hours of experience, she offers practical insights and strategies you can use to grow your digital revenue. When she isn’t polishing her Time Magazine Person of the Year Award, she’s spending time with her flock of ducks.
-
WebFX is a full-service marketing agency with 1,100+ client reviews and a 4.9-star rating on Clutch! Find out how our expert team and revenue-accelerating tech can drive results for you! Learn more
Try our new free SEO checker at SEO.com
Boost your site’s search performance with our free SEO Checker. Analyze your website for optimization tips on titles, headers, content, speed, and more.
Get your Free SEO reportTable of Contents
- What is Indexing?
- How Google Indexing Works
- How to Check if Your Site is Indexed
- How to Index Your Website on Google
- 1.Create an XML Sitemap
- 2. Use Internal Links
- 3. Create Unique, Valuable Content
- 4. Get Rid of Low-quality Pages
- 5. Build Backlinks
- 6. Check Your Robots.txt File
- 7. Check Your Noindex Tags
- 8. Check Your Canonical Tags
- Next Steps After Getting Google to Index Your Site
Optimize Your Website Faster with SEO.com
Effortlessly find opportunities and monitor performance with this user-friendly tool designed by the SEO experts at WebFX!
Try it for FreeTry our new free SEO checker at SEO.com
Boost your site’s search performance with our free SEO Checker. Analyze your website for optimization tips on titles, headers, content, speed, and more.
Get your Free SEO report