On this blog you can read a lot about search engine optimization. Within this context it is crucial to know how the biggest search engine works. That is essential knowledge when we talk about SEO.
The focus in this post is on Google because this search engine has around 90% market share in searches on desktop (that fluctuates a bit) and even more than 90% on mobile. So it’s important to know how the Google search algorithm works in 2020. Worldwide, the percentages are a bit lower, with a lot of uncertainty and discussion about this, especially in the US.
Bing and DuckDuckGo on the rise
There are 2 other search engines on the rise at the moment: Bing and DuckDuckGo. DuckDuckGo is a privacy-friendly search engine, which is therefore growing substantially in a relative sense. In an absolute sense, the market share of DuckDuckGo is negligible. Which does not mean that there is no potential to it.
Bing is from Microsoft and is being pushed by Microsoft to users of Windows. The default search engine in Windows is of course Bing. And in some other ways, Microsoft is also trying to promote their browser Edge and search engine Bing. Bing is therefore also slightly on the rise. Nevertheless, Google remains by far the largest search engine in the world. So how does it work?
Positions are not fixed
The positions in Google are not fixed. New webpages are being added, webpages are disappearing, work is being done to improve the visibility of webpages … And then there are things like personalizing the search results. I will come back to that later, first I will take you through the process through which search engine positions are determined.
The search engine process: crawling, indexing and ranking
Search engine results pages (SERPs) are created after a user enters a search query into a search engine. And these SERPs are the result of a complex calculation, which is based on the algorithm of a search engine. It starts with crawling web pages, which are subsequently indexed and then given a position (rank) by the algorithm that goes to work based on that search query.
So, the process of a search engine has 3 steps: crawling, indexing and positioning.
Every search engine has its own algorithm
It is not without reason that you get (slightly) different search results with Bing or DuckDuckGo than with Google. Every search engine has set up the algorithm differently, which leads to different results. You see the biggest difference with non-English languages. Google is by far the best at that. For example, if you type Dutch search terms in google.com, you will get many more Dutch-language results than with other search engines. Google is really a language specialist.
Crawl: without links, little happens.
The entire process therefore starts with crawling. That is searching web pages for their content by the Google Bot or Spider. Please note that I explicitly refer to web pages here. Google does not index websites, but web pages. Every webpage is crawled by the Google Bot to determine what relevant information is on it. But how does the software know which pages to crawl? Through links and because webmasters add their websites and sitemaps to the Google Search Console and specifically request the search giant for indexation.
Links lead the Google Bot to new pages
Without sign posts or names you won’t be able to find a specific street. Links to pages are like signposts or street names. GoogleBot knows where new pages are because of these signs. These signs can be external links, internal links or links in a sitemap. So always make sure that web pages that you want indexed are at least included in the sitemap and that you add that sitemap to the search console including a request for indexation.
Page in the sitemap
It depends on your CMS how you add pages to the sitemap. If you use Yoast SEO to generate a sitemap, you can indicate this via ‘Advanced’ in the Yoast box. Under “Allow search engines to display this URL in search results?” choose “No”, which sets the page to noindex and does not include it in the sitemap. If all goes well, this is set to “Yes” by default, indicating that the page should be included in the index.
After the webpage has been crawled, either because Google finds a link or after submission to the search console including indexation request, it will appear in the index. That is the huge library that contains all those pages. Pages that are on noindex are not included in the index. This index is therefore the huge archive that Google uses to display results when someone enters a search query in the search engine.
Rank: every search query is a new calculation
After someone enters a search query into the search engine, the index is quickly consulted and the search engine generates a search result based on the algorithm. This ranking (positioning) is a new calculation every time. Basically, that can result in a different search result every time for the exact same search term.
Another search result for the exact same search term?
How is it possible that the same search term can produce a completely different search result? That starts with the fact that every search query is a new calculation. In addition, all kinds of factors are weighed in that calculation. Factors that play a role in these differences are:
- Device on which the search query is entered (PC, mobile)
- Clicks on results in previous search results
- Be logged in with your Google account
You can sometimes click on previous search results yourself, because under the result it says “You have visited this page before on ../../….” In that case, these search results can be placed higher than in a previous search result. I don’t see this as personalization, it’s not really about the person. Personalized search results occur less than they used to.
Personalized search results: what now?
So there is personalization. In the previous example there was 1 item that I would classify as personalization, that is ‘being logged in to Google’. Furthermore, there has been far-reaching personalization, where you were shown totally different search results than someone who could sit next to you. If you searched for ‘Egypt’, you received news and history because you were interested in politics, news and history. For example, the other person could actually see results with travel, because he was eager to travel.
I hardly come across these personalized results anymore, do you?
The algorithm: 3 pillars
Google’s algorithm consists of 3 pillars:
- Authority or popularity
I deliberately did not put numbers, because there is no hierarchy in it. Sometimes the signals from one pillar will outweigh the other, and in another situation it will be the other way around.
It goes much further …
Those 3 pillars are the top layer, with many hundreds of factors and signals underneath. And they also change regularly, because the whole world is constantly changing. For example, the increasing use of mobile phones is a major change in the world. And that resulted in the Mobile friendly update. And security on the internet is becoming increasingly important, which in turn resulted in Google taking HTTPS into account as a ranking signal. And so it goes on and on.
Continuous updates, even daily
Google has indicated several times that there are daily updates. Some small, others large. Major updates get their own names, often given by SEO experts who have discovered this update.
Do you now have a better understanding of the search process and the algorithms behind it? I now hope you understand how the Google search algorithm works in 2020. Or do you still have questions? Let me know in a comment, I am very curious. Thanks!