Search engine optimization is the work that ensures Google and Bing find your website, keep its pages in their index, and display them to those looking for your services. The process follows three constant stages: crawling, indexing, and ranking. According to Ahrefs, people in Israel search for 'search engine optimization' about 400 times a month (November 2026). This guide is part of the full SEO service at SGO and explains what each engine checks and how to measure progress.
- Every search engine operates in three stages: crawling, indexing, and ranking.
- A page that hasn't been indexed won't show up in search results, even if the content is excellent.
- Google publishes clear speed thresholds: 2.5 seconds for loading, 200 milliseconds for response, and 0.1 for stability.
- On Bing, you can report a new page immediately via IndexNow, a protocol launched in October 2021.
- We measure clicks from non-branded searches and the leads they generate.

Table of contents
How does a search engine find and rank a website?
A search engine does not see a site the way a visitor does. It sends software called a crawler to download the code, save the data, and decide which pages to display and in what order. Every step depends on the previous one. A site with great content that a crawler cannot reach stays out of the results, which is why SEO always starts with the technical foundation.
Step one: Crawling
The crawler reaches your site through links from other sites, your sitemap, and internal links between pages. Before it begins, it reads the robots.txt file, which specifies which areas it is permitted to enter. According to Google's documentation, Googlebot reads up to 500 KB of this file and ignores anything beyond that. A page with no internal links is called an orphan page, and crawlers reach it rarely.
Step two: Indexing
After crawling, the engine analyzes the page: its topic, the language, whether it duplicates another page on the site, and which URL is the canonical version. Only indexed pages can appear in results. A page the engine crawled but decided not to save will appear in Search Console with the status 'Crawled - currently not indexed.' This usually indicates a quality issue or duplicate content.
Step three: Ranking
During the ranking phase, the engine compares all pages that answer a search. It evaluates user intent, backlinks, mobile loading speed, and the overall authority of the site. Google switched to mobile-first indexing and announced in 2023 that the transition is complete, meaning the version being tested is what is seen on a phone.
Which technical thresholds do search engines publish?
While some search engine rules remain secret, much of the logic is shared in official documentation. These are the technical benchmarks to verify on every website before publishing new content:
| Stage | What search engines do | The published threshold |
|---|---|---|
| robots.txt file | Reads the rules before crawling | Google reads up to 500 KB of the file |
| Page crawling | Downloads the page code | Googlebot reads up to 15 MB of each HTML file |
| Sitemap | Discovering new URLs | Up to 50,000 URLs or 50 MB per file |
| Loading (LCP) | Measures the load time of the largest content element | Up to 2.5 seconds |
| Responsiveness (INP) | Measures the response to a click | Up to 200 milliseconds |
| Stability (CLS) | Measures shifts of on-page elements | Up to 0.1 |
| URL inspection | Allows you to check if a page is indexed | Up to 2,000 checks per day per property via the API |
The three speed metrics in the table are called Core Web Vitals. Google measures these from real users and checks the 75th percentile, meaning three out of four visits must meet the threshold. The interaction metric INP replaced the previous FID metric in March 2024. Since 2014, a secure connection using https has also been a ranking signal for Google.
Initial audit for existing websites
For an existing site, we first check the indexing report in Search Console. We look for important pages that were not indexed, duplicate pages competing for the same keyword, and redirect chains. Then we move to mobile speed, and finally to content. This order prevents a situation where new articles are written for a site the search engine cannot crawl properly.
How does Bing SEO differ from Google?
Most searches in Israel go through Google, but Bing is a separate engine with its own index. A page indexed on Google is not automatically indexed on Bing. Bing provides its own management tool, Bing Webmaster Tools, where you can track searches driving impressions, crawl errors, and indexing status.
Rapid indexing for new pages
On Bing, you can report a new or updated page the moment it is published via the IndexNow protocol. Launched by Bing and Yandex in October 2021, this protocol pings the engine with the URL instead of waiting for a crawler to find it. In Google, reporting is done via sitemaps and the URL Inspection tool in Search Console. Note that Google's Indexing API is strictly intended for job postings and live broadcasts, not for articles.
How to connect a website to Bing
The shortcut is to import your site from Search Console into Bing Webmaster Tools. The import transfers verification and the sitemap, saving you from manual setup. Once connected, check the crawl error report, as Bing often identifies issues Google ignores, such as broken redirects or pages that return an error to its crawler only.
What content do search engines prefer?
Search engines aim to display the page that best answers the query. A page that addresses the user's search in the first paragraph, followed by detailed information, usually gets more clicks than one starting with a generic intro. A clear structure using H2 and H3 headings, comparison tables, and FAQs at the end helps the engine understand the page and helps the reader find what they need.
How long it takes for a new page to appear
An Ahrefs study found that only 1.74% of new pages reach the top ten on Google within a year of publication. In the same study, 72.9% of the pages in the top ten were over three years old. Therefore, SEO works better when strengthening pages that already receive impressions, while simultaneously adding new pages for low-competition keywords.
Internal links
Every article on the site should link to the service page it supports. This helps the engine understand which page is the main page for a topic, and the authority the articles gain is passed to it. For example, an article about SEO for a new business will link to the main SEO page rather than competing with it for the same keyword.
How do you measure search engine optimization success?
The primary tool for measuring Google performance is Search Console, which stores data for the last 16 months. For Bing, data is found in Bing Webmaster Tools. In both tools, you should filter out searches that include your business name, since those users already know you. Real growth is measured by clicks from general searches, such as service types and city names.
What is included in the monthly report
A quality SEO report includes four metrics: rankings for keywords that drive revenue, clicks from non-branded searches, the number of indexed pages, and leads generated from the site. Impressions alone are insufficient, as a page can receive thousands of views without a single click. In those cases, we refine the title and description shown in the results.
Checking indexing after publication
Check if a new page is indexed one week after publishing. Any page still missing from the index after two weeks is prioritized for fixing: we check for internal links, its presence in the sitemap, and ensure no tags are blocking indexing.
How much does search engine optimization cost?
At SGO, pricing is based on the scope of work. Our basic package starts at ₪1,500 per month, the growth package at ₪2,900, and the professional package at ₪4,900. The difference lies in the number of pages managed monthly, the volume of new content, and the technical scope. Full details are available on our SEO pricing page.
How SGO optimizes for Google and Bing together
At SGO, we have built hundreds of websites, and many clients partner with us for ongoing SEO. Our clients have over 3,500 keywords on Google's first page, and our own site has a Domain Rating of 57 on Ahrefs. For every site we manage, we connect both Search Console and Bing Webmaster Tools to report every new page to both engines. You can find full service details on our SEO services page, or learn about starting a new site in our article on SEO for new businesses.
Want to know what is holding your website back on Google and Bing?
Leave your details and we will audit your site’s crawling, indexing, and speed. You can read about the service and our workflow on the SEO at SGO page.
Search engine optimization FAQ
What is the difference between SEO and paid ads?
Should you optimize your site for Bing too?
How long does it take to see results?
What is IndexNow?
Which speed thresholds does Google check?
Why isn't my page appearing on Google?
Want more customers from your site? SGO specializes in SEO that drives traffic and leads.

