I’m getting 500 and 404 error. First of all i want to fix the 500 Server error. To fix that error i upgraded my plan to more ram and CPU which cost me 225$ per month which comes with 15GB RAM and 4 Core CPU. But i can see that the google is crawling some of my search term and also some RSS feed url… Is it normal ? or do i need to do something to fix it ?
Here is the screenshot :
You check with google webmaster first. If it is still showing you the error then try to fix it with the guideline that webmaster is saying.
Check each URL under GoogleBot user-agent here https://httpstatus.io/
If URL returns 200, it means everything is fine.
Then you can mark it as fixed.
The best way to find google crawler error to Google webmaster tool and fix them. To find the list of all 404 pages, you can login to Google webmaster account and under diagnostics click on Crawl errors.