15 replies to this topic

[khan11]

hello guys,

i dont know much about seo system,but i heard google crawls any website after 1 week,is it true or its google's choice to crawl any site?



i know its kiddy question but i cant solve this in my mind.

[T100]

You can summit your site to google for crawling and google will index your page shortly. If you do not publicize your website, google will not know your existence until someone who likes your site put a hyperlink pointing to your site. Google will then follow the link and crawling onto your website. Google bot is a very diligent bot, you have to expect a high bandwidth use if you intend to have google index your page. You can preserve your bandwidth by editing a robots.txt file. You may either prohibit google from crawling your site, or prevent it from indexing some of your pages. (images take up the most bandwidth.)

[khan11]

i dont know how to create robots.txt file so you will be laughing to read that i created robots.txt and written nothing in it and then uploaded on my webserver coz i dont know what to write in it.

[Sinari]

I think that Google crawls them in a week- but I have yet to be listed in the Google search anywhere. I've been in the "crawling" list for a few weeks, but still nothing. Not sure how it works yet, but any advice would definently help.

[Florisjuh]

Google Crawl bot is one of the smartest bots ever, if your site has usefull information on it the bot will find it

[kbsp]

so these bots use up a lot of bandwidth how much are we talking im saying like for a forums like this...and i hear they even crawl peoples pm is that true

so they download files off the site too?

[Malexos]

I don't think they search peoples PM's, otherwise, my whole dirty conversation with my girlfriend would've been public . . . Aww, crap!

(No, they can't crawl PM's, without your password, and username, just public pages with no special privalages needed)

[Dooga]

Google usually crawls every site that people submit, with an exception of sites that were previously banned from their system (usually because you used some "under the table" method to increase your rank, which includes stuff like redirecting other sites, having too many irrelevant keywords, etc.)

[khan11]

i think or im sure that google dont needs any user or pass to login to crawl the pages coz alot of times i check from my admin panel that someone is crawling many pages without getting logged in.This means its the google bot.Its up to webmaster that if he put google ads in Private Messaging page then google will surely crawl that area.

Guys i heard that we can put some code to not allow google to crawl the page we dont want to.So if some of you know please share here.

Thanks in advance.

[arnz]

I've noticed that on some boards, that Google comes up under the "users" browsing bars when it comes to viewing some Invision powered forums.