[Pro] Google and crawling

Sorry I just put this in Off topic - not sure where this should go so though I’d try here too -

Hi - Does anyone here have any tips to get Google updating websites content faster? I add content (not pages) all the time but takes weeks to show up in Google? The BBC for instance has new content daily and appears in Google searches daily? Are they paying for a service? I’ve tried looking all over the web and do what is suggested - sitemaps, resubmit etc, and I can see by my stats that the site is crawled twice a day? Yet the content cant be? If I cut and paste a newly added sentence from my site this will not bring up the sentence in google for weeks - yet i do the same with a BBC news item added that day, and wham its in Google - Any help, explanations - would be great - thanks in advance Jus


freewaytalk mailing list
email@hidden
Update your subscriptions at:
http://freewaytalk.net/person/options

This is mainly based on how important Google see your site as being; if the site is popular and updated frequently (like the BBCs) then it will get indexed more frequently automatically. If you’re site is new, or has few links to it (making it appear unpopular) you can add a sitemap Google Sitemaps - Freeway - Softpress Talk which specifies how frequently particular pages on your site get updated (hourly, daily, weekly etc).

Hope this helps,

Joe

On 14 Mar 2010, at 10:48, Justin Easthall wrote:

Sorry I just put this in Off topic - not sure where this should go so though I’d try here too -

Hi - Does anyone here have any tips to get Google updating websites content faster? I add content (not pages) all the time but takes weeks to show up in Google? The BBC for instance has new content daily and appears in Google searches daily? Are they paying for a service? I’ve tried looking all over the web and do what is suggested - sitemaps, resubmit etc, and I can see by my stats that the site is crawled twice a day? Yet the content cant be? If I cut and paste a newly added sentence from my site this will not bring up the sentence in google for weeks - yet i do the same with a BBC news item added that day, and wham its in Google - Any help, explanations - would be great - thanks in advance Jus


freewaytalk mailing list
email@hidden
Update your subscriptions at:
http://freewaytalk.net/person/options


freewaytalk mailing list
email@hidden
Update your subscriptions at:
http://freewaytalk.net/person/options

Hi thanks - I have added sitemaps but I didn’t know you could edit to tell it how often to crawl the content? Is this easy would I open the file in dreamweaver (I use freeway but can’t do the editing in there) and see were to add this?

Thanks


freewaytalk mailing list
email@hidden
Update your subscriptions at:
http://freewaytalk.net/person/options

Am I missing something here? Is it possible to tell a sitemap to tell google to use the most current updated pages? Or if the site is unpopular there’s no point as google will only crawl your site and use the content once every couple of weeks?

Please help. Basicaly I just want to know if it’s worth the effort keeping the site updated with hot topics so will drive people to a site? Which would be a waste of time if google only visits once a month! The hot topic would be outdated.

Thanks


freewaytalk mailing list
email@hidden
Update your subscriptions at:
http://freewaytalk.net/person/options

Hi Justin,

The sitemap has an element called change frequency for each page which you can set to hourly, daily, weekly, monthly etc. Setting this to hourly doesn’t guarantee that Google will index your site hourly, only Google can decide if it wants to do that, but at least it’s a step in the right direction eh?

If you sign up to a http://www.google.com/webmasters/tools account you can see how frequently Google indexes your site in a month. A site I made that hasn’t been updated for a while gets crawled a number of times a month (almost daily).

I wouldn’t get too hung up on Google though, it’s your users that will benefit from the news, at firt this may only be a few people but the numbers will grow if you keep putting in the effort.

Joe

On 14 Mar 2010, at 14:39, Justin Easthall wrote:

Am I missing something here? Is it possible to tell a sitemap to tell google to use the most current updated pages? Or if the site is unpopular there’s no point as google will only crawl your site and use the content once every couple of weeks?

Please help. Basicaly I just want to know if it’s worth the effort keeping the site updated with hot topics so will drive people to a site? Which would be a waste of time if google only visits once a month! The hot topic would be outdated.

Thanks


freewaytalk mailing list
email@hidden
Update your subscriptions at:
http://freewaytalk.net/person/options


freewaytalk mailing list
email@hidden
Update your subscriptions at:
http://freewaytalk.net/person/options

Thats the problem I’m trying to solve, if someone googles a topic that I just wrote about (I dont mean google my name then a topic - i mean just a generic topic) because google only updates every 2 weeks then my story will not show up? Until that is, its out of date? I have a webmaster account and can see my site is crawled daily but the pages contents doesn’t show up in google for weeks?? Heres an example - I changed a sentence on my webpage - when I select the new sentence and paste in google - it does not bring up my site - if I select the old sentence and place in google it does bring up my site? This will go on for weeks then finally I can see my new sentence bring up my site… yet the BBC adds stuff everyday and will bring up that site daily - even if the content is minutes old? They must pay for this service? I understand pr, sitemaps etc but I have looked at all this - was just wondering if theres a solid way to get google to index new content, then show in search results even if the site doesn’t get millions of hits a day?


freewaytalk mailing list
email@hidden
Update your subscriptions at:
http://freewaytalk.net/person/options

I would suspect the only sure way is to create a site like the BBC
that has lots of visitors and lots of updated content.

I think the real key here is visitors. The more visitors the site gets
the more respect Google will give it. And the more frequently Google
will visit to index it.

The more visitors you get the more likely you will get inbound links
to your content, these links the spiders will follow and update the
changed content.

From the sites we run, Google comes round about every two weeks.

Kind Regards
David Owen

On 14 Mar 2010, at 19:10, “Justin Easthall” email@hidden wrote:

Thats the problem I’m trying to solve, if someone googles a topic
that I just wrote about (I dont mean google my name then a topic - i
mean just a generic topic) because google only updates every 2 weeks
then my story will not show up? Until that is, its out of date? I
have a webmaster account and can see my site is crawled daily but
the pages contents doesn’t show up in google for weeks?? Heres an
example - I changed a sentence on my webpage - when I select the new
sentence and paste in google - it does not bring up my site - if I
select the old sentence and place in google it does bring up my
site? This will go on for weeks then finally I can see my new
sentence bring up my site… yet the BBC adds stuff everyday and
will bring up that site daily - even if the content is minutes old?
They must pay for this service? I understand pr, sitemaps etc but I
have looked at all this - was just wondering if theres a solid way
to get google to index new content, then show in search results even
if the site doesn’t get millions of hits a day?


freewaytalk mailing list
email@hidden
Update your subscriptions at:
http://freewaytalk.net/person/options


freewaytalk mailing list
email@hidden
Update your subscriptions at:
http://freewaytalk.net/person/options

Exactly, and precisely what I was saying in my first response. No-one pays Google (apart from for Google ads/sponsored results) since Google want a level playing field. It’s all down to popularity; the more popular the site, the more frequent the index rate. Popularity is largely based on links to your site, which by and large get created by updating content regularly. Just remember that aren’t going to get daily indexing overnight, but then no-one else is either…

Joe

On 14 Mar 2010, at 19:36, David Owen wrote:

I would suspect the only sure way is to create a site like the BBC that has lots of visitors and lots of updated content.

I think the real key here is visitors. The more visitors the site gets the more respect Google will give it. And the more frequently Google will visit to index it.

The more visitors you get the more likely you will get inbound links to your content, these links the spiders will follow and update the changed content.

From the sites we run, Google comes round about every two weeks.

Kind Regards
David Owen

On 14 Mar 2010, at 19:10, “Justin Easthall” email@hidden wrote:

Thats the problem I’m trying to solve, if someone googles a topic that I just wrote about (I dont mean google my name then a topic - i mean just a generic topic) because google only updates every 2 weeks then my story will not show up? Until that is, its out of date? I have a webmaster account and can see my site is crawled daily but the pages contents doesn’t show up in google for weeks?? Heres an example - I changed a sentence on my webpage - when I select the new sentence and paste in google - it does not bring up my site - if I select the old sentence and place in google it does bring up my site? This will go on for weeks then finally I can see my new sentence bring up my site… yet the BBC adds stuff everyday and will bring up that site daily - even if the content is minutes old? They must pay for this service? I understand pr, sitemaps etc but I have looked at all this - was just wondering if theres a solid way to get google to index new content, then show in search results even if the site doesn’t get millions of hits a day?


freewaytalk mailing list
email@hidden
Update your subscriptions at:
http://freewaytalk.net/person/options


freewaytalk mailing list
email@hidden
Update your subscriptions at:
http://freewaytalk.net/person/options


freewaytalk mailing list
email@hidden
Update your subscriptions at:
http://freewaytalk.net/person/options

I totally understand what’s being said here. I guess my problem is that the nature of my site isn’t about being linked everywhere or having a readership - I would just like my content to update quicker than 2 -4 weeks - but by the sounds of it I have no choice.

Thanks for the input

J


freewaytalk mailing list
email@hidden
Update your subscriptions at:
http://freewaytalk.net/person/options

It’s a difficult issue, Google is cruel. It sometimes seems to be favouring popular sites (has traffic). I do hope Google algorithms don’t sway to much in favour of traffic being a factor, as this will end up a self fulfilling prophecy making popular site rank better and so visited (the rich are getting richer analogy). Are they living up to their motto ~ Do no Evil?

I heard it said recently, I think on SEOMoz that you don’t want a web site that it only source of traffic is a search engine (or a very high percentage). As this openly shows that no one is revisiting again site directly and is an indicator of poor content. If you think that the case look at your web content and question why, and get it fixed.

David Owen { Freeway Friendly Web hosting and Domains }

http://www.ineedwebhosting.co.uk | http://www.PrintlineAdvertising.co.uk

On 15 Mar 2010, at 08:50, Justin Easthall wrote:

I totally understand what’s being said here. I guess my problem is that the nature of my site isn’t about being linked everywhere or having a readership - I would just like my content to update quicker than 2 -4 weeks - but by the sounds of it I have no choice.

Thanks for the input

J


freewaytalk mailing list
email@hidden
Update your subscriptions at:
http://freewaytalk.net/person/options


freewaytalk mailing list
email@hidden
Update your subscriptions at:
http://freewaytalk.net/person/options

Thanks - totally understand - All I ask is when I change something it doesn’t take a month to show, Im looking at the remove URL stuff on google and I believe if I remove a URL then add again this could sort the issue… see here from google…

Why do you want to remove content from search results?

The page has changed and I want the outdated information removed

If the content of the page (whether the content itself, or the page title or other information) has changed, the changes will be reflected in Google’s index the next time we crawl your site and refresh our index. However, until that time, Google’s cache may still contain the original content and as a result this content can still appear in search results. In this case, you can request that the cached version of the page be removed from search results. This will also remove the page snippet.


freewaytalk mailing list
email@hidden
Update your subscriptions at:
http://freewaytalk.net/person/options

FOUND IT! Its in Webmaster tools remove URL - lets hope it doesn’t take two weeks!

Cached copy of a Google search result

Remove the cached copy and description of a page that is either outdated or to which you’ve added a noarchive meta tag.


freewaytalk mailing list
email@hidden
Update your subscriptions at:
http://freewaytalk.net/person/options

I could be wrong but, if you remove it and add it again then they will do both things on their monthly cycle so you’ll be in the same situation. I’d love to be proved wrong though.

Joe

On 15 Mar 2010, at 09:35, Justin Easthall wrote:

FOUND IT! Its in Webmaster tools remove URL - lets hope it doesn’t take two weeks!

Cached copy of a Google search result

Remove the cached copy and description of a page that is either outdated or to which you’ve added a noarchive meta tag.


freewaytalk mailing list
email@hidden
Update your subscriptions at:
http://freewaytalk.net/person/options


freewaytalk mailing list
email@hidden
Update your subscriptions at:
http://freewaytalk.net/person/options

Hi Justin,

Why rely non Google - control it yourself!
If all else fails you could try adding some Javascript to your page(s).
Freeway > Page > HTML Markup…

Here’s two examples. The 1st is quick and dirty but works.
The 2nd has more Browser checking included and actually counts the time too.
Neither will take effect of course until Google crawls again to your page(s),
but once that’s happened your page(s) will REFRESH according to the ‘setTimeout’.

Try the 1st script and just paste code after Body, change the ‘location’ URL for the page you’re refreshing and the ‘setTimeout’.
This alone may be enough to resolve your issue.

Hope that helps?
Adrian


REFRESH THIS PAGE EVERY 60 SECONDS




On 14 Mar 2010, at 10:48, Justin Easthall wrote:

Sorry I just put this in Off topic - not sure where this should go so though I’d try here too -

Hi - Does anyone here have any tips to get Google updating websites content faster? I add content (not pages) all the time but takes weeks to show up in Google? The BBC for instance has new content daily and appears in Google searches daily? Are they paying for a service? I’ve tried looking all over the web and do what is suggested - sitemaps, resubmit etc, and I can see by my stats that the site is crawled twice a day? Yet the content cant be? If I cut and paste a newly added sentence from my site this will not bring up the sentence in google for weeks - yet i do the same with a BBC news item added that day, and wham its in Google - Any help, explanations - would be great - thanks in advance Jus


freewaytalk mailing list
email@hidden
Update your subscriptions at:
http://freewaytalk.net/person/options


freewaytalk mailing list
email@hidden
Update your subscriptions at:
http://freewaytalk.net/person/options

Thanks will try - Ill try the first one on the home page - If I literally just cut and paste the first sample - it will work (for the index page)?


freewaytalk mailing list
email@hidden
Update your subscriptions at:
http://freewaytalk.net/person/options