Monday, January 08, 2007

Analyzing potential SEO problems on large sites

When you are dealing with really large sites that get millions of visits, it can be a daunting task to see potential problems. The advent of Google’s Webmaster Tools has helped with identification of problems. However it’s not perfect, especially when you have millions of pages. Thus going back to basics can be the best way to check your logic.

If you are linux savvy, I would recommend grep’ing through you log files and start with the simple things. Here are a few examples of what you can look for.

more access_log | grep Googlebot | grep 302 > somenewfile this is a great way to see if you have any potential redirects that may be causing problems. 302’s are still notorious for causing issues.

You can also look for simple errors, like a 404

more access_log | grep Googlebot | grep 404 > somenewfile

I recently went through several sites and found an abundance of errors just by diligently looking at a small sample of logs. All you need is about 100meg’s or so, for a decent sample size.

Labels: ,

0 Comments:

Post a Comment

<< Home

My Photo
Name: Aaron Shear
Location: San Francisco, California, United States

I have been in the search industry since the late 90’s, no not 10-20 years. My career started early in the search Day’s at Inktomi, where I supported large search portals. For example, MSN, AOL, iWon, Hotbot, CNet too name a few. After Inktomi I became a freelance consultant. I consulted for a few of the Top SEO’s around 2002 time frame; obviously the market has changed since then. After consulting I joined a small SEO firm called SEO Inc as the CTO. At SEO Inc. I successfully optimized some of the largest clients including IGN, Sony, VEGAS.com, Beaches and Sandals Resorts to name a few. Even though SEO Inc was a ton of fun, I still wanted the ultimate SEO challenge. I moved on as the global head of SEO for Shopping.com an eBay company. This challenge was an interesting one, how do I optimize a site with 50 million products? Every month I helped the business grow by leaps and bounds. I am now consulting for mostly enterprise e-commerce clients. Yes there is more too me than this profile shows, but you will just have to ask.

Powered by Blogger

Add to Google Subscribe in Bloglines