Ok, so Yahoo Search decided to try and make our lives a little easier by offering a way to tell Yahoo Search that dynamic parameters in the URL (the ?’s and &’s and this=that) can be ignored on request.

Today comes a new wave for search engines with the first-ever Beta launch of ‘Dynamic URL Rewriting’ in Site Explorer. The new feature provides the ability for site owners to alert Yahoo! of the dynamic parameters in URLs that they’d like Yahoo! to ignore, which we’ll then automatically rewrite accordingly.

Hm, nice. But it gets even more interesting. You can actually give the dynamic URL parameter you want Yahoo to crawl your page(s) with. For example, leave the parameter “?show=full” in the URLs to be crawled.

• Login to Site Explorer from Yahoo! Search.
• Add to My Sites and then authenticate any sites that you own or manage.
• For any sites that you have authenticated, you’ll see a ‘Dynamic URLs’ tab.
• On this tab you can enter parameters you want us to either remove from URLs or always crawl with a specific value.
• Once you enter the parameter, we’ll show you the # of URLs we estimate will be affected.
• After you confirm the action, we’ll modify our crawler such that every time we see a URL from your site with that parameter, we’ll automatically rewrite it within our system as per your instruction.

Next Yahoo shows us why this may be useful for webmasters:

• A more efficient crawl of your site, with fewer duplicate URLs being crawled.
• Better and deeper site coverage, as we’ll be able to use our crawler capacity to find and index more new content on your site.
• More unique content discovered, as we’ll handle more dynamic parameters in your URLs (if you remove the content-neutral dynamic parameters).
• Fewer chances of crawler traps, or web page sets that can cause an infinite number of requests or a poorly constructed crawler to crash.
• Cleaner and easier-to-read URLs displayed in the search results.
• Better site ranking due to reduced fragmentation of links and anchor text to your site’s pages.

See in more detail on Yahoo Search Blog.

Share This