If you are using Simple Machines Forum, then you must have seen URLs which have PHP Session ID (/?PHPSESSID= or /?P=) parameter in them. In this post we will see How to prevent such URLs from getting crawled by various search engine bots like Google, Bing etc.?



What is PHP Session ID & Why it is important?

In order to keep all browsing sessions interconnected (even for visitors who have disabled cookies in their browser), PHP Session Id is required in SMF. If you disable it (somehow), then previous browsing session of all forum visitors may get lost.

In order to remove any previously indexed session id URLs and block all other similar URLs from getting crawled in future, make use of either of the following tricks:

1. Use Robots.txt

You can block all Search Engine Bots from crawling session id URLs by using a robots.txt file with following lines in it:

User-agent: *
Disallow: /?PHPSESSID

OR

User-agent: *
Disallow: /?P

The above lines also instruct crawlers to drop all previously indexed URLs having session id parameter in them.

But, above lines in robots.txt file will also block various Advertising Bots like MediaPartners-Google (AdSense Bot) etc., which crawls your content for displaying contextual advertisements! In order to get rid of this problem, either you can use following lines:

User-agent: *
Disallow: /?PHPSESSID

User-agent: MediaPartners-Google
Allow: /

OR

User-agent: *
Disallow: /?P

User-agent: MediaPartners-Google
Allow: /

or you can implement following!

2. Use rel=”canonical”

Update: This mod is not required if you are using 2.0.6 version of the forum software.

canonical page is the preferred version of a set of pages with highly similar content and for making any particular page primary, we need to use rel=”canonical”.

When you implement rel=”canonical” for your Homepage, all Boards and all Topics, Google and other search engine bots will only crawl and index the primary page URL and will ignore all other URLs! For implementing canonical tag in SMF, install Anti-SID canonical tag.

This MOD gets installed and works without any problem. In case, you are experiencing any problem with this MOD, then you can post your query here