Notice: This website is an unofficial Microsoft Knowledge Base (hereinafter KB) archive and is intended to provide a reliable access to deleted content from Microsoft KB. All KB articles are owned by Microsoft Corporation. Read full disclaimer for more details.

INF: How to Allow Search Crawlers to Index the MSIB Web Site


View products that this article applies to.

This article was previously published under Q314892

↑ Back to the top


Summary

Microsoft Solution for Internet Business (MSIB)-based Web sites are programmatically configured to prohibit external search engines from crawling (searching) the site, because a given site may contain sensitive documents. If you want to allow external search engines to crawl your MSIB-based site, you must index your site and then change the settings for the Content Management Server content on your site.

↑ Back to the top


More information

Use either of the following methods to configure your MSIB-based Web site:
Configure the site on a posting-by-posting basis by using the Site Builder program.

You can do this manually in the Site Builder Channels program. Right-click each posting, and then select Properties. On the Publishing tab, click to select the Web Robots Can Crawl Links and Web Robots Can Index This Posting check boxes. Note that when you configure settings with this method, the settings are not inherited by new postings, and therefore it is necessary to repeat this process for new postings.

-or-
Use code to configure the entire site.

Properties set by this script are inherited by all existing postings in the channel, as well as by any new postings. Save the script in a template of your choice, and then run the script to change properties for all channels and postings in a single Web site, as follows:
<!-- #include virtual="/nr/system/access/resolution.inc" -->
<%

On Error Resume Next

Dim strThisURL, pChannels

' Check if user has right to edit and approve the site.
If Not Autosession.UserCanModifySite Or Not Autosession.UserCanApprove Then
	Call Autosession.ManualLogin("Please log-in as administrator.", "/ChannelAllowRobot.asp")
End If

strThisURL = Request.ServerVariables("URL")
 
' Check that we're in update mode
If Not Autosession.isModeUpdate then
	' someone has called this ASP page directly. Redirect them back to this page in update mode.
	' NOTE: you cannot switch modes mid-page, therefore we must redirect back to this page.
	Response.Redirect( strThisURL & "?" & autosession.rootchannel.queryStringModeUpdate)
End If

Set pChannels = Autosession.RootChannel.Channels

Response.Write "Changing all channels and postings to allow robot following and indexing...<p>"

Call SetChannelsToAllowRobot(pChannels)

If Err.Number = 0 Then
    Call Autosession.CommitAll
	Response.Write "All Changes are committed successfully."
Else
	Call Autosession.RollbackAll
	Response.Write "Error " & Err & ": " & Err.Description
End If

Set pChannels = Nothing

'================================================
Sub SetChannelsToAllowRobot(ByRef pChannels)

	Dim pChannel
	
	On Error Resume Next

	For Each pChannel In pChannels

		pChannel.IsRobotFollowable = true
		pChannel.IsRobotIndexable  = true
		
		Response.Write "Channel ""<b>" & pChannel.Name & "</b>"""
		Response.Write " <font size=-2>IsRobotFollowable=" & pChannel.IsRobotFollowable 
		Response.Write " IsRobotIndexable=" & pChannel.IsRobotIndexable
		Response.Write "</font><br/>"
	
		If pChannel.Postings.Count > 0 Then
			Call SetPostingsToAllowRobot(pChannel.Postings)
		End If
		
		If pChannel.Channels.Count > 0 Then
			Call SetChannelsToAllowRobot(pChannel.Channels)
		End If

	Next

End Sub

'================================================
Sub SetPostingsToAllowRobot(ByRef pPostings)
	
	Dim pPosting

	On Error Resume Next

	For Each pPosting In pPostings

		pPosting.IsRobotFollowable = true
		pPosting.IsRobotIndexable  = true
		pPosting.Approve
		
		Response.Write "&nbsp &nbsp Posting ""<b>" & pPosting.Name & "</b>"""
		Response.Write " <font size=-2>IsRobotFollowable=" & pPosting.IsRobotFollowable 
		Response.Write " IsRobotIndexable=" & pPosting.IsRobotIndexable
		Response.Write "</font>"
	
	Next
	

End Sub

%>
					

↑ Back to the top


Keywords: KB314892, kbinfo, kbhowto

↑ Back to the top

Article Info
Article ID : 314892
Revision : 4
Created on : 2/23/2007
Published on : 2/23/2007
Exists online : False
Views : 365