Doug on SharePoint

Stuff on SharePoint and other things.

Weird XML problem adding AAMs or taking the CA role on a new server

The other day I was upgrading the WFE servers to 64Bit and were rotating some WFEs in and out of the farm. While doing so I was moving the Central Admin around and got a failure running the command psconfig –cmd adminvs –provision.

The exception in the PSCDiagnostics log was this:

10/28/2009 10:48:59 7 ERR An exception of type System.Xml.XmlException was thrown. Additional exception information: Unexpected end of file while parsing Name has occurred. Line 27, position 14.

System.Xml.XmlException: Unexpected end of file while parsing Name has occurred. Line 27, position 14.

at System.Xml.XmlTextReaderImpl.Throw(Exception e)

at System.Xml.XmlTextReaderImpl.ParseQName(Boolean isQName, Int32 startOffset, Int32& colonPos)

at System.Xml.XmlTextReaderImpl.ThrowTagMismatch(NodeData startTag)

at System.Xml.XmlTextReaderImpl.ParseEndElement()

at System.Xml.XmlTextReaderImpl.ParseElementContent()

at System.Xml.XmlLoader.LoadNode(Boolean skipOverWhitespace)

at System.Xml.XmlLoader.LoadDocSequence(XmlDocument parentDoc)

at System.Xml.XmlDocument.Load(XmlReader reader)

at System.Xml.XmlDocument.LoadXml(String xml)

at Microsoft.SharePoint.Administration.SPAlternateUrlCollection.HasMissingUrl(String xml)

at Microsoft.SharePoint.Administration.SPContentDatabase.UpdateAlternateAccessMapping(SPAlternateUrlCollection collection)

at Microsoft.SharePoint.Administration.SPAlternateUrlCollection.UpdateAlternateAccessMappingInContent()

at Microsoft.SharePoint.Administration.SPAlternateUrlCollection.Update()

at Microsoft.SharePoint.Administration.SPAlternateUrlCollection.Add(SPAlternateUrl alternateUrl, Boolean fUpdate, Boolean throwIfExists)

at Microsoft.SharePoint.Administration.SPAdministrationWebApplication.Provision()

at Microsoft.SharePoint.Administration.SPWebServiceInstance.Provision()

at Microsoft.SharePoint.PostSetupConfiguration.CentralAdministrationSiteTask.ProvisionAdminVs()

at Microsoft.SharePoint.PostSetupConfiguration.CentralAdministrationSiteTask.Run()

at Microsoft.SharePoint.PostSetupConfiguration.TaskThread.ExecuteTask()

You will also be able to see these errors in the Application Event log with event IDs 100 and 104 and they essentially say the same thing.

The app pool and the web site provisioned but the Central Admin doesn’t show that server as having that role, the CA even works from that server. While doing some mucking around (that’s a technical term if you don’t know it) I tried to change the AAMs to include the new servers name in the AAM list. Well basically I got the same error just in the “friendly” SharePoint error as seen below

Finally broke down and gave Microsoft a call and after some looking around I have found out that there was an issue that was fixed in the re-release of the SharePoint’s August CU. Well that was fine but what exactly is it that was going on?

You can see in the KB what was going on: http://support.microsoft.com/kb/2000628

In a nutshell, in the central admin content db a field for the AAMs is too short — nvarchar(1023). The AAM XML in the db is truncated anything after 1023 characters and you have a malformed XML and thus the end of file exception.

This was causing a problem adding the CA because it needs to read that field and update it…This will also be a problem with any web apps which the AAM XML exceeds the 1023 limit – you would reach this pretty quickly if you added 4-5 AAMs for your sites. Here is an example of the XML, I formatted it to make it easier to read:

<AlternateDomains Count=”4″ Name=”Some Web App”>
<AlternateDomain>
<IncomingUrl>http://SomeIncomingURL#1.com</IncomingUrl&gt;
<UrlZone>Default</UrlZone>
<MappedUrl> http://SomePublicURL.com </MappedUrl>
</AlternateDomain>
<AlternateDomain>
<IncomingUrl> http:// SomeIncomingURL#2.com </IncomingUrl>
<UrlZone>Default</UrlZone>
<MappedUrl>http://SomePublicURL.com</MappedUrl&gt;
</AlternateDomain>
<AlternateDomain>
<IncomingUrl>http:// SomeOtherIncomingURL#3.com </IncomingUrl>
<UrlZone>Default</UrlZone>
<MappedUrl> http://SomePublicURL.com </MappedUrl>
</AlternateDomain>
<AlternateDomain>
<IncomingUrl> http://SomeOtherIncomingURL#4.com </IncomingUrl>
<UrlZone>Default</UrlZone>
<MappedUrl> http://SomePublicURL.com </MappedUrl>
</AlternateDomain>
</AlternateDomains>

That was about 870 characters, give-or-take. You can see how we can reach the 1023 limit pretty easily.

Disclaimer: DO NOT DO THIS ON YOUR PRODUCTION FARM WITHOUT GETTING APPROVAL FROM MICROSOFT SUPPORT OR YOU MIGHT BECOME UNSUPPORTED!

Now the workaround (approval was given) was to run the following on the Central Admin Content Database (needed to do this on the CA content DB because adding the new Central Admin was failing) Delete from DatabaseInformation where Name = ‘AlternateAccessMappingXml’

That was it…deleting from the DB allowed the process to read the XML from the field (which wasn’t there) and continue without error. Now if you still have AAMs that will cause the XML to exceed 1023 then it will still be truncated but the write operation will still succeed and you can go about your business. Next time there will still be an problem

Something to note: If you are consolidating a lot of portals into one central SharePoint portal and you plan to point your old URLs to the new portal, you should consider updating the system to at least August CU.

10/30/2009 Posted by | Administration, SharePoint, Upgrade | , , | Leave a comment

Another Content Deployment Problem:

Wow just when I thought we had our content deployment jobs fixed….

I worked another content deployment problem, this time I have the error!

Group cannot be found. at Microsoft.SharePoint.SPGroup.InitMember() at Microsoft.SharePoint.SPGroup..ctor(SPWeb web, ISecurableObject scope, SPUser user, String GroupName, Object[,] arrGroupsData, UInt32 index, Int32 iByParamId, SPGroupCollectionType groupCollectionType) at Microsoft.SharePoint.SPGroupCollection.get_Item(String name) at Microsoft.SharePoint.Deployment.RoleAssignmentXImport.UpdateAssignment(ImportStreamingContext context, SPWeb web, ISecurableObject obj, Boolean bAdd, String strUser, String strGroup, String strRole) at Microsoft.SharePoint.Deployment.RoleAssignmentXImport.ProcessElement(ImportStreamingContext context, XmlReader xr, SqlSession session) at Microsoft.SharePoint.Deployment.SqlImport.Run() at Microsoft.SharePoint.Deployment.SecurityObjectSerializer.SetObjectData(Object obj, SerializationInfo info, StreamingContext context, ISurrogateSelector selector) at Microsoft.SharePoint.Deployment.XmlFormatter.ParseObjectDirect(Object objParent, Type objectType) at Microsoft.SharePoint.Deployment.XmlFormatter.DeserializeObject(Type objectType, Boolean isChildObject, DeploymentObject envelope) at Microsoft.SharePoint.Deployment.XmlFormatter.Deserialize(Stream serializationStream) at Microsoft.SharePoint.Deployment.ObjectSerializer.Deserialize(Stream serializationStream) at Microsoft.SharePoint.Deployment.ImportObjectManager.ProcessObject(XmlReader xmlReader) at Microsoft.SharePoint.Deployment.SPImport.DeserializeObjects() at Microsoft.SharePoint.Deployment.SPImport.Run()</code>

Huh…What Group?

Let me give you a little bit of background on the environment. Content is being deployed to an externally facing server and consists of news & stuff and an area that is behind an FBA login. The path is set with the following: Deploy User Names is not checked and the Security Information is set to Role Definitions Only (not groups?). The main site is anonymous and the sites behind FBA have inheritance broken (more on that later).

On the source, a document lib had its inheritance broken and all of the groups removed, they didn’t want any publishers putting anything in there…(This is what was causing the job to have the error, more in a minute)

Still, what group?

First place to start is checking the content deployment logs….once it happens it will continue until you find that group…

You probably don’t have the ULS logging turned up for Content Deployment so you will need to do that…Once you have it turned up, run the job that is failing. It will error, check the logs…You should be able to find the error and a line or two above that error you should be able to find the site that that is causing that error. It will say something like not being able to get the group information.

Let’s talk about how inheritance is working with regards to content deployment (remember I am not pushing security, just Role Definitions – not groups?). When inheritance is broken, by default all of the group/security information remains on the site, we have all seen this. When the job is run, I expected to see the inheritance broken with all of the source groups still the same – not the case. What happened was inheritance was broken and all security information was removed – I need to check if some hotfix or SP2 changes this behavior and I will update.

Next…look at the site that is having the problem and compare the source and destination group information. Wait, the job removed all of the groups from the destination and the source groups were removed already….ok they are the same! Yes, they look the same but remember we have another variable, the changes that content deployment is trying to push to the destination.

Still haven’t figured out what group?

In figuring out what group, I inherited on both the source and destination and started comparing the two….

HA! In this case, there were some (3) SharePoint groups that were on the source and not the destination, so with that said all I did was add the groups on the destination at the same level (site collection for me) and ran the deployment job. Success!

Next the inheritance was broken on the inside and ran the job Success! – lastly I deleted the groups on the source and ran the job – SUCCESS!

That fixed it….

I will research why some of the groups were there and not others….remember, Role Definitions Only is not supposed to push the groups but some are pushed and some are not. I will update with what I find.

Recap:

  1. Turn up logging for Content Deployment
  2. Run the failed job
  3. Check the ULS logs for the error and determine what site is having the problem
  4. Start looking for differences in the groups
  5. Add the groups that are not on the destination that are on the source
  6. Run the job and hope you have found everything.
  7. If not wash, rinse, repeat…

07/18/2009 Posted by | Content Deployment | , | 2 Comments