Issues provisioning SharePoint 2010 Managed Metadata fields
I’ve been working with the managed metadata functionality provided in SharePoint 2010 fairly extensively over the past few months. While it is a great feature and works well when used through the UI it has several rough edges that can cause problems when you are deploying features that use managed metadata fields. Some of these have been documented elsewhere but some of these haven’t so this will be the first of two posts covering (1) some of the potential problems using managed metadata (this post), and (2) a robust way of provisioning SharePoint 2010 managed metadata fields.
Deploying managed metadata fields using features
The majority of fields in SharePoint can be reverse engineered by looking at the field schema and this can then be used to deploy this in a feature. In the case of lookup fields and managed metadata fields (which inherit from the lookup field) these require an additional step to ‘wire them up’ to the correct lookup data programmatically. Wictor Wilén explains this process in his excellent article on ‘How to provision SharePoint 2010 Managed Metadata columns’. While this approach works for deploying standalone site columns there are a few problems you might encounter:
- If you create a list definition that uses this site column you get the error: Failed to get value of the “{0}” column from the “Managed Metadata” field type control. See details in log. Exception message: Invalid field name. {00000000-0000-0000-0000-000000000000}.
- The managed metadata column does not appear in the search refinement panel as per columns created through the UI.
- If the managed metadata features are not activated in the site the field will not work (and no obvious errors will be displayed).
The first issue is to do with the fact that managed metadata columns rely on a second note field to function correctly. This is mentioned in a forum post referenced in the comments of Wictor’s article but even when you add this in you run into the second problem: the managed metadata column does not appear in the search refinement panel as per columns created through the UI. This is something I haven’t seen covered anywhere else so in this article I’ll have a detailed look at how the managed metadata field functionality works so that we can correct these problems when creating list definitions that use managed metadata fields.
Understanding managed metadata fields
When we add a managed metadata column through the UI SharePoint creates two fields:
- A field with a type of either TaxonomyFieldType or TaxonomyFieldTypeMulti that is a lookup to the hidden list at /Lists/TaxonomyHiddenList. This is also covered by Wictor in his article Dissecting the SharePoint 2010 Taxonomy fields. The TaxonomyHiddenList contains several different columns including the term id for each term and is populated with terms used within the site collection to speed up retrieval. See Wictor’s article for a more in depth look at this functionality.
- A field with type note that contains information required to link the taxonomy field to the term set used by the field.
When we add a managed metadata column to a list SharePoint also ensures that the following fields are added to the list:
- TaxCatchAll
- TaxCatchAllLabel
Additionally two event receivers are added to the list if they are not there already. These are the TaxonomyItemSynchronousAddedEventReceiver for the ItemAdding event and the TaxonomyItemUpdatingEventReceiver for the ItemUpdating event. When you add or edit a document in the list or library the event receivers combine the values of all the taxonomy fields into the TaxCatchAllLabel column and the values of all the hidden note fields (i.e. the term set Id’s) into the TaxCatchAll column.
To show how this works I’ve included an example of the values for a document that contains two managed metadata fields; a RegionalOffice field that contains the single value ‘New Zealand’, and a Brand field that contains the values ‘Coca-Cola’ and ‘Cadbury’.
<z:row xmlns:z='#RowsetSchema'
ows_RegionalOffice='1;#New Zealand'
ows_RegionalOfficeTaxHTField0='New Zealand|6871660f-e140-4b1c-b149-ba9a027ea782'
ows_TaxCatchAll='6;#Abmm+eDzxEuvhrN8E9zF6w==|1aTvcq+j3EWjvkflkCVZuQ==|qQcPeBShEkCSTO5wF+Qfyg==|qx72Nq9QJk2iuNTagZG41A==;#5;#Abmm+eDzxEuvhrN8E9zF6w==|1aTvcq+j3EWjvkflkCVZuQ==|bY7N208XYE6yBHY0t7uFBA==|qx72Nq9QJk2iuNTagZG41A==;#1;#Abmm+eDzxEuvhrN8E9zF6w==|wn1zigp2eEaDmxywjP+5Og==|D2ZxaEDhHEuxSbqaAn6ngg==|RZYJJrApBEWbEGMjlxjpFQ=='
ows_TaxCatchAllLabel='6;#Cadbury#Љ|;#5;#Coca-Cola#Љ|;#1;#New Zealand#Љ|'
ows_Brand='5;#Coca-Cola;#6;#Cadbury'
ows_BrandTaxHTField0='Coca-Cola|dbcd8e6d-174f-4e60-b204-7634b7bb8504;Cadbury|780f07a9-a114-4012-924c-ee7017e41fca'/>
Inspecting the values we can see:
- The RegionalOffice column of type TaxonomyFieldType contains the lookup value referencing the ‘New Zealand’ entry in the TaxonomyHiddenList (this value is copied to the hidden list from the term set when users add it to an item within the site).
- The RegionalOfficeTaxHTField0 column of type note contains a reference to the term in the term set.
- The TaxCatchAll column that contains a list of all term set id’s in all managed metadata columns (this is easier to see in the SharePoint manager screen shot above). As the name suggests this ‘catches all’ the managed metadata values for the list item so both the regional office and brand values are included.
- The TaxCatchAllLabel column of type LookupMulti that contains a list of all term set names in all managed metadata columns.
- The Brand column of type TaxonomyFieldTypeMulti contains a lookup to the ‘Coca-Cola’ and ‘Cadbury’ entries in the TaxonomyHiddenList.
- The BrandTaxHTField0 column of type note contains a reference to the ‘Coca-Cola’ and ‘Cadbury’ terms in the term set.
Managed metadata search extraction
This process of combining all the metadata values for a list item is important when investigating how managed metadata fields appear in search refinements. During the search crawl process if SharePoint finds values in the TaxCatchAll field it will turn the associated metadata fields into managed properties. It does this by creating a special crawled property that starts with ows_taxId_ and then the name of the field, and a managed property that starts with owstaxId. For example if we have a field called RegionalOffice then we will end up with a crawled property named ows_tax_Id_RegionalOffice automatically mapped to owstaxIdRegionalOffice (in addition to the crawled property ows_RegionalOffice that is not mapped to any managed properties).
If this is working correctly you should be able to click on the crawled property to see the items that contain your managed metadata field as shown below.
This allows us to perform search queries on managed metadata field values, and also allows the refinement panel to display the metadata columns and values.
The problem
When you add a site column or content type to a list through the UI, SharePoint automatically adds the TaxCatchAll and TaxCatchAllLabel columns and the taxonomy event receivers. When you create a list definition via a feature, however, these are not added automatically. Simply adding in the TaxCatchAll columns isn’t enough as this will be empty as there are no event receivers to populate this field. As a result the search crawler will not automatically create managed properties to allow the search refinement to work. These aren’t hard to add in but it took me a lot of investigation to find out so hopefully this saves others some time.
In the next post I’ll show how we can use this to provision the managed metadata fields in a way that allows them to participate in the automated process of appearing as search refinements. I’ll also include some updates to the process of provisioning SharePoint 2010 managed metadata fields to mitigate some of the potential problems I’ve found when using this in real life scenarios.
Stay tuned!
Update 12/3/11 – The next post The complete guide to provisioning SharePoint 2010 Managed Metadata fields is now available.
You may be interested in my problem.
I created a content type hub , created a TaxonomyField using CAML ,a content type using CAML, wired up the termset using code
I created List Templates and List Instances based on this content type and all worked perfectly (didnt neeed to create hidden Note fields at any time)
Then I needed to add more content types to my provisioned list instance.
I created a feature which contained my new content type , with a FieldRef to my Taxonomy field.
So far so goo.Then I tried binding the new content type using UI , OM and using ContentTypeBinding.
It appears to be added correctly using any of these techniques , but When I try to use my new content type I get the Failed to get value of the “{0}” column from the “Managed Metadata” field type control. See details in log. Exception message: Invalid field name. {00000000-0000-0000-0000-000000000000}.
After lots of investigation I found that.
1. I could manually fix the problem it my removing the field from the lists version of the content and adding the lists version of the field back onto the content type.
2. I could fix it programatically by inspecting the PLIst and finding my Taxonomy field , pulling out the TextField guid (which I hadnt added) and creatin a SPFieldLink lists version of my content type.
3. As an aside I also found that If I added the content type to the original list template definition and provisioned new list instance it provisions with no problems.
So I am left wondering
1. Should I have deployed a hidden Note Site Column in the Content Type Hub and wired it up up before I started creating sites and list instances (I hope not after reading http://insidesharepoint.net/post/2010/11/05/Using-taxonomyfields-in-Sharepoint-2010-Part-I.aspx)
2. How can I be sure that future UI or CAML generated content types / Taxonomy fields won’t have similar problems
3. How come it all works when the content type is part of the list template , but not if I add programatically or using content type binding
4. Is this happening because the OnAdded method of the TaxonomyField type is not being called? How cold I get around this?
Probably a lot of questions and probably not enough detail , but if you have seen this before any guidance ould be much appreciated.
Thanks
Stuart Evans
7 Mar 11 at 9:01 pm
@Stuart I’ve found that when I get that error message the TextField property of the taxonomy field is set to {00000000-0000-0000-0000-000000000000}. So the first this I would do is check this using using SharePoint Manager 2010. Like you I’ve noticed that sometimes things still work ok but in certain situations this breaks. I have a feeling that in some cases the OnAdded method of the TaxonomyField works (e.g. when adding a managed metadata site column through the UI) and sometimes it doesn’t. If the TextField is not set correctly you might want to try setting that via code and creating the note field if it doesn’t exist. Additionally have a look at my follow up post – I’ve tried adding a ContentTypeBinding with an additional content type that references a taxonomy field (as you describe) in the example project and things appear to work correctly.
Ari Bakker
8 Mar 11 at 10:56 pm
I had an issue yesterday very similar to what Stuart describes.
I started out following Wictors post, and then after re-provisioning my solution on a site with real content that used the fields, i started getting the 00000000-0000.. error.
On inspection i noticed that i had 15 hidden notes fields for one of the taxonomy fields (the only one being used alot) eg. …TaxHTField0 – 14.
I then set DisplaceOnUpgrade=”true” on all taxonomy fields, added the hidden notes fields and added TextField with note field guid to my feature receiver.
This worked for new list content types, but for the existing ones i saw no change (makes sense as they are database instances of the ghosted site CT).
I then “touched” the site column, just by going to edit mode and back in UI. Now the error changed to the specific notes field guid i set in TextField! So same error, just the guid instead of zeros.
As Stuart explains, you can solve this in several ways. I opted for manual, as the customer was adding documents to a few doclibs only, so i needed those working fast. Both remove/add linkfields directly on the list, or remove/add taxonomy field from the list CT seems to solve the problem.
I also noticed the facets missing in search. Will definetely try your mapping approach when i get around to it!
Good writeup!
Anders Rask
9 Mar 11 at 8:40 am
@Anders – thanks for the info. Another good reason to create the note field and set the TextField during initial provisioning! I cover how to do this in the next post.
Ari Bakker
13 Mar 11 at 1:06 pm
Excellent article. I ran into similar issues as soon as yesterday and this is exactly what I was looking for.
Bilal Ahmad
6 Apr 11 at 2:33 pm
FYI… Hope this helps someone else.
I was getting the error:
Exception: Microsoft.SharePoint.UserCode.SPUserCodeSolutionExecutionFailedException: Unhandled exception was thrown by the sandboxed code wrapper’s Execute method in the partial trust app domain: An unexpected error has occurred. —> Microsoft.SharePoint.UserCode.SPUserCodeSolutionProxiedException: Could not load type ‘Microsoft.SharePoint.SPServiceContext’ from assembly ‘Microsoft.SharePoint, Version=14.900.0.0, Culture=neutral, PublicKeyToken=71e9bce111e9429c’.
I killed the vssphost4.exe process, bounced IIS (iisreset), bounced the OWS Timer (net stop sptimerv4;
net start sptimerv4) and removed the dll for the custom fields solution from the GAC. One or more of these things made the error go away.
Kris Huggins
14 Apr 11 at 7:41 pm
Great article!
I have an extremely annoying issue regarding taxonomy fields. In our scenario we have set up a number of document libraries, a number of content types and are using the content organizer to structure the content. Most of our columns are taxonomy fields, all added manually. Really all work has been done manually, exccept importing the term set using the Managed Metadata Service Application import tool. Well, everything was great and all.
Later on we started utilizing the metadata navigation feature for these taxonomy fields. Despite having quite a few documents in the libraries, with the respective taxonomy values, we got no results using the metadata navigation. So we checked all the settings and whatnot, which rendered us pretty clueless. We could create a new view and specify filtering for a particular taxonomy value (term), which worked fine. We could not use the list view column filtering either; same issue, no results.
So we started to dig a bit deeper. As it turns out, the “TaxHTField1” field for all these columns seemed to have a corrupt value. Instead of a value on the form “label|guid” we have “|guid” as the value. We have an issue opened with Microsoft Support, but they are as clueless as us. The ows_TaxCatchAll and ows_TaxCatchAllLabel fields however seems to be valid.
We have absolutely no idea what is causing these issues. Also, for a few documents this issue have disappeared over time and subsequently the metadata navigation has begun to work. But this is only for a few documents, relatively speaking. I am not positive that this “label|format” error really is causing the filtering issue, but that is the only error we have found inspecting the libraries and the hidden taxonomy list.
Any ideas?
Johan Nordholm
10 May 11 at 2:48 pm
@Johan Have a look at http://ilyal-technology.posterous.com/sharepoint-2010-managed-metadata-bulk-import – it appears there is an issue with the content organiser and importing terms using the term store import tool and Ilya has detailed the problem and some workarounds.
Ari Bakker
10 May 11 at 3:22 pm
Andrew Connell also has a good article on this topic at http://www.andrewconnell.com/blog/articles/sharepoint-2010-managed-metadata-in-depth-look-into-the.aspx
Ari Bakker
22 Jul 11 at 1:06 pm
[…] developers used Ari Bakker’s blog and code as their basis, specifically the code and inthis excellent blog post. Ari began his journey with this excellent blog post from Wictor Wilén. Over time we had […]
Correctly Provisioning Managed Metadata Columns
23 Jun 12 at 12:22 am
[…] (SharePoint creates, for every Managed Metadata filed, crawled properties and managed properties as described here) […]
Search Driven Applications with Office 365 / SharePoint Online – Part II | SharePointDevWiki.com
25 Sep 13 at 11:01 pm
[…] written a good bit about this these two topics. Both Wictor Wilen (here & here) and Ari Bakker (here & here) have some great posts on these subjects. Yes, some of the stuff you’ll find here […]
Managed Metadata: In Depth Look into the Taxonomy Parts | SharePoint interests
31 Oct 13 at 3:07 am
[…] Wilen (here & here) and Ari Bakker (here & here) have written a good bit about this subject. Now I’d like to give you my opinion. […]
Managed Metadata – Creating Managed Metadata Columns | SharePoint interests
31 Oct 13 at 3:09 am
[…] (SharePoint creates, for every Managed Metadata filed, crawled properties and managed properties as described here) […]
Search Driven Applications with Office 365 / SharePoint Online – Part II | NothingButSharePoint.com
29 Nov 14 at 10:35 pm