This post is the first of a small series about the mapping of the representation of the core entities of MOSS. I use this info to make a complete mapping in the MOSS Feature Generator, but I thought it could be useful on its own. This first article is about Content Types, as you might have guessed from the title. Since I strive for completeness, let me know if you think I missed anything (or I'm dead wrong). The actual content is an attached word-file (it is indeed a rather tiny link).

Most of the property mappings are incorporated in the MOSS Feature Generator. In future versions, all mappings should be part of the MFG. I'm working now on a little restructuring of the code, so that it has become possible to better organize functionalities into features. Supporting this in the treeview has been a bit of a hassle, though. Next issue is keeping all Guids and ID's in sync. To date, this is still a manual operation.

Note that in the table with the actual mapping, I used existing documentation as much as possible, with some humble annotations of me thrown in, marked with (Alfred).  

Alfred

In this post I'll Show you take how to generate features by First creating a Virtual Site, consisting of all your selected functionality from the original MOSS site. If you have created the Virtual Site, you can create all the features at once, using default settings, or you can generate them one by one ,giving you more control.

At this time, I have not included a great deal of logic into build process. It is planned for the next version of the MFG. So when you start editing Content Type ID's for example, you'll have to keep them in sync manually when you refer to them from List Templates. 

Therefore, it is not practical to generate hundreds of features at once. Package them into groups of ten or so.

If you want to have this information in a slightly more readable version, goto http://www.codeplex.com/mossfeaturegenerator and check out the MFG Quick guide.

The steps to create features from the MOSS GUI:

1 From the Main menu, choose Build à Site

A form now opens in which to the left a MOSS site will be shown (only Site Columns, Content Types, List Templates and List Instances). To the right, the functionalities you selected for the creation of features is shown.

   2 Enter a site Url and click Go. Then select a site.

The functionalities of the MOSS site will now be shown.

3 Drag en drop the functonalities from the real MOSS site to the Virtual Site, consisting of all the features you select.

4 If you want to change properties, dubbelclick an item, and an appropriate properties screen will open. This is the same screen as when you create a functionality directly from a MOSS site.

5 If you wish to change the properties of the feature.xml, press, Specify Feature. A form will open, allowing you to specify properties for this specific feature.

 

6 Create a single feaure by adjusting the path and pressing the Create Feature. If you create a 12\Template\Features directory structure, you can use

WSPBuilder afterwards.

 

7 When you wish to create all the features at once, specify the path and press Create Features on the main form.  Note that when you create features in this way, you will be prompted with a form allowing you to specify a feature file for the combined Site Columns. The program cannot deduce the properties for this file from the collection of site columns.

 

8 If there are any XSD Validation Errors, the Show Schema Validation Errors button will be available.

 

9 By pressing the Show Schema Validation Errors button, you may open a form containing info about the validation errors.

 10 You now have your generated Feature.xml, Elements.xml, Schema.xml and your Forms and Views on the ListTemplates.

11 Your original functionality gets reproduced when you install and activate the features.

Today, the MFG, version 0.7 has been released. It allows you to generate MOSS 2007 Features quickly and accurately, according to the functionality you created by using the MOSS Gui. It generates XML files by reading the properties for the Site Columns, Content Types and List from the object model.

New features include the creation of MOSS Features with drag & drop from your original MOSS 2007 Site, a better separation between Feature (feature.xml) and functionality (elements.xml, schema.xml, AllItems.aspx, other forms and views), and more control on the properties of features and functionality.

You can find the MFG at http://www.codeplex.com/mossfeaturegenerator

Above some figures, to give you an impression. I'll update you soon with a more elaborate description.

 Alfred 

When looking on the net for information about validating features against their XSD, I could not find information on how to do this. Now this might be so obvious that nobody takes the bother. Or the existence of the schema files for Sharepoint 2007 is just related to its use on intellisense in Visual Studio, because it is not perfect.

First, I tried the XmlValidatingReader, but this is obsolete (or at least the methods I used), and the XmlValidatingReader proved very very mild in its validation.

Next, I used an example of Rajdeep Kwatra using XmlReaderSettings. (http://rajkwatra.blogspot.com/2007/06/validating-xml-in-net.html), which I adjusted for the use with Sharepoint. Rajdeep has explained all the Xml bits quite nicely, so I concentrate on some Sharepoint aspects. You need to add the CoreDefinitions.xsd file to the xmlSchema, before the wss.xsd, or you'll get complaints that one of the core definitions cannot be found. Doing so, you'll find that from the Schema.xml of SpFields, unlegal attributes are produced for the field nodes for Lookup types and Person/users. These attributes are WebId, UserSelectionMode, UserSelectionScope and Version. Also, quite a large number of validation errors resulted from the validation of the schema.xml files. I yet have to look into this.

Below is the code I used. xmlPath is the path to the feature.xml, elements.xml or schema.xml file. xsdPath is the path to the folder containing the Sharepoint schema files, typically "C:\Program Files\Common Files\Microsoft Shared\web server extensions\12\TEMPLATE\XML";

        public static void ValidateXml(string xmlPath)

        {

            currentXmlFile = xmlPath;

            XmlSchema xmlSchema = XmlSchema.Read(new XmlTextReader(xsdPath+"CoreDefinitions.xsd"),

                new ValidationEventHandler(validatingReader_ValidationEventHandler));

            XmlSchema xmlSchema2 = XmlSchema.Read(new XmlTextReader(xsdPath + "wss.xsd"),

                new ValidationEventHandler(validatingReader_ValidationEventHandler));

            XmlReaderSettings xmlReaderSettings = new XmlReaderSettings();

            xmlReaderSettings.ValidationType = ValidationType.Schema;

            xmlReaderSettings.Schemas.Add(xmlSchema);

            xmlReaderSettings.Schemas.Add(xmlSchema2);

            xmlReaderSettings.ValidationEventHandler += new ValidationEventHandler(validatingReader_ValidationEventHandler);

            using (XmlReader xmlReader = XmlReader.Create(new XmlTextReader(xmlPath),

                    xmlReaderSettings))

            {

                while (xmlReader.Read()) ;

            }

        }

        static void validatingReader_ValidationEventHandler(object sender, ValidationEventArgs e)

        {

            validationErrors.Add(new MFGValidationError(e,currentXmlFile));

        }

I used this code in the MOSS Feature Generator, so that, when you generate features, you can inspect if there are Schema validation errors.  

Alfred

While in the testing  process for the upcoming release of the 0.7 version of the MFG, I was actually amazed to see how well rather complex views could be generated.

I tried to make it as difficult as possible, by created lists with the MOSS Gui, and then creating complicated views. When I say complicated, I mean complicated to make, without the use of a generation tool. With the MOSS Gui, creating a Gant-view or conditional rendering is easy.

Most notably, I generated the following views:

• showing all columns(default view)
• sorted on two columns
• filtering based on two columns
• group by two columns
• showing totals of a column, paging and a different style
• calendar
• datasheet
• gant

showing all columns(default view)

sorted on two columns

filtering based on two columns

group by two columns

showing totals of a column, paging and a different style

calendar

datasheet

gant

Now, that all seemed to work quite well. On the downsite, I also tried to generate a KPI-List, but this didn't function. Lists created with the generated template missed the content types to generate KPI's from, and the option to enable content types was not available. I'll have to look into this.

The above tests were run with an intermediate version of the 0.7 release, but since the code was not changed on this respect. I think it should work just the same with the 0.6 release. If you do find any problems, bugs, missing functionalitity, please notify me, at my mail adres, or post them at www.codeplex.com\mossfeaturegenerator

Alfred 

The next version of the MFG will contain a Treeview of the features you generate, with their properties. The features will be linked automatically when you drop them into the treeview. In this way, I hope to enable you to create a set of related features (ContentTypes based on Fields, ListInstances based on ListTemplates etcetera) with ease. 

Since I have not been posting new updates, it seemed good to give you a preview what is to come soon. Other updates will be the support of ListInstances and the seperation of Logical elements from their features, which give a greater deal of control on how the feature.xml, elements.xml and schema.xml-files are generated.

 For the current version of the MFG, see CodePlex.

Alfred

The  MOSS Feaure Generator is very well suited to be used with the WSPBuilder. One using both in a combination, you can get from zero to solution packaged feature in minutes. All you need to do is generate the features in the directorystructure required bij WSPBuilder. You can find both at CodePlex:

 http://www.codeplex.com/wspbuilder

http://www.codeplex.com/mossfeaturegenerator

Alfred

Hi,

 I moved application and the Visual Studio solution to CodePlex, here: 

MOSS Feature Generator

Alfred 

One of the problems you face when setting up a MOSS 2007 product line concerns the MOSS Power User. In a traditional environment, software comes from the development team, or is unsupported. In a MOSS environment however, it is not allways possible to be as rigid as this, since Power Users have been explicitly given the possibility to create list, content types etcetera. From a business perspective, this is a great virtue.

For developers however, the artifacts of Power Users may prove to be an issue of concern, for instance because they lack structure (columns that should be the same are re-created for each lists, preferably each of them slightly different, things like this). One strategy is to let the artifacts of Power Users be unsupported, until there is some formal business demand to support it. From this point on, the development team will create its own version of the artifacts, which have the required quality, and which can from then on be deployed like features (or site definitions, if need be).

The MOSS Feature Generator can be used in this scenario as well, because it grabs the List instances, and makes a ListTemplate and List Instance Feature of it, which can tailored if necessary. The same goes for Content Types and columns. So it is not only suited for creating Features in a structured manner, but it can also be used to make features out of the work of Power Users. In this way, the business knowledge of the Power Users is combined with the quality software engeneering of developers. Offcourse, when tried in a real world scenario, it will not be as easy as this. I think it will have to go this way, however.

Alfred 

As I promised in my previous post, I added the Visual Studio solution of the MOSS Feature Generator. This contains the code for the list template generator as well as the code that was previously contained in the CTFC (Content Type Feature Creator). In my next post, I will add Feature Creation for List Instances. I then plan to do a little reengeneering, so that you can yourself decide if you wish to create Columns, ContentTypes, ListTemplates and ListInstances in one, or in several features. I'll also be extending the customization of the feature. The attachment shows up only if you select the post itself.

Alfred 

Since my last post I've been busy extending the idea of generating features to the domain of list templates. This proved to be more difficult, since lists don't come with the nicely formatted xml in the content database, that provided me an easy solution generating content types.

On the other hand, the gains would also be bigger, since creating a list without the help of a generator involves modifying a schema.xml file, which has a rather intimidating complexity (to me at least).

What I wanted to do, is creating lists in a two or three step process:

1. Creating a list using the MOSS GUI;
2. Optionally editing the Forms, by using the Sharepoint Designer;
3. Generate a ListTemplate Feature with Feature.xml, Elements.xml, Schema.xml and Forms (typically AllItems.aspx, DispForm.aspx, EditForm.aspx and NewForm.aspx), and other forms that result from the creation of additional views in the MOSS gui.

Getting the generated schema.xml right wasn't particularly easy. I used the SchemaXml properties for views, forms etcetera, but none of them where usable right out of the box. In the end however, I succeeded. I've been testing against a dozen or so lists, and the features are generated without any problems. I have also not tested incorporating forms that are changed with the Sharepoint designer. This will be the topic for one of my next posts.

I think that generators should provide as much extensibility points as possible to be of real value in a product line. Although not optimal at this time, you'll be able to adjust/influence generation in the following ways (roughly from minor to major effort):

• Adjusting the list within the MOSS Gui, like creating views, altering sort-orders etcetera;
• Adjust attributes from the Feature Generator Gui. This includes the major attributes like name, description, FeatureID, and also paths/urls that are defined in the schema.xml;
• Adjusting forms in Sharepoint Designer;
• Altering generated files by hand (the painstaking process you know so well).
• Altering the code in the generator by using Visual Studio;

Below are the steps neede to create a list from the Gui in MOSS and then generate it as a feature.

1 Create a list in the MOSS GUI

2 Customize the list

In the settings page for a list you may add new columns, change their order or create new views. In the General settings, you can change the title and description, the versioning settings and the content types for the list (first allow content types under advanced settings). Changes to items under the Permissions and Management and the Communication sections are not supported at this time.

3 Select creating a New List  Template

Select File, then New, then List Feature

.

4 Select a List Instance

Paste the URL to your top level site in the SiteCollection Url box. Then Select one of the sites in the Site dropdown.

From the Lists, select the instance for which you want to create a feature.

5 Create the Feature files

The properties of the selected list will now appear in the List Feature Creation Form. You can alter Title, Description, Scope and Version.

For the FeatureID you have a few options. By default, a new GUID is generated for the FeatureID. If you want to keep the original FeatureID, uncheck the Create New Feature Checkbox. If you want to control the creation of the GUID yourself (if you need the GUID in other features, for example) you can do so by pressing the New button to the right of the FeatureID textbox.

In the middle section, you can customize the creation of the schema a little. To do this, check the Override Defaults CheckBox. Note that this section does not show the attributes of the List instance, because the values provide in a List instance are not valid for a schema.xml for a List Template.

The defaults in this section will create a Folder with the same name as the the rootfolder for the feature. The Schema.xml and the Forms will be created in this directory.

You can't change the document template, fields or contents types at this time. It is assumed that you do this in the MOSS GUI. In fact, if the properties are not what you would have liked, you can change the list in the MOSS GUI, and return to step 4. All your changes will be visible right away.

Finally, change the path, and press Create Features. In the specified direction you will now find you Feature.xml, Elements.xml and in the subfolder below, a schema.xml, and files for manipulating the items and the views. This typically includes an AllItems.aspx, a DispForm.aspx, an EditForm.aspx and a NewForm.aspx, and an extra form for each view you added in the MOSS GUI.

Wrap up

I do not provide extensive customization options in the MOSS Feature Generator Gui, since the idea of it all is that you will do the editing withing the MOSS GUI.

 Since I have now generators for Content Types (Columns included) and List Templates, I changed the solution to support both. And ofcourse I had to rename Content Type Feature Creator to something more generic like MOSS Feature Generator.

In the next few days I'll be concentrating on the integration of the features. A problem at this time is that the generation of the Fields and the ListTemplate is not related. So you will have to adjust FeatureID's by hand when you want to export both Fields and ListTemplate.

A dozen of lists to test is not nearly enough. So if you have functionality that is not supported (or downright errors), please drop me a line. I think that, if we combine forces, we should be able to drastically reduce development time for Sharepoint 2007 solutions, making it an even better alternative for customers. In the attachment ( the rather tiny link below) you will find the executable. I'll put the Visual Studio solution in my next post. The attachment, by the way, will only appear if you select this article, not if you view this article in the blog.

Alfred 

Features for Content Types and Site Columns can be quickly generated by using CTFC. There are basically two main steps involved:

1. Create features by using the MOSS 2007 User Interface
2. Generate features based on what I had created.

Please note however that in this version (1.1) is not tested rigidly. It is more a prove of concept, to demonstrate and evaluate the possibilities and limitations.

I will now explain the steps in a little more detail.

1       Create features by using the MOSS 2007 User Interface

This is the process by which a power user might create a user from the MOSS 2007 user interface. A number of different options exist for creating the Content Type,  some of which are supported out of the box, and some are not.

Supported are the inheritance of existing Content Types, and the usage of newly created Custom Site columns, for which a separate feature will be generated.

Some other advanced properties have not been tested, for instance the coupling to a workflow.

The creation of custom site columns and content types in the user interface is described by a number of people, for instance Shady KhorShed at http://shadiesm.blogspot.com/2007/11/moss-create-site-column.html, or Jan Tielens at http://weblogs.asp.net/jan/archive/2006/05/09/445763.aspx

2 Generate features based on what I had created.

The generation of features from the stuff you created in the MOSS user interface involves two steps.

Database form

In the database form, data for columns and content types is retrieved from the database by following steps:

• a) Specify the path to your content database, You can find this path by opening the Central Administration, selecting Application Management, then "Create New Web Application" and finding the server name. This is a bit counterintuitive, but you can not find the server by inspecting the existing web application(s). Then, again from the Application Management tab, select Content Databases. Select the web application, and you'll get the content database(s). If you provided logical names, you'll see here which one is your target.
• b) Press "Load", and if your connectionstring was OK, you'll see content types and columns you created in step 1, appear in the lists.
• c) Select the content type for which you want to create a feature and press the Next button. The feature form will now open.

Feature form

In the features form, you see the properties of both the custom columns and the content type features.

• a) Optionally you may change the settings supplied;
• b) Select a path to create the feature to
• c) Press the Create Features button and the features will be created to the specified path

 In the attachment you will find the executable program, help file and sources

 Alfred

Update: In my new post, I show you a new tool for both the creation of features for content types and columns as for lists

 Alfred

I was looking for information on the generation of features, but I could not find any information yet. Splendid! Good start for some coding. What I wanted to do is:

1. Create features by using the MOSS 2007 User Interface
2. Generate features based on what I had created.

By the way, if you have never created a Content Type feature, it would help you understand the text below to glance over how its done, for example by reading this post by Ton Stegeman: http://www.sharepointblogs.com/tonstegeman/archive/2006/07/19/creating-contenttypes-in-sharepoint-2007-by-using-a-feature.aspx

Using custom site columns and custom content types as a start, it appeared to be possible to decrease development time significantly. I've not gone through a formal testing process, and I needed some hacks. So I expect that some more issues will come up. But I do not think I'll ever code columns or content type features from scratch again. It's also a handy tool for to study more complex content types like those inheriting from other content types.

Looking in the content database, I noticed that with a "select definition from ContentTypes where definition is not null" I could retrieve both columns I had create in the MOSS user interface, and content types I had created there. I was a little surprised that there was actually XML in the table, resembling closely the XML I used to type in to create my custom site columns and custom content types.It was easy enough to grab the data from the content database and generate feature.xml and elements.xml files from it (never try to write to the content database!). The files thus generated however refused to be activated. I had to tidy up the XML to get them working.

Well to describe what I did in a more organized manner:

1 Use Sql Server to retrieve the URL for the MOSS content database.

I used this connectionstring: "Data Source=VPCWIN2003R2;Initial Catalog=WSS_Content_8080;Integrated Security=SSPI";

2 Read the relevant data from the database, and fill two lists

Since both columns and content types are in the same table, I filtered them based on thier element name.

                SqlCommand cmd = new SqlCommand("select definition from ContentTypes where definition is not null", connection);
                SqlDataReader rdr = cmd.ExecuteReader();

                while (rdr.Read())
                {
                    string xmlString = (string)rdr[0];

                    string nodeType = XmlHelper.GetElementName(xmlString);
                    if (nodeType.Equals("Field"))
                    {
                        lstColumns.Items.Add(xmlString);
                    }
                    else
                    {
                        this.lstContentTypes.Items.Add(xmlString);
                    }
                }

3 Create a feature.xml file for the columns 

I used values I keyed in from a form to generate a valid feature.xml (more on the little tool I created below)

4 Create an elements.xml file for the columns

From all columns I created in the Moss User Interface, I had to filter those relevant for the selected content type. For this I could just iterate over the the <FieldRef> elements in the content type and verify select the column if there was a match.

         //returns the columns that are relevant for the content type
        private static string GetRelevantFields(string contentType, string[] columns)
        {
            string retVal = "";
            XPathNodeIterator it = GetRootNodeIteratorFromString(contentType);
            XPathNodeIterator it2 = it.Current.SelectDescendants("FieldRef","",false);
            while (it2.MoveNext())
            {
                string id=it2.Current.GetAttribute("ID", "");
                for (int i = 0; i < columns.Length; i++)
                {
                    if (columns.Contains(id))
                    {
                        string fieldString = columns.Insert(6, " xmlns = \"http://schemas.microsoft.com/sharepoint/\"");
                        retVal += fieldString;
                    }
                }
            }
            return retVal;
        }

Unfortunately, when I added the XML thus obtained to my elements.xml file, I received an error when installing the feature. It appeared that, when you do not add a xml namespace, an empty one is added to the xml when you add the node. That is why I had to insert the right namespace to the columns, before adding them to the return value.

 5 Create the content type feature.xml file

This was equivalent to the columns feature.xml file

6 Create the content type elements.xml file

This would have been just the creation of a xml file from the data in the content database, but then there proved to be some issues:

a) An empty namespace was added, as decribed in (4). I added a namespace as a property (I have not been doing XML in code for some years, there'll be a more elegant way, I bet)

b) For content types that derived from the Page content type, or a sub type of Page, a number of attributes were introduced with empty values (Customization, PIAttribute, PITarget, to name some). I did not understand these properties, so I removed them by using a regular expression

c) There seemed to be a problem with the xml of the <XmlDocument> nodes, so I removed them by using a regular expression

7 Install features

I copied the feature directories to Features folder and installed them by using stsadm as described on http://msdn2.microsoft.com/en-us/library/ms442691.aspx. I have not automated this part yet.

I tried a dozen or so content types, which could all be created and activated. I did not try using each and every column or content type from the User Interface, so just consider this a prove of concept. 

Since increasing productivity of MOSS solutions is one of my main interests, any feedback would be welcome.

Alfred