Introduction – Review of Part 1

Wow, nearly a year without writing, time goes way too fast!  In the first article, the objectives were to properly explain the high level terms that are related in deploying SharePoint in the field: Enterprise Content Management, Document Management, Collaboration, Records Management, eDiscovery,and Web Content Management.  It is necessary to understand these concepts as they directly relate to what you are trying to do with SharePoint. 

While it is typical to have multiple products, by understanding how each of these areas interact between each other is critical as you will have to either integrate, or migrate.

The goal isn’t about technology, it’s about defining the right scenarios and focusing on bringing solutions to these – this will equal to adoption.

Alright, I get it, Scenarios, Scenarios, Scenarios.  But what are the SharePoint workloads?

There’s a lot of viewpoints for this, but the typical ones are about Collaboration, Enterprise Content Management, Social, Search, Web Content Management, and Business Intelligence.  Let me try to break these down a bit.

Collaboration – This is about capturing all sorts of information right up from the end-users.  This can be in various formats: documents, micro-blogging questions & answers, emails, structured web pages (i.e.: page layouts), unstructured web pages (wiki), opinions through blogs, forums, etc.  Remember: think scenarios!  When elaborating collaboration scenarios, they tend to be split between Team-Based Collaboration, and Document-Centric Collaboration (often referred in ECM, or as a subset in ECM).

Enterprise Content Management – This is your large document stores.  In my experience, this essentially come in 2 types of usage with SharePoint.  (1) As a document center for some document-centric departments (HR, Legal, Finance) where these end-users are working directly in these workspaces for storing and finding back documents.  (2) as an archive where you’d like to massively store documents with no collaboration and limited read operations.  These files may be uploaded by a system (i.e.: financial/insurance documents), or through a process for when collaboration has ended on some document types.

Social – This is an expanding one.  Today, this is about enterprise micro-blogging, colleagues and following them, finding expertise, and following communities.  This is impacting how we build these communities, forums, and even simple things such as Questions & Answers.  The main goal is to allow for people to quickly share information, and to have a broader reach for getting answers.

Search – Most people defines search as that little textbox that allows you to perform keyword searches.  While this is certainly true, there’s more to this than meets the eyes.  With proper planning, you can create search-driven applications. For example, why would potentially create an HR resume Document Center along with a Finance PO Document Center; compared to, say, a Record/Document Center containing both?  The reason would likely be because your usage scenarios are very different and driven by customer requirements – automatic search-driven pages makes it more compelling for end-users to adopt the system.  Enterprise Search is also more than simply crawling all your data sources – it’s about making relations between them and about automatically categorizing the information (automatic metadata entry based on patterns).

Web Content Management – This is your typical web site for broad audience.  However, trends are changing this space.  For the public Internet areas, social is impacting these web sites.  You may need to allow people to provide feedback and respond to them, it could be end-user tagging & ratings.  For intranet sites, social and collaboration are impacting what you need to present as your end-users requires to have centralized dashboards for both the enterprise/CEO messages, along with their tasks, community messages, etc.

Business Intelligence – BI is a world in itself and is completely driven by usage scenarios.  SharePoint 2010 offers different BI capabilities that can meet most, if not all, of the end-user scenarios through the help of SharePoint Designer, PerformancePoint, Excel, and other surrounding products and technologies.

Define your general enterprise strategies

Collaboration Strategy

Collaboration is typically where most of your information entry will be done – unless the information is coming in from automated means (scans, faxes, etc.).  The important part here is that there’s an end-user involved and adoption will be determined on how easy and intuitive it is for that user.  A good collaboration strategy starting point is to address what kind of initial solutions will be provided (team sites, project team sites with specific metadata, meeting boards, etc.)  -- this isn’t so much about features but rather high-level generic end-user solutions.

The 2nd element I’d strongly suggest is the surrounding services framework such as providing a solution for Site metadata.  This provides a good solution to find back sites, but also to automatically tag information based on these site metadata.  I also look at providing a framework for communicating change back to users, such as new solutions or features, upcoming maintenances, etc.

3rd, I’d include some sort of collaboration portal which will act as the starting point for creating a collaboration workspace, providing information on how to use the platform and what the solutions are, what the policies are, etc.  This area must also provide an area for end-users to provide feedback and request assistance.  If there are business requirements leading to it, it’s okay to have templates for “Generic Project Site”, “IT Project Site”, “Marketing Project Site” – the reason for this is that they may have different types of content, policies, workflows, and metadata associated to them.  By doing this, you make it easier for end-users to comply to the corporate policies and making it easier for them to use the system.

Last, I’d consider levering the site metadata solution (#1) to include site security classification.  At the site creation, with simple questions regarding what the site may contain, you could automatically apply infrastructure security parameters depending on whether it will host (for example) public, secret, or top secret information.  Your security compliance requirements may define that top secret information be on a segregated network, uses encrypted database, and requires a 2-form authentication.  Also, you may need to tag all information (in the records management scenarios) with confidentiality parameters – by doing it at the site level, you could automatically tag all the information without requiring end-user input.

Collaboration is where SharePoint has excelled and brought awareness to end-users of the other workloads.  Its integration point with other Microsoft points are numerous and essential to end-user adoption.  This is also where we have a typical gap with traditional ECM products that are mostly document-centric collaboration, often without an intuitive interface for most end-users.

Note that SharePoint is designed for your to maximize the number of Site Collections, but also encourage you to have some sort of site collection retentions.  Putting it simply, create a Site Collection for a specific but unique purpose.  For example, you would create a Site Collection for each IT projects, and maybe another Site Collection for the IT Department which would list all of their projects through site metadata and search.  Each projects have a different site collection because they have a different retention than the next project, different members and security, quotas, etc.  You could share information policies and content type through the Content Type Hub for example.

However, You would not create an IT site collection with sub-sites for each project as this would make it much more difficult to maintain and operate.

ECM and Records Management

Your general ECM strategy must define the high level flows of information, compliancy requirements, and information security requirements.  Note that this isn’t just for documents but for all types of information (paper, IM, social, etc.).  The following could be an example of a high-level information flow for a company using SharePoint, SAP, Documentum, and another tool for archival:

Records Management is your typical deep tree (folders) of documents/information kept for retention and compliancy purposes.  The scenarios are typically “record keeper driven”.  If all end-users have access, it’s likely through search.  The goal is not to have end-users enter the content manually there – they generally get lost in the folder structure and that defeats the purpose. 

Instead, the document is routed (automatically for known content, at the right time; or manually for selected content by end-users) to a drop folder which, based on metadata rules, will move the document to the right folder.  This ensures that the record center is kept optimal for storage and adequate performance – while following the compliance requirements, without impacting end-user adoption.

SharePoint provides this capability with the “Send to” configurations which can be defined per Web Applications, and attached on content types and/or information policies.  This “Send To” effectively provides the capability to Copy, Move, or Move and leave a link.  The later automatically tags the document with a unique identifier providing access to the document from anywhere (transparently provided by the SharePoint/FAST search engine).

While this warrant a specific post on just that subject (and I’ll try to do it soon, promise!), integration between ECM products must be looked at properly.  Of course, all of them claims to integrate with SharePoint, but that depends on your interpretation of what integrating means.  In my opinion, Web Parts are NOT integration (it’s more like a window to looking outside, or to some other black box).  While they can fulfill some scenarios, they do not constitute a proper end-user integration. 

From a high level view, if you want SharePoint as the front-end and also have other ECM products such as OpenText LiveLink or EMC Documentum, you cannot simply state that ‘all document will be in Documentum’, install web parts or one of their respective deeper integration product, and be done with it.  Before integrating, you’ll have to define what and when integration will occur.  By far, these products are mostly designed as document-centric collaboration and/or for retention archives.  The challenge in integrating these will be on the configuration of this what and how – this is no easy feat.  In fact, typically, when information stems from end-user driven collaboration, it’s easier to use the SharePoint “Send To” features to send it to the SharePoint Record Center (using the drop folder, or using the content organizer altogether) in between, and then integrate that with the other ECM product.  Of course, you’ll have to ask yourself why you are using 2 tools at that point – but, in my experience, it’s been (by very far) the easiest/lowest impact to the end-user/cheapest way to integrate them.

As a final note regarding ECM integration, for this post, it should be driven by the archival users requirements, not specifically by IT.  Compliance requirements starts before the information is entered through SharePoint so you will likely have to define the information rules both in SharePoint and that other tool.  If you have end-user requirements, they will be done through SharePoint and most of these do not work well if SharePoint is not the back-end.  Be very careful, complete end-to-end integration with other ECM products is not easy, if possible.  It is possible that future implementation of CMIS may help, but it's no silver bullet today.

ECM integration, in my opinion at this time, should be kept for specific departments that already have document-centric collaboration scenarios with that other ECM product.  For example, Finance already uses Documentum and would like to have SharePoint as a front-end and for other features.  You could then add a My Documentum for SharePoint Web Part to represent that segment of Documentum in the Finance site collection’s home page.  Features such as Office Web Apps, SharePoint alerts, SharePoint workflows, and SharePoint permissions would not apply for the information presented by that web part.

Lifecyles

With SharePoint, you have to look into both the Site Lifecycles, Document Lifecycles, and Information Lifecycles (not everything’s a document).  Here’s an example:

This describe a process on which site owners must renew the site for continued use.  The process should also define, based on the solution domains and the types of information that they contain, if and how the important information are moved to a longer term storage (i.e.: Document Center –ish solution).  At the latest, this would need to occur automatically prior to a site being deleted.

For example, the structured information will get routed automatically to the right location.  On the other hand, you have a choice regarding unstructured information – whether you allow them to be deleted, or whether you move the last version of that document to a specific ‘unstructured’ container of your record manager, which would have a generic retention but would also allow for a process to be define so that owners are notified to tag the information at that time – if necessary.

Note that it’s impossible to structure all information from the beginning (if ever) – you need to provide guidance on what you will do with the unstructured information.  No matter what, if you aren’t completely structured today, you’ll have to answer that question for any information you have today.

Search strategy

An enterprise search strategy must not only look at the repositories that must be crawled, but also how security trimming will apply.  For example, the SharePoint Indexing Connector for Documentum provides the ability to use a SQL database that will map the Documentum LDAP accounts (if not using Kerberos) to the corresponding AD account used in SharePoint.

The search strategy must also look at how results are brought back to end-users in form of ‘central search center’, contextual search, search-driven application, profile-influenced search (i.e.: FAST Search returning results based on your company role, for example).

Last, it should also look at how the indexing pipeline can be leveraged to automatically tag information on documents, based on its content.

This strategy is particularly complex when dealing with multiple different types of systems, across geographical locations (WAN), and different authentication mechanism.

Social strategy

Typically, when I mention a “Social Strategy”, the first thought from the customer is about the social policy that they have or need – which details what employees are allowed on public social sites such as Facebook and Twitter – and their corporate obligations to it.  While this is definitely true, required, and part of the social strategy, there’s more to it.

It’s not just about providing blogs and wikis capabilities, those are merely a format that might be used.

In the enterprise, this strategy must provide guidance on how to use social features and technologies to address business requirements.  For example:

• How can employees find people by their name, roles, region, expertise and skills, or even by project;
• How can employees identify information that should be tagged as an Intellectual Property (IP), possibly may even making them IP, Community IP, and Managed IP depending on how many people refers that information;
• How can employees ask questions to the right audience;
• How is a community managed, who can create them, what kind of communications are allowed, what is allowed in the community, etc.;
• Where and how information can be tagged and rated by individuals
• How do we provide a repository for any employees to enter content in a free-form format that makes it easy for them to do so, how are these maintained/approved, and if/how this employee participation can be identified, evaluated, and rewarded in some fashion.

The last is important – times are changing, employee retention is difficult and so is documentation.  Providing to much of a fixed format with large document models is cumbersome, time-consuming, and essentially not efficient – it takes too much time to find back the information and/or to read all the documents.  Think about providing some kind of enterprise Wikipedia for people to easily share information that they deem valuable, with the ability for tagging/rating by end-users.  Popularity and search (which requires planning) will make the system invaluable.

This is all about how you connect people to people and to information.

Governance

Big topic, and not one I’ll cover in details in this post.  There are several posts on this subject too.  The reason I wanted to point it out is that governance has been coined as an issue in SharePoint.  Depending on your point of view, it’s both true and false.  Does SharePoint have some governance enforcement features in it, of course.  Are there other governance enforcement features that are often needed? very likely.

The reason it’s been a problem is multi-fold: SharePoint is often installed without much of a vision and grows like most file storage system – you can’t find back the information properly, it’s a bunch of disconnected features, there’s no end-user solution, capacity planning is an afterthought, etc.  A governance is a continual process on how you glue in business requirements and technical requirements.  It’s a set of policies, procedures, and guidance to help IT build the right solutions to the end-users, and to provide the end-users the flexibility they require.  It’s a necessity now since most IT organizations are big, complex, and overwhelmed by procedures (for better and worst) that it may have lost its touch with the business.

It’s not a one time job, and one size doesn’t fit all.  The governance should outline the strategies that are mentioned in this article, and provide an easy way for end-users to request solutions.  These strategies must evolve as the business evolves.

Also, although most governance work today are separate engagements, it should be part of any project that augments the overall governance.  Personally, I built the documentation of governance deliverables in wiki-ish format, and it’s also built around stories that are meaningful to the readers – whoever they are in the organisation, from the end-user to the architects, including the developers and all roles associated with the platform.

Wrapping up Part 2

I hope this was helpful in looking how to breakdown standard installations of SharePoint.  SP2010 provides a lot of capabilities but making them useful to your business takes some planning and guidance.

Part 3 will be looking at how to integrate ECM products together in more details.

Introduction

SharePoint made its name with collaboration tools that integrates very well with other Microsoft products and is the foremost tools for user adoption.  With SharePoint 2007, Web Content Management capabilities were integrated from CMS, and it was also the start of more Document Management and Record Management functionalities.  With SharePoint 2010, it now scales to the demand of large enterprises, and brings together the value of social computing for these enterprises.  After all, the most important asset to a company are its employees, and SharePoint is all about connecting people.

Historically, companies often had separate products for collaboration, documents, records, and web content.  These tools may have integrated together, but the story was rarely great, especially during update times.  ECM was made for augmenting the findability of documents – and it started with metadata + classification. 

Enterprise-wide complete success in ECM is however somewhat mitigated.  Simply put, a person typically enters some metadata (if any) when creating the document, and won’t enter a lot of them – it’s often the default values.  The other typical challenge for classification was with asking the users which file plan number it was.  While I’m greatly oversimplifying, even that isn’t done properly for a complete enterprise.  Typical successes are for departments or small teams that typically handle records from the get-go, not for documents you iterate over and then become final.  

It’s also not intuitive for all users to work on documents on their computer or file shares, and then manually bring it to a separate store meant for “final documents only”.  In some situations, customers are using these stores for “published/official documentation”.  This makes the problem much more obvious as it’s typical for some types of documents to have multiple published versions (i.e.: you update a document for a new project phase) – users forget to update the published store.

Even if considering only SharePoint 2010, the challenge comes first in understanding these workloads and how closely they are related.  This series of post will attempt at consolidating information for a better guidance for these multi-workload scenarios, how it works with SharePoint, and how it works when integrating with other ECM products.

• Part 1, this post, will explain the the general terminology and the key components from the SharePoint platform
• Part 2 will discuss how to sort out a strategy for your multi-workload environment with sample document lifecycles
• Part 3 will discuss how SharePoint can and should integrate with other ECM products
• Part 4 will cover enterprise (SKU) areas of SharePoint, such as BI, FAST, and Project, that adds to the core values of SharePoint with no additional software costs

While we had some core elements of ECM in SharePoint 2007, we leaped ahead with SharePoint 2010.  Features like the Managed Metadata Services with its Term Stores and Content Type Hubs makes it easy and efficient to control and share taxonomies; the content organizer and multi-stage retention capabilities are efficient; and the in-place records are a new and welcomed component for business collaboration spaces.  All of this relies on a serious .NET and Workflow Foundation platform, but also with social components such as the Activity Feeds if necessary. 

If you hadn’t thought of SharePoint in the past for ECM, and more so if you struggled with implementing ECM with other products, now’s the time to take a serious look at SharePoint 2010.  Microsoft Consulting Services can definitely help you make the best use of your platform, enhance adoption, and reduce costs.

Let’s go ahead with some common definitions.

Enterprise Content Management [ECM] explained

The Association for Information and Image Management [AIIM] currently defines ECM as :

“Enterprise Content Management (ECM) is the strategies, methods and tools used to capture, manage, store, preserve, and deliver content and documents related to organizational processes. ECM tools and strategies allow the management of an organization's unstructured information, wherever that information exists.”

According to Wikipedia, it mentions that this statement was last revised in early 2010.  I’d say they are rightfully revising these statements in order to include new ways for sharing information.  Previously, ECM was defined as the technologies rather than the strategies.

In the field, part of the challenge is that companies are trying to purchase an ECM product and hope they’ll simply install it and content will be managed.  As stated, ECM is not a product, it’s a strategy which may require a few product to fulfill depending on the choices made.  That strategy must include the people factor, both from an adoption and usability perspectives.  The old saying of “These 12 metadata are all required and everyone will fill them according to their file plan” didn’t work.  It’s time to look at a strategy that will enter these information from its context with minimal user input.

Let me give you an example of contextualizing metadata for a document: you create a project workspace for a single specific project.  This project will likely define most of the information that you would have asked for each documents.  A strategy to ask this information a single time per project and automatically tag all relying documents from their first version is much more efficient.  The same goes if you need to augment with information coming from the user’s context.

Another important aspect is that ECM covers all types of content, not just document.  These can be in the form of imaging, physical records, notes, emails, voicemails, meetings, blogs/wikis/other web content, IMs, etc.  I’ve seen many implementations where companies only included documents due to product limitations.  While it’s a good start, make sure that it fulfill your ECM and eDiscovery requirements.

Document Management [DM] explained

The AIIM currently defines Document Management as :

“Document management, often referred to as Document Management Systems (DMS), is the use of a computer system and software to store, manage and track electronic documents and electronic images of paper based information captured through the use of a document scanner.”

Typical features of Document Management are :

• The ability to checkout and check-in a document ensuring integrity for authors and readers
• Version control that allows segmenting what readers and authors see, and the ability to rollback to previous versions
• The ability to audit access and changes to the information and its permissions
• The ability to qualify the information with metadata
• The ability to start simple and complex workflows for collecting feedback, approve, or start a process
• The ability to reuse document templates

Historically, the technology was made in such a way so that DM systems were an hierarchical tree based structure of folders and documents.  In my experience, these systems were mostly implemented within specific departments such as Legal and HR who had a process in place for documents.  They also typically didn’t go for collaboration or social.  There will likely always be some types of documents that won’t require social or collaboration, but as we find new ways to share and participate, these types of documents will lower quickly.

What’s important here is that there should be multiple DM repositories in any enterprise.  Some will be collaborative, some won’t.  Your Solution Architecture will determine the repositories you will have such as collaborative, financial statements, audio/video archives, and web content (these are merely examples).

Collaboration and Social explained

The AIIM currently defines Collaboration as :

“Collaboration is a working practice whereby individuals work together to a common purpose to achieve business benefit.”

I think this statement is vague on purpose.  Collaboration assists users in working together on content and deliverables.  The final deliverable is typically either web content or a document, but the collaboration features also captures the other shared information during the process.  The key thing here is that Collaboration is scenario focused.  Rolling out SharePoint for blank Team Site is typically not responding to a scenario.  Look for examples like these:

• You have a meeting workspace site that integrates with Microsoft Lync to deliver the presentation for remote workers.  The meeting clerk takes notes using OneNote that connects to the meeting information and appends the notes Live.  Meeting attendees can review these notes with their Windows Phone 7 and apply changes.  Finally, the clerk finalizes these changes and creates a Meeting Minutes document.
• Your colleague finalized a document and posted it in a workspace.  You notice that your colleague added that document through the activity feeds and, after reviewing it, you assign it a rating that will affect search relevancy.
• You have a set of company policies defined in a web content management system.  You allow users to enter comments on these policies which facilitate the discussion the communication with employees, allows them to post notes on these policies which help the HR team gauge the employees feedback.
• You post a question through the activity feed, someone who’s defined as an expert in the subject of which you asked the question responds to you through his phone.  This response is then elevated as a format Q&A element in that subject’s community site.
• Your job is to respond to media queries and, when such a query pops up (through something like this morning’s newspaper), you create a team site where you aggregate information from all your content sources in order to create an official media response.

As you can see from these samples, Documentation Management is really a part of collaboration.  After all,  checkout/check-in, version control, metadata, and workflows are all form of collaboration – BUT so are others.  This is important when you consider integrating other ECM products as the way they do this integration will impact your collaboration and social capabilities – or it will affect the user experience, and thus, adoption and employee retention.

Social can be viewed as an extension of collaboration functionalities where we move towards openness and freedom for users to author content.  Quoting the AIIM again, social software should have :

• Search: allow users to search for other users or content
• Links: group similar users or content together
• Authoring: include blogs and wikis
• Tags: allow users to tag content
• Extensions: recommendations of users or content based on profile
• Signals: allow people to subscribe to users or content with RSS feeds

…and…

• Freeform: no barriers to authorship, i.e. free from a learning curve or restrictions.
• Network-oriented: all content must be Web-addressable.
• Social: stresses transparency (to access), diversity (in content and community members) and openness (to structure)
• Emergence: must provide approaches that detect and leverage the collective wisdom of the community.

Personally, I’m buying this as I can see where content is more up to date when it’s freeform.  Historically, with documents, we have to rely on the few individuals who were making these documents and they tend to be too busy.  While it’s open, there are still controls in place before approving an update, but this allows for people who are expert in a specific section of a document to contribute there.  It also allows people who identifies an error (such as something that isn’t true anymore) to complement the information.  When working with documents only, you either don’t have the ability to change this, or you’d have to go through the author who may not be there anymore or who may be too busy to update the documentation.

SharePoint Social capabilities are ultimately the My Sites coupled with the Activity Feeds and Tags.  Think of the Activity Feeds as a mix of Facebook wall and Twitter – with a business purpose.  The core framework is very solid.  You can look at NewsGator’s Social Sites for SharePoint 2010 to see what you can do with the out of the box framework but with a much improved GUI.  They merge the activity feed with community sites and allow for asking questions to experts – and so much more.

In content management, Social can also take the form of your company’s information on networks such as Facebook and Twitter – you may (probably) need to be there, but that content also needs preparation and be recorded.  Your ECM strategy will eventually have to plan this ahead and define a way to both be legally compliant and also fluid enough so that you can respond fast on these external networks.

Record Management [RM] explained

The AIIM currently defines Records Management as :

“Enables an enterprise to assign a specific life cycle to individual pieces of corporate information from creation, receipt, maintenance, and use to the ultimate disposition of records. A record is not necessarily the same as a document. All documents are potential records, but not vice versa. A record is essential for the business; documents are containers of "working information." Records are documents with evidentiary value.”

Historically, Document and Record Management have been done together since the hierarchical tree based structure of folders and documents was a convenient way to apply records directly.  Products were made as such and you can get a 2 in 1 easily.  However, User Adoption and proper metadata entries have been a big challenge for these solutions.

What I particularly like about the definition is that “Records are documents with evidentiary value”.  I’ve often been confronted with customers that believe that all documents are records.  When you engage in the discussion and show examples of documents that are likely not a potential record (i.e.: draft documents, the document you create for some office event like Christmas, etc.), they’ll change to all ‘enterprise documents’ with or without a clear indication on how/when a document becomes “enterprise”.  Things can turn in circles for quite some time.

In the end, a record starts with a human intervention.  You define it as a final document by setting a property/metadata, by right-clicking on it, etc.  You might be able to automate to some level, like automatically making all last published versions of documents in a project workspace as records when you close the project.  The action of closing the project is likely manual as well, but it could be automated through the project end date.  Last, you may have repositories that only contains records and thus, the manual action is that the user is navigating there to drop the document (he likely created/collaborated on the document elsewhere) – unless it’s an automated system that put the document there.

On a more practical level, you would define the document templates for a project workspace and get elements such as :

• Unspecified document types are to be kept for the project duration + n years.
• Documents of type Meeting Minutes are kept for n years
• Documents of types “Envisioning”, “Planning”, “Designing”, “Architecture”, “…” are kept as records with the workspace for the workspace duration then moved to the archive and kept for a total of n years
• …

By planning ahead, you can design your workspaces so that it’s automated for users.  When going in for a meeting’s workspace, if they create a new document, it’s automatically a Meeting Minute document and will inherit metadata from the site and the meeting (after all, you are creating the meeting for specific reasons which will likely translate to some metadata for the meeting minutes).  All of the official document types are readily available in the Project Documentation library.  Site Owners may add other document libraries for their collaboration requirements, and these will fall under the “Unspecified document types catch all”.

The nice thing about this quick sample is that all social capabilities are enabled when interacting with information and documents at the project level.  This is where it has values from a people’s perspective.  When the official documentation becomes a record, it stays in the workspace and shouldn’t be moved right away – this helps on the user side as they are still working on the project and want a single place for all of the project’s documentation.  The project’s collaboration workspace will eventually close out.  It’ll likely become read-only when the project ends, and may simply be deleted a few years after the project end.  All official documents will have been moved at the archive before the site’s deletion.

Records Management isn’t an IT project.  It sounds silly, but I’ve seen it.  While IT facilitates the technology piece, and may have some sort of architect role close to information management, but it needs to be business driven with the legal staff.  To make it possible, you’ll probably have a general strategy and then break down in multiple small projects to define the policies.  The key factor being smaller projects – trying to do a whole large enterprise in a single shot doesn’t work.  While IT will still drive the general technology platform, each business units must drive their own information requirements (with possibly the help of architects coming from the IT department).

Records Management has always been about policies, but one of its big requirements in the past was for findability, or rather the ability to find back published documentation.  Search engines have evolved in such a way that findability is not as much a requirement for Records Management, but eDiscovery is now a major driver for RM, and that is our next subject.

eDiscovery explained

Citing the AIIM again, eDiscovery is defined as :

Discovery is the term used for the initial phase of litigation where the parties in a dispute are required to provide each other relevant information and records, along with all other evidence related to the case.

As such, eDiscovery is for all electronic information, not just documents.  As I mentioned in the ECM section, it can take many forms such as imaging, physical records, notes, emails, voicemails, meetings, blogs/wikis/other web content, IMs, etc..  This is another reason why trying to take a document-only strategy will not lead to success. 

At this level, SharePoint 2010 is designed to support compliance but may or may not be your main tool for eDiscovery.  This will depend on your email/IM archival strategy (amongst other things).  You do have the ability to store the archives of these systems closely with SharePoint so that you can lead eDiscovery with SharePoint, but you should define your strategy first.  How do your legal council work in these events? should they hold the records directly and/or should they copy what they found in a specialized team site for the litigation? How do they identify records?  How do they audit and report back the actions on records?

This article provides starting information on Records & eDiscovery for SharePoint: https://msdn.microsoft.com/en-us/library/ee557329.aspx.

Publishing, or Web content, explained

When SharePoint 2007 came out, it had merged the ‘old’ Microsoft Content Management Server (MCMS) in the platform and it was then called the Publishing features.  Typical SharePoint environments in the past have been mostly using either Publishing or Collaboration features, and often in separate environments even for Intranets.  SharePoint 2010 democratized the Publishing features by making its use much more convenient – it’s approach is close to that of Wikis.  Let me break down what the Publishing features contain:

• The ability to checkout and check-in web content ensuring integrity for authors and readers
• Version control that allows segmenting what readers and authors see, and the ability to rollback to previous versions
• The ability to audit access and changes to the information and its permissions
• The ability to qualify the information with metadata
• The ability to start simple and complex workflows for collecting feedback, approve, or start a process
• The ability to reuse web templates
• The ability to have an authoring optimized environment, and a visitor optimized environment with synchronization of content between the 2
• The ability to have the same page in multiple languages
• Web Analytics
• The ability to have dynamic queries

If you go back to my “Document Management explained” section, you will notice that the first few items are exactly the same but refers to Web content rather than Document.  That’s because the only difference is the format presented to the user and that’s all it should be for the most part. 

Since publishing features are just like documents but in web format, plan ahead!  Ask yourself how your information should be consumed!  If all the users are going to use these on a web portal, don’t go creating documents just to transform them – it’s more complex and doesn’t add value.  With SharePoint 2010, you can make this web content as records just as much as documents.

And looking at social features, they all have a web content piece.  The new thing is that they are also integrated with our devices such as smartphones.  I can definitely see a future for an enterprise Twitter-like capability that could be used from your business productivity suite (Office), the web, your IM (Lync), and SharePoint.  Publishing merged with social features definitely still has a future!

What happens with these wikis, blogs, and notes?

Web Content can also be in other format such as blogs, wikis, or even a simple note board – at some level, they are all essentially a textbox that allows you to enter content.  This can lead to some interesting discussions with large enterprises that haven’t explored Web 2.0.  I’ve seen customers that added blogs in a hurry because “it’s popular and we have to be social”, but weren’t accepting comments!

• Blogs are web content articles allowing interaction between the author and readers through comments.  At a minimum, it provides capabilities for querying archives by months, and supports RSS for syndication with other devices.  Blogs are typically from a personal perspective, but it can have a different meaning such as a product blog or a team blog.
  • If you have a blog, you should allow comments, and it’s expected that the author responds to them as well!
• Wikis are when you open up authoring to a large audience, it could even be the whole enterprise that are allowed to update content.  A common misconception is that wikis cannot be as structured as document management.  That is not true, with SharePoint 2010, you can apply policies and workflows on wiki pages just as much as documents.  For example, you can define a content owner that needs to approve changes before it’s seen by everyone else.
  • Enterprise Wikis in SharePoint 2010 are actually based on the publishing features and merges the idea of a few content field and a general ‘free’ content area found in wiki pages.
  • The nuance in this general free content area is that it’s up to the user to define the desired structure.  While discomforting to some, it’s actually the goal as it allows everyone to participate in their own way.  It’s better to have content detailed in some form, than have a super structured documentation that isn’t there because people don’t like the structure or cannot take the time to fit in.
  • The term Wiki can and should be used differently for different solution.  You may have an enterprise wiki for official information which may have an approval workflow for page creation, and content owners for each pages.  You may have a wiki library in each project team sites so that people can log information as they go, this wiki’s lifecycle will be part of the site lifecycle only.

In short, you should have a strategy on how blogs & wikis are used and presented, not really on how to block it.  They are a format, not a productivity enemy, and in fact, they are a productivity helper.  In the past years, some more conservative companies were adamant about blocking blogs & wikis because employees shouldn’t write about personal stuff.  The interesting part is that those employees are intelligent enough to write personal views on company policies or products in their company blog environment;  if they want to share their week-end, they’ll go on Facebook, they won’t do it on the company blog.  The 2 primary reasons for that is that it’s identifiable and also because people want to write so that they can either be recognized or can make things better.

How SharePoint changed the game

SharePoint 2007’s features were superb and came out at a great timing.  Most document management systems were used for either specific departments or where used as the archive where employees would have to manually copy their files there when they deemed it final.  This left a vortex of files stored massively on file shares which offer limited (any?) features.  These files are unmanaged and have been piling in without retention forever.

With SharePoint’s ability to easily create workspaces as needed, it was a slam dunk.  Integration with the Office client was built-in without a separate deployment was a big plus.  And making all of this available through a web interface made it convenient.  Traditional content management systems were sure to try to point out where SharePoint was missing features, or how theirs was better, but the reality is that SharePoint is a more than a good-enough all-around platform.

SharePoint’s integration is top-notch and is a key figure with Microsoft infrastructure.  Active Directory? check.  Exchange? check.  ForeFront Security? check.  Configuration Manager? check.  Monitoring? Check.  Data Protection? check.  Lync? check.  Office? check..  ForeFront Identity Management? check.  Beyond that, they make AD, Exchange, Office, and Lync more relevant!  All companies struggle with maintaining employee information (the one where the employee should update) – SharePoint does it and talks back to the AD store.  You need to manage distribution lists for communities? plug in with Exchange and ForeFront Identity Manager and you got it on all front.  You collaborate and it goes to your activity feed which can be viewed through Lync or Outlook! 

This is definitely a better together story.  SharePoint connects people, not just information.  That’s where it won in user adoption, and that’s where the competition has been lacking.  SharePoint is also an all-around platform, not just for a niche.  Can you find a better product in any particular aspect for SharePoint? probably and you can go ahead and debate what is still missing anyway.  The reality is that products shouldn’t be thought in silos anymore.  Integration costs are staggering and rarely offer what you want.

The last ECM magic quadrant from Gartner (https://www.gartner.com/technology/media-products/reprints/microsoft/vol14/article8/article8.html) shows how much Microsoft has evolved in this sector:

I hear SharePoint is severely limited, is that so?

Ahh the famous 100GB limit or 2,000 list limit.  Newsflash: these were hard-coded limits, or break points.  While they were recommended software boundaries or supported and tested limits, they should be thought in context.  First off, the 100 GB limit, it’s a recommendation per content database for 2007 due to recovery times.  With 2010, the current limits are at 100GB per site collection, 200GB per content database, and 1TB limit for mostly-read scenarios. (i.e.: Record Center)  If you need a single site with more data, consider using RBS and plan your disaster recovery accordingly.

2nd, the 2,000 limit.  That’s now 5,000 by default.  This isn’t the limit in a list/library, it’s per view!  This is simply because you will start having performance issues at some point if you keep retrieving all your data.  You wouldn’t see these issues with a traditional system simply because either you didn’t have enough data, or you were able to plan ahead and make sure you weren’t retrieving too many items.  With SharePoint, the end-users have more flexibility and it comes with responsibilities.  Unfortunately, some companies didn’t plan ahead and let their users do to much.  This can be easily mitigated through rapid training and governance.

The actual physical limits are more difficult to discern because there are many facets to them.  For example, you won’t design your repository the same way for a video archive, financial statements, or project documents – they don’t have the same size and thus a different limit will be in play.  For file size, you can consider RBS, multiple data files in SQL, or even the Content Organizer to distribute the load across multiple site collections (it could even be a tenant).  I’m confident that SharePoint can play in the 100s of TeraBytes of content – you just need a lot more planning for that size than for a WCM portal!

Like any system, SharePoint should be planned.  Governance should be done and re-evaluated on a regular basis as new requirements are identified.  Active monitoring should be done to assess the environment and identify upcoming pain points before they happen.  Companies have to find the right mix between flexibility and control by providing capabilities to the end users, but also sufficient boundaries and training so that it won’t negatively affect other users.

SharePoint can definitely handle larger ECM implementations – but it requires proper planning and adequate resources, just as you would with any other software providing ECM capabilities.  The challenge has been that SharePoint is easier for anyone to use, and often with little governance and monitoring, as such, problems are being identified late and are more difficult to resolve.  Also, the Product Group has been very transparent with articles such as the Software Boundaries and Enterprise Content Storage Planning that describes the supported limits.  ALL products have limits somewhere, Microsoft simply chose to write them in order to make it easier to design your solution.  I’m confident that the Product Group will always look at ways to augment these supported limits.

SharePoint isn’t just a product, it is a platform with a strong ecosystem of partners

If you are missing a small piece for a key organization requirement, there is likely a 3rd party that adds to SharePoint due to its rich platform.  Competitors will argue that they have this built-in.  To be honest, the reality of content management systems is that they are a party of merged companies.  A lot of these acquisitions may have changed under a same branding umbrella but they aren’t fully integrated yet neither. 

For example, look at OpenText, one of the strong ECM player, they had LiveLink which is now rebranded as OpenText ECM Suite, but also includes the other Hummingbird, Captaris, RedDot and Artesia in it.  These were either competitors or add-on software that were merged in their ECM suite.  When you want different segments of their ECM suites, the cost goes with it too.  The challenge comes with making sense of the licensing and patching – it’s not easy.

That’s what I find interesting because 3rd party companies adding to SharePoint will adhere to best practices from the get go and will look just as much as being part of the product.  They have to do this to stay relevant as the SharePoint platform evolves.  When looking at augmenting SharePoint features, the buy vs build debate is important.  There are some very good partners for things like workflows, social, governance reporting, metadata driven permissions, and many more.  They all share one thing in common, they augment SharePoint through what its underlying platform offers – that is definitely more integrated than a product acquisition sharing the same name.  It also shows how rich that underlying platform (.NET and Workflow Foundation) is. 

And last, it also allows for these expert partners to update more frequently as necessary.  Take the Social partner, NewsGator for example, they can make 10-12 releases between each SharePoint release.  It’s relevant to them as the Social space changes rapidly (Twitter had 400k tweets per quarter 3.5 years ago, it had 65 million per day in June 2010 – back when SharePoint 2010 started planning, Twitter wasn’t much on the radar).  SharePoint did create an awesome Activity Feed framework though, and NewsGator filled the GUI gap and will continue evolving it.  The same goes for other things like RBS or Workflows.

Final words for Part 1

I hope you found this article practical for defining the general concepts on ECM & SharePoint.  SharePoint 2010 is a great product and will even be better in the ECM space with future updates and releases.  It combines both the ease of use, adoption, and strong ECM capabilities – and it also brings a solid Microsoft (Office, Active Directory, Windows, etc.) and partner ecosystem.

The upcoming parts will discuss how to plan these multi-workload environments, how we integrate with other ECM products, and what additional value the Enterprise SharePoint offering gives you with no additional software cost.

I’ve been trying out the Balsamiq mock trial tool and ended playing around with it for a presentation on options for site metadata for SharePoint.  So here’s what the breakdown is (and you can download the whole thing in a single PNG by clicking on it).

Here’s the breakdown (in my own SharePoint in pictures way!)

Option #1 : Central list

Note that option #1 can pose a security risk by showing list information to users who do not have access to the list.

Option #2 : Custom list in each site

Note: Decent option with limited development if 2010.  More difficult with maintain content types with 2007/WSS3.

Option #3 : Using the Publishing features with a Page Layout

Note: Best option for look & feel, limited development required.  More difficult with maintain content types with 2007/WSS3.

Option #4 : Custom dev, Web Part with site information on home page

Note: Interesting option for both 2007/2010 for limited data types.  It's a simple Web Part that can be added to web pages that will show site metadata.  Model can be defined with Features or custom code.

So what would I choose?  Option 3’s the most visually interesting if you have SharePoint Server; Option 2’s just about the same but with list items and somewhat more awkward for editing.  Option 4’s good for both 2007/2010 environments and allows for updates through simple feature upgrades, but it’s custom and doesn’t support a lot of data types.  My personal choice would be #3 if you can, and I’d definitely stay away from option #1 almost all the time.

I’ll detail Option #3 in a follow-up post.

Maxime

First thing I looked was the permissions on the Managed Metadata Service (MMS) and it was fine.  I tried creating another metadata column and, when I was clickong on “Customize your term set”, I was getting a pop-up error saying the service might not be available.  I looked at the ULS log and found that I was getting an ArgumentException on the CreateGroup method, but I couldn’t find the group name.  Using Fiddler, I found that it was using the “Untitled” group name.

As it turns out, if you do not enter a column name and click on that radio button, it automatically creates a group named “Untitled”.  If you cancel the column creation or browse away, the group is still there!  In fact, if you enter a column name before clicking on the radio button, it still creates the group but there’s a better error handling – if it detects that the group already exists, it will append “_1” at the end of the group title.  This doesn’t happen when it uses the default “Untitled” group name.

Unfortunately, at this point, the only fix is to always use a column name before clicking on the radio button.  The only supported way to remove that “Untitled” (and empty) group name, that I found, was to delete the MMS, or possibly through the API.  The “Term Store Management” tool, available from the Site Collection settings will also allow you to delete the group – if the site collection is still there, or if you know which one created it.

A small fix from the PG would be to use the same append with “_1” for Untitled as well.

Maxime

While adding tenants, if you are logged as a Farm Administrator (that has all the necessary permissions), I would get an error on the New-SPSite command for the member site!  Looking at the ULS logs, I could see an Access Denied exception in the TaxonomyFieldAdded feature receiver.  I looked at the MMSA & UPA permissions in all shapes and forms and couldn't figure out what was the problem.  The only way I could add a tenant was using the Farm account ==> which is really bad.  As it turns out, while working on the search crawl issue (previous post), I switched my classic authentication web application to Claims-Based authentication and it also fixed this problem.

So the rational of this, and it was a recommendation to begin with (but hey, I had the option so I tried classic authentication anyway :)), use Claims-Based Authentication for Multi-Tenancy in SharePoint.  While you can make it work with classic authentication, it will require a dedicated search crawl account + crawl rule per tenant, and it will require you to add tenants with the Farm account, which isn't recommended.

Note: if you do a single farm, you can have classic mode authentication working fine for at least the web application user policies.

Happy tenancy!

Following my previous post on search crawl with multi-tenancy, https://blogs.msdn.com/b/maximeb/archive/2010/10/24/configuring-sharepoint-2010-search-for-crawling-host-name-site-collection-tenants.aspx, I found a little bug when synchronizing user profiles for multi-tenants.  As a recap, the setup is the following:

• Services farm with Search, Managed Metadata Services, and User Profile
  • Search Content Source : https://hosting.contoso.local
• Hosting farm with tenants for :
  • Contoso
  • Adventure Works
  • Woodgrove
• Active directory :
  • Customers OU
  • an OU for each tenant under the Customers OU
• Single Web Application hosting all tenants with host-name site collection.
  • Note: With 2010, it’s now possible to have more flexible managed paths.  it allows you to have an explicit path for /admin, /cthub, and /mysites/personal for each tenants with simply 4 managed paths (total!)
• URLs (okay, I know, they could use some work for real world!!):
  • https://hosting.contoso.local (the hosting web application, nobody’s really navigating here)
  • https://contoso.contoso.local
  • https://adventureworks.contoso.local
  • https://woodgrove.contoso.local
• Tested with SharePoint 2010 RTM
• The fact that I have 2 farms probably doesn’t have anything to do with the issue at hand

To learn about multi-tenancy and how to set it up, which I hope you have done if you are reading this blog post, read Spence Harbar’s excellent post series here: https://www.harbar.net/archive/2010/09/14/rational-guide-to-multi-tenancy-with-sharepoint-2010-part-six.aspx.

The mistake I did was quite simple to reproduce : before creating my tenants (or at least, before linking them to the UPS – see below), I ran a User Profile Services synchronization with Active Directory.  The UPS-Sync contains a connection to the contoso.local domain, which in turn synchronizes only the Customers OU and all its child – effectively syncing all customer accounts in a single connection.  In this sample, I had 3 accounts for each tenants, and 3 tenants, so a total of 9 accounts were imported.

Connecting a tenant to the UPS is done with the following command (in this example, for AdventureWorks):

$upaProxy = Get-SPServiceApplicationProxy | where-object {$_.DisplayName –eq <<ProxyName>>} Add-SPSiteSubscriptionProfileConfig -id $sub –SynchronizationOU “AdventureWorks” –MySiteHostLocation "https://adventureworks.contoso.local/mysites" -MySiteManagedPath "/mysites/personal" -SiteNamingConflictResolution "None" -ProfileServiceApplicationProxy $upaProxy

If you did a user import profiles before adding the first tenant, the total user count in the user profiles will now be 18!!!  You add another tenant and it becomes 27, 36, 45, etc.  The worst part is that each tenant can administer all user profiles!  Obviously, something’s messed up .

Now the good news is that you simply have to add a single tenant before doing the first user profile import synchronization and you are set to go.  In my example, if I simply run a sync after the first tenant, it will only import 3 accounts – effectively taking into account the SynchronizationOU sent with the Add-SPSiteSubscriptionProfileConfig command – even if the connection points to an OU that contains 3 child OUs totalling 9 accounts.

It works very good as long as you create at least one tenant before user profile sync.

Happy tenancy!

I’ve been playing with SharePoint 2010 multi-tenancy in the past 2 weeks and I had some issue setting up search so that it can crawl tenants properly.  The setup is the following:

• Services farm with Search, Managed Metadata Services, and User Profile
  • Search Content Source : https://hosting.contoso.local
• Hosting farm with tenants for :
  • Contoso
  • Adventure Works
  • Woodgrove
• Active directory :
  • Customers OU
  • an OU for each tenant under the Customers OU
• Single Web Application hosting all tenants with host-name site collection.  Update: The web application was configured as Windows (NTLM) authentication.
  • Note: With 2010, it’s now possible to have more flexible managed paths.  it allows you to have an explicit path for /admin, /cthub, and /mysites/personal for each tenants with simply 4 managed paths (total!)
• URLs (okay, I know, they could use some work for real world!!):
  • https://hosting.contoso.local (the hosting web application, nobody’s really navigating here)
  • https://contoso.contoso.local
  • https://adventureworks.contoso.local
  • https://woodgrove.contoso.local
• Tested with SharePoint 2010 RTM

To learn about multi-tenancy and how to set it up, which I hope you have done if you are reading this blog post, read Spence Harbar’s excellent post series here: https://www.harbar.net/archive/2010/09/14/rational-guide-to-multi-tenancy-with-sharepoint-2010-part-six.aspx.

Getting back to this post’s subject, you typically have a single search crawl account that has (automatic) access to all web applications through a user policy providing Full Read access.  This ensures a simple and efficient to all site collections of all web applications.  Now since I have 2 different farms, I had to manually give access to my search crawl account (contoso\sp_search) to my web application on the hosting farm.

The problem can first be seen when you try to log in to one of the tenant, say https://adventureworks.contoso.local.  If you try to log in as the search crawl account, you’ll get an access denied.  The reason for this is due when we create the site and connect it to UPS:

$upaProxy = Get-SPServiceApplicationProxy | where-object {$_.DisplayName –eq <<ProxyName>>}
Add-SPSiteSubscriptionProfileConfig -id $sub –SynchronizationOU “AdventureWorks” –MySiteHostLocation 
    "https://adventureworks.contoso.local/mysites" -MySiteManagedPath "/mysites/personal" 
    -SiteNamingConflictResolution "None" -ProfileServiceApplicationProxy $upaProxy

Edit: the command that restricts the account was the previous one when you provision tenant (in Spence's blog): Set-SPSiteSubscriptionConfig –id $sub -FeaturePack $customerFeatures -UserAccountDirectoryPath "OU=$customerName,OU=Customers,DC=contoso,DC=local".  Thanks Spence ;)

The SynchronizationOU property, which really takes an OU name and not its distinguishedName (as it should) UserAccountDirectoryPath, specifies where the users of this site reside in Active Directory – effectively enforcing who has access to the site with the exception of the farm administrators.  This allow a partition when assigning permissions so that you cannot give a Woodgrove account access to an AdventureWorks site for example – they won’t even see them.

Since giving Full Read to a single account doesn’t work, the only fix that I found is the following (UPDATE: SEE NEXT SECTION)

• Active Directory:
  • Create an Active Directory group for “Tenant Search Crawls”
  • Create a search crawl account for each tenant in their respective OU
  • Add each of these search crawl account in the Active Directory group
• Services farm:
  • Create a user web application policy providing Full Read to the new Active Directory group
  • In search administration, create a crawl rule for each tenant similar to:
    • https://adventureworks.contoso.local, includes in crawl, specify account to <<its respective search crawl account>>
    • Note: you can use the PowerShell command New-SPEnterpriseSearchCrawlRule to create this rule through script when adding a tenant – however, you’ll either have to run a remote command in PowerShell, or execute this portion on the services farm.  If your tenants are in the same farm as the services, then you can automate this easily.

Update and new/better solution: I was told that this was working for Office 365 so I started digging deeper on the differences we might have between the two.  The 2 that came to mind were the fact that I have a remote farm (not really typical for a tenant scenario), and the 2nd was that I'm was using NTLM for the tenants web application.  As it turns out, switching the web application to claims (read: https://blogs.technet.com/b/speschka/archive/2010/07/20/migrating-from-windows-classic-auth-to-windows-claims-auth-in-sharepoint-2010-part-2.aspx) will fix the issue and the Web Application policies will now work properly ==> i.e. the search crawl account, which has Full Read from a Web Application User Policy.

If you absolutely need to do Windows Authentication (don't know why, and see my upcoming blog post, you'll get another issue there), then you'll need the first solution.

Crawling will now work properly.

We ran into this problem while setting up some test farms where we’d get an exception while running the powershell for New-SPConfigurationDatabase.  The exception simply mentions “This farm has pending upgrades error” and it would fail.  The funny thing is that it’s a new farm with no installation.  We had tested the script before and it was running fine.  So I tested running the wizard and it worked fine!

As it turns out, this new farm had one difference: the SharePoint 2010 bits were installed, but so where the Office Web Apps bits.  So we took another test environment that had the same base (SP2010 + Office Web Apps, no wizard/farm setup), and then I ran psconfig –cmd upgrade –inplace b2b –wait –force.  The command returned an exception but it did configure one setting – I reran the New-SPConfigurationDatabase after and it ran fine.

So if you script (PowerShell) an installation that has SP2010 + Office Web Apps, do run the psconfig command, except an exception, and move on with your script – it’ll run fine.

Maxime

In the past couple years, I’ve often gotten the question “how can I effectively find sites (collections) based on enterprise metadata?”.  For example, we may want to find all projects based on the organizational hierarchy, or the enterprise may have major company pillars that projects are based on, or to find all ‘XYZ-related’ projects, etc.

With SharePoint 2007, you could use the Site Directory feature which was providing a limited single-dimension listing site collections.  However, it doesn’t scale very well to 1000s of site collections and it’s not securely trimmed.  This feature is removed from SharePoint 2010 (except for upgrade scenarios).

My goal was to provide something for both 2007 and 2010 that brings more features and value.  Essentially, the ability to define metadata at the feature level (metadata for the whole organization), the site definition level (metadata for the template), and at the provisioning level (metadata based on provisioning questions).  Furthermore, site owners can edit property values through a standard web part on the site’s home page – with the ability to have read-only properties that are (re-)defined programmatically or through the feature.  Last, but not least, since the metadata is added directly on the site’s page, the results are securely trimmed to what your users can actually see.

The solution is effectively simple: provide a metadata definition that a web part can read, add META tags to the home page, and allow for user editing.  Search will be able to index those META tags from which administrators can create Managed Properties.  From there, the standard search results can then be refined (2010), or faceted (2007 with the Faceted search component from CodePlex, https://facetedsearch.codeplex.com/).  You can decide to provide custom interfaces with standard SQL Search development.

The solution logically resembles this:

Physically, the solution requires a somewhat complex metadata definition that is essentially a class that is serialized in XML and stored in the SPSite.RootWeb.AllProperties.  This XML can be brought by the Site Definition or Feature properties, from which the Feature Receiver will read the properties and add them to the property bag; or it can be added through simple custom code.  This definition is complex due to providing features such as multilingual display names, allowing for read-only properties, etc.

Feature Properties

Here’s an excerpt of the sample Feature.xml:

<Feature  Id="3aba9d3d-228a-443d-85b3-1f35d8aa6858"

          Title="[Maxime Bombardier] Site Collection Metadata Sample"

          Description="This is a feature that samples how to use the MaximeBSiteCollectionMetadata feature receiver and web part."

          Version="12.0.0.0"

          Hidden="FALSE"

          Scope="Site"

          DefaultResourceFile="core"

          xmlns="https://schemas.microsoft.com/sharepoint/"

          ReceiverAssembly="MaximeB.SharePoint.SiteCollectionMetadata, Version=1.0.0.0, Culture=neutral, PublicKeyToken=501530d13e13b1c5"

          ReceiverClass="MaximeB.SharePoint.SiteCollectionMetadata.MaximeBSiteCollectionMetadataFeatureReceiver"

          >

  <ActivationDependencies>

    <ActivationDependency FeatureId="6ede1ea7-4396-4cbc-b3f2-a757ca203463" />

  </ActivationDependencies>

  <Properties>

    <Property Key="MetaDef-ConfidentialInformation" Value="&lt;?xml version=&quot;1.0&quot; encoding=&quot;utf-16&quot;?&gt;

<SiteCollectionMetadataDefinition xmlns:xsi="https://www.w3.org/2001/XMLSchema-instance" xmlns:xsd="https://www.w3.org/2001/XMLSchema">

  <Name>ConfidentialInformation</Name>

  <AllowEditing>true</AllowEditing>

  <DataType>TrueFalse</DataType>

  <Value>true</Value>

  <Choices />

  <DisplayNames>

    <SiteCollectionMetadataDisplayName>

      <Code>1036</Code>

      <Text>Information confidentielle</Text>

    </SiteCollectionMetadataDisplayName>

    <SiteCollectionMetadataDisplayName>

      <Code>1033</Code>

      <Text>Confidential Information</Text>

    </SiteCollectionMetadataDisplayName>

  </DisplayNames>

</SiteCollectionMetadataDefinition>" />

    <Property Key="MetaDef-Region" Value="&lt;?xml version=&quot;1.0&quot; encoding=&quot;utf-16&quot;?&gt;

<SiteCollectionMetadataDefinition xmlns:xsi="https://www.w3.org/2001/XMLSchema-instance" xmlns:xsd="https://www.w3.org/2001/XMLSchema">

  <Name>Region</Name>

  <AllowEditing>true</AllowEditing>

  <DataType>Choices</DataType>

  <Value>North America</Value>

  <Choices>

    <string>North America</string>

    <string>Europe</string>

    <string>Canada</string>

  </Choices>

  <DisplayNames>

    <SiteCollectionMetadataDisplayName>

      <Code>1036</Code>

      <Text>Region</Text>

    </SiteCollectionMetadataDisplayName>

    <SiteCollectionMetadataDisplayName>

      <Code>1033</Code>

      <Text>Region</Text>

    </SiteCollectionMetadataDisplayName>

  </DisplayNames>

</SiteCollectionMetadataDefinition>" />

    <Property Key="MetaDef-ProjectName" Value="&lt;?xml version=&quot;1.0&quot; encoding=&quot;utf-16&quot;?&gt;

<SiteCollectionMetadataDefinition xmlns:xsi="https://www.w3.org/2001/XMLSchema-instance" xmlns:xsd="https://www.w3.org/2001/XMLSchema">

  <Name>ProjectName</Name>

  <AllowEditing>true</AllowEditing>

  <DataType>Text</DataType>

  <Value>Site Collection Metadata Project</Value>

  <Choices />

  <DisplayNames>

    <SiteCollectionMetadataDisplayName>

      <Code>1036</Code>

      <Text>Nom de projet</Text>

    </SiteCollectionMetadataDisplayName>

    <SiteCollectionMetadataDisplayName>

      <Code>1033</Code>

      <Text>Project Name</Text>

    </SiteCollectionMetadataDisplayName>

  </DisplayNames>

</SiteCollectionMetadataDefinition>" />

    <Property Key="MetaDef-Model" Value="&lt;?xml version=&quot;1.0&quot; encoding=&quot;utf-16&quot;?&gt;

<SiteCollectionMetadataDefinition xmlns:xsi="https://www.w3.org/2001/XMLSchema-instance" xmlns:xsd="https://www.w3.org/2001/XMLSchema">

  <AllowEditing>false</AllowEditing>

  <DataType>Text</DataType>

  <Value>Collaboration Team Site by Maxime</Value>

  <Name>Model</Name>

  <Choices />

  <Active>true</Active>

  <DisplayNames>

    <SiteCollectionMetadataDisplayName>

      <Code>1033</Code>

      <Text>Model</Text>

    </SiteCollectionMetadataDisplayName>

    <SiteCollectionMetadataDisplayName>

      <Code>1036</Code>

      <Text>Modele</Text>

    </SiteCollectionMetadataDisplayName>

  </DisplayNames>

</SiteCollectionMetadataDefinition>" />

  </Properties>

</Feature>

The complexity of the definition comes from the limitation of storing in the AllProperties property bag : it’s a simple name/value collection so I need to store text.  If you look carefully, the definition is actually encoded Xml with the following properties: Active, AllowEditing, DataType, Value, Name, Choices, and DisplayNames.

Note: you need to paste it as-is, I had some issues when adding spaces/tabs.

Site Definition Properties

At the site definition, it’s basically the same concept:

      <SiteFeatures>

... removed ...

        <!-- Site Collection Metadata -->

        <Feature ID="6ede1ea7-4396-4cbc-b3f2-a757ca203463" />

        <!-- Site Collection Metadata Sample -->

        <Feature ID="3aba9d3d-228a-443d-85b3-1f35d8aa6858">

          <Properties xmlns="https://schemas.microsoft.com/sharepoint/">

            <Property Key="MetaDef-SiteOwnerName" Value="&lt;?xml version=&quot;1.0&quot; encoding=&quot;utf-16&quot;?&gt;

<SiteCollectionMetadataDefinition xmlns:xsi="https://www.w3.org/2001/XMLSchema-instance" xmlns:xsd="https://www.w3.org/2001/XMLSchema">

  <Name>SiteOwnerName</Name>

  <AllowEditing>true</AllowEditing>

  <DataType>Text</DataType>

  <Value>Maxime Bombardier</Value>

  <Choices />

  <DisplayNames>

    <SiteCollectionMetadataDisplayName>

      <Code>1033</Code>

      <Text>Site Owner Name</Text>

    </SiteCollectionMetadataDisplayName>

  </DisplayNames>

</SiteCollectionMetadataDefinition>" />

... removed ...

Custom Code Properties

From a custom code perspective, I provided managers to create, serialize, and deserialize these properties, and I also provide a simple Windows application from which you can define your properties, copy them to clipboard, and paste them in your Site Definition or Feature properties.  These managers can be found with the namespace MaximeB.SharePoint.SiteCollectionMetadata.SiteCollectionMetadataDefinitionManager.  If you create a MaximeB.SharePoint.SiteCollectionMetadata.SiteCollectionMetadataDefinition and pass it to the GetStringFromMetadata method, you can store the returning value in the SPSite.RootWeb.AllProperties with a key that starts with “MetaDef-“ and it will automatically work with the Web Part.

The solution supports for these data types:

• Text
• Choice (single selection, dropdown list)
• Yes/No (checkbox)
• Date (displayed as Text, planning to upgrade this in the future)

Solution Download

The solution code can be found here:  while the Web Part’s WSP can also be found here : .  Note that the solutions are with Visual Studio 2008 as I needed the solution to work with SharePoint 2007.  I also used WSPBuilder 1.0.6 : https://wspbuilder.codeplex.com/.

The solution contains the following projects:

• RedirectHomePageForSearch: in some instances with SharePoint 2010, the default page is not returned in the search results – it’s replaced by the site’s URL (i.e.: https://contoso versus https://contoso/default.aspx).  The trouble is that when you edit the page, only the https://contoso/default.aspx will be re-indexed by an incremental crawl.  This project simply adds a feature that you can activate which will create a redirect page as your welcome page, which will redirect to your ‘previous’ welcome page.  You may not need that depending on your results.
• SiteCollectionMetadata : This is the main project that defines the metadata functionality, provides the Web Part, and packages this in a WSP.  This is the only required solution.
• SiteCollectionMetadataSample : This is a sample project with properties defined at the site definition and feature definition levels.
• SiteCollectionMetadataSerializationTester : This is a Windows Application that allows you to create your property definitions and paste them in a Site Definition or Feature definition.

Here’s what the Windows Application looks like:

Sample usage

Here’s a quick list of properties that can be practical:

• Project Name
• Description / Keywords
• Classification level (low/medium/high business impact)
• Technologies (list)
• Region/Country
• Company pillars / main themes
• Year / Date
• Start/End dates
• Customer name
• Site model name (i.e.: if you have multiple ‘project’ templates but want to aggregate them to target content)
• Model version
• Team, Manager, “Hierarchy” names
• In theory, you could also create a ‘My Workspaces’ web part based on this metadata and a custom search query. 

(if you have more, add to comments!)

The flexibility of the model is practical.  Some enterprise may use a common template for projects, but they want to have a different look per business divisions, and they’d like to target content to each of them.  You could either have a custom provisioning process, or a custom site definition that defines which business division it’s in, and your ‘content/communication’ web part would be able to use the values and target content accordingly.

Note that you’ll have to create Managed Properties, and/or make the properties searchable, to use them with Search.  Also, when you add a new Managed Property, it requires a Full Crawl until it takes effect on all pages.  However, any updates on the metadata will be picked through an incremental crawl.

What happens when I activate/deactivate the feature?

When the feature’s activated, it will go through the feature and site definitions to sync properties.  If the property is already defined, and it allows editing, the definition will be updated but the value will remain the ‘old’ one.  If the property doesn’t allow editing, the definition and value will be updated.

When the feature is deactivated, the definitions will remain, however, the Active property will be set to False, effectively removing all META tags upon a subsequent crawl.  Reactivating the feature will reset the definition to Active by default.

Recap – what do I really need?

Last, a recap on what you need to do :

• Deploy the WSP which will add a DLL in the GAC and a feature that adds the Web Part
• (Optional) Define metadata in the feature’s property definition, or in the Site Collection’s feature activation property definition.
• (Optional) Define metadata through code, at or after the site collection provisioning process.  Note that they can be combined.
• Note: while the last 2 steps are optional, you have to use one of them if you want metadata!
• Add the SiteCollectionMetadata Web Part on one of your site collection page (typically the home page).  You should remove its title display so that visiting users do not see it.
• Crawl the pages that defines new properties (at the least)
• Define Managed Properties for the newly found properties
• Recrawl the pages (or do a Full crawl, easier if you can)
• (Optional) Consume the properties through a customized search tab, either with refinements or facets, or through keyword search.

What’s next

• Get feedback and act on it
• Update code with error handling and comments
• Support for date data type
• Support for multi-select dropdown
• Support for cascading dropdowns (which would require AJAX)

DISCLAIMER: This post is provided AS IS with no warranties and confers no rights.  This is not an official Microsoft response but rather a view at exploring options with IE6 and SharePoint 2010.  Also, my personal opinion is that using IE6 with SharePoint 2010 should be a last resort after evaluating risks and concerns compared to a migration.  But I’m also a consultant and have to look at all solutions to better guide customers effectively with their business in mind, not just wishful thinking that all companies can simply upgrade easily.  The intended workload in this post is for team collaboration as this is the primary requirement where companies don't want to wait until their upgrade to IE7+ is done.

Official supportability statement

First off, officially, Internet Explorer 6 (IE6) is not supported.  The reason for that is simple, not everything was tested for IE6, and also, the new ribbon interface does not work with IE6.  I’ve posted screen shots for the standard SharePoint 2010 templates with IE6 here : https://blogs.msdn.com/b/maximeb/archive/2010/04/01/what-sharepoint-2010-looks-like-with-internet-explorer-6.aspx.

The official browser supportability statements can be found here:

• SharePoint Foundation: https://technet.microsoft.com/en-us/library/cc288142.aspx
• SharePoint Server: https://technet.microsoft.com/en-us/library/cc263526.aspx
• Project Server: https://technet.microsoft.com/en-us/library/ff631137.aspx
  • Note: for Project Server, forget any browsers but IE7 and IE8 until SP1 where they will try to match the standard browser supportability of SharePoint Server 2010.  IE6 will not work for Project Server.

Note regarding publishing sites: for visitors, you are typically managing the GUI so you have to make it work with IE6.  For authors, I would strongly suggest to upgrade their browsers and use the 2010 interfaces as it’s much more effective and you are unlikely to have too many authors that you wouldn’t be able to manage their workstations.

Upgrade is the best solution

Now that we got that out of the way, in consulting with large companies, government agencies, and with financial services organizations, it’s typical to have Windows XP + IE6 in those organization.  While the official story is that IE6 is about 5% of the browser market, they are 100% of my customers with 100k of SharePoint seats.

While IE6 is supported until 2014, these organizations should obviously look at upgrading IE6 for several security, performance, and functionality reasons.  However, my experience is that these organizations also depends on IE6 through add-ons and ActiveX that do not work on IE8.  For example, you may have a 3rd party application that uses an add-on, but upgrading that add-on to work on IE8 requires a substantial upgrade of the back-end systems it talks to as well!  Migrating the browser then becomes a full blown upgrade.

This contributes to the shift away from Internet Explorer where these organizations are starting to deploy FireFox or another browser – which starts a very big and different challenge with managing this tool, deploying patches, having the user start the right browser for the right thing, etc.

MED-V is the next best thing

If upgrade is not the solution, the next best thing is to look at a solution such as MED-V where you stream one version of the browser.  It requires some investment in deploying MED-V, and to deploy the right icons to start the right browser version depending on where the user is going.  This could done from a portal or through a light deployment on the user desktops.  This allow to quickly have IE8 centrally while keeping IE6 on the desktop while the massive upgrade to IE8 project can go on.  When it does upgrade, you can then provide IE6 centrally for the users and few applications that needs it.  This solution should be evaluated and seriously considered before the next and last one.

I can’t upgrade, can anything work with IE6?

If you can’t upgrade, and you can’t use MED-V, can you still use IE6 with SharePoint 2010?  We know it’s not supported (I hope I was clear by now), and this is why I put a disclaimer that it’s not an official Microsoft response, however, the question is “Is there anything that works with IE6 to give me enough time to go through my upgrade?”  After all, if you take a 2007 site and upgrade it to 2010, it still looks like it was before which still works with IE6. 

Before going into details of what works and what doesn’t, let me quickly discuss the supportability statement and how it relates to making a support call.  What it primarily means is that if you make a support call, you will have to reproduce the problem with a supported browser before.  It also means that there might (will) be some areas where the UI will not look right to your users.  As with any solution, you will have to look at the solution, validate it against your use case, and identify any gap + resolutions.  It’s about risks management and mitigations.  If you don’t want any risks with using the platform today, stop reading and go with SharePoint 2007.  However, if you plan to migrate to 2010 in a near future, that also brings risks, concerns, and costs.

SharePoint 2010 functionality matrix with IE6

Without further ado or warnings, here’s the breakdown of what I tested with IE6 and SharePoint 2010:

Legend:

• Works with IE6 ()
• Works with IE6 but slight visual impact ()
• Does not work with IE6 ()
• Did not test / out of scope ()

Intended scope of this evaluation

My intended scope of this evaluation is for customers that are looking at implementing their first enterprise SharePoint solution but are still using IE6, or that they have a Team Site type installation with 2007 and want to migrate to gain some of the 2010 benefits.  However, outside of my scope are all enterprise features such as Office Web Apps, FAST, and external data.  Social networking with 2010 is unfortunately out of this scope as well due to its requirement to IE7+.

Underlying this is that companies will migrate in the next 1-2 years to IE7.

How to make it work

The basis of my evaluation was using the 2007 templates, update them to 2010, and simply use them.  The main components to update are :

• In the ONET.XML definition, with the <Project> element, add a property called UIVersion with a value of “3”.
• In the ONET.XML definition, with the <Configuration> element, add a property called MasterUrl with a value of “_catalogs/masterpage/default.master”
• Either use PowerShell to set the SPSite.UIVersionConfigurationEnabled to true, or use the little feature that does the same thing if you activate it (with your Site Definition).
  • This allows the Visual Upgrade page to enable the buttons so that you can upgrade from the interface.  If you don’t want to allow it from there, you can omit this item.

Once you are ready to visually upgrade, you can simply run a PowerShell command.  Look at Todd Klindt’s blog at https://www.toddklindt.com/blog/Lists/Posts/Post.aspx?ID=173. 

You can download my sample templates here that are for demo purposes only but can help you kick start (NOTE: they are not production ready, especially the search one).  The link is : https://cid-869ff26f3797a09a.office-df.live.com/self.aspx/MaximeB-MSDNBlog/SharePoint%20and%20IE6/IE6CompatibleModels.zip or the icon below.

Impacts of the solution

From a technical perspective, this is very little different than doing a migration from 2007 to 2010 with the exception that I left out the My Sites.  The only negative impact could come from some area where I didn’t test IE6 and that it wouldn’t visually work properly.  There's always a possibility that some web part have new properties (Ajax enabled?) that could have some effect.  I did NOT test everything, this is a start for discussion for environments that are transitioning to IE7+ but requires SharePoint collaboration prior to full deployment.  If you go this route, which I repeat is not supported, you will need to test everything properly.  That's the major impact.

On the positive side, you gain several key functionalities for users, but also a lot of them for the administrators.  More importantly, you aren’t forcing to spend a lot of money for a migration project in the near future.

What about My Sites?

I purposely did not test the new My Sites with IE6. Well actually I did but I knew that the template would not work with IE6. Furthermore, I don’t want to go into creating a template that will work with IE6 (but yes, it’s possible). With the intended scope, I assume that companies will migrate to IE7+ in the near future so they shouldn’t develop for IE6 on templates that will not migrate great.

There is so much with the new My Sites that I’d rather recommend having a small SharePoint 2007 farm just for My Sites as a trusted my site location. You can even use SharePoint 2010 search to crawl it so your My Site farm can really be very small.

Once you have your browser upgraded, you can port over the farm and migrate it so that you gain all new capabilities.

Can I still do Social?

So the Note Board and tags are working – it’s a start. The rest comes mostly from the My Sites so I’m skipping those. The activity feeds will still be fed, but you won’t see them until you move the My Sites to 2010.

Screenshots with IE6

You can download all pictures here : https://cid-869ff26f3797a09a.office-df.live.com/self.aspx/MaximeB-MSDNBlog/SharePoint%20and%20IE6.  You can select to download all as a zip from the menu as well.

Figure 1: Team Site

Figure 2: Team Site with more stuff

Figure 3: Shared Documents

Figure 4: Shared Documents with Metadata driven navigation

Figure 5: Shared Documents Library Settings

Figure 6: Search this site

Figure 7: Search Portal

Figure 8: People Search

Figure 9: View All Site Content

Figure 10: Site Collection Settings

Figure 11: Record Declaration Settings

Figure 12: SharePoint Designer settings

Figure 13: Term Store Management tool

Figure 14: Visual Upgrade Settings

Figure 15: Solution Gallery

Figure 16: Discussions/Forums

Figure 17: Edit Web Part

Figure 18: Metadata column

Figure 19: Social Ratings

Figure 20: Workflows

Figure 21: My Profile 2010 version on IE6