Towards the end of 2019, Microsoft announced it’s first new service since the launch of Microsoft Teams: Project Cortex. This new initiative has the potential of becoming a game-changer in how we contextualize corporate content. Most notably, Project Cortex focuses on creating Knowledge Networks that will help organizations enable true Knowledge Management across the enterprise.
Now, if all of that sounds like a whole lot of buzzwords, you’re not alone. In this article, I’ll explain how AI (Artificial Intelligence) will soon help us turn corporate information into knowledge and what you can do to prepare.
From Data Management to Information Management in SharePoint
Before we dive into the subject of AI in SharePoint, let’s take a moment to appreciate how far SharePoint adoption has evolved.
It wasn’t that long ago when we were all over Document Management. Back in those early SharePoint days, organizations saw the potential to use the Microsoft-powered technology as a considerably improved file share replacement. Full of anticipation, they created document libraries, added folder structures and uploaded their corporate documents from other platforms (for instance, Lotus Notes) to SharePoint. For some, this approach paid off because SharePoint Search was powerful enough to index documents and search based on keywords.
Others soon realized that the real power of SharePoint lies elsewhere. Once they went beyond simple keyword search and explored the metadata, classifications and information architecture, they often found that they were able to achieve much more than just managing documents. They started to manage information.
That’s how we arrived at Information Management: the next logical step after tackling Document Management. Where Document Management concentrates on the administration of documents in an enterprise, Information Management goes far beyond that. It covers not just documents, but news, articles, announcements, as well as items in a knowledge base and data records. You can look at Information Management as a kind of umbrella item that includes Document Management.
What information management looks like in practice
Today, when an organization publishes an article or an announcement, it usually does so by publishing a new modern page in SharePoint rather than a document. As the article or announcement is based on a modern page, it uses a specific Content Type (for example, Corporate News) and metadata to tag content.
Here is an example. The metadata used to tag a news article announcing the hire of a new HR Director could look like this:
- Audience: All staff
- Type of Content: News
- Scope: Internal
- Location: North America
- Department: HR
- Publisher: HR
- Confidentiality: Public
- Valid until: February 28th, 2020
- Responsible: Mark Hammond (CEO)
Ideally, all of the content you publish in SharePoint contains tags and follows a document structure. Tagging ensures that your organization can work with information it stores. For example, the organization can create rules to prioritize this HR announcement to employees it is most relevant to, in this case, those located in North America and those working in the HR department. Tagging with metadata is the only way to ensure that the organization can create rules or business processes that use the information, like a tailored publication process regarding corporate news or applying information protection policies to corporate business reports.
Technically, rules can be based on queries and those queries rely on Content Types to determine that a document is (for example) a business report. Humans can recognize a business report by looking at the document, rules and queries can’t.
How AI turns Information Management into Knowledge Management
If you have been working with SharePoint for a long time, this is probably nothing new. What I just explained is the established best practice for Information Management. And that’s part of the problem! Organizations have been doing this for years, but the working style has changed. Many requirements of the modern Information Management systems just can’t be accomplished by just following the established best practice. That’s because Information Management is transitioning to Knowledge Management. What’s the difference between those two, you might ask? Information Management connects content to structures such as Content Types or metadata. On the other hand, Knowledge Management connects content to people.
The goal of Knowledge Management is to make knowledge sharable and actionable within the organization. From a technical standpoint, Information Management utilizes SharePoint Search, Content Types and metadata to help employees find information quickly.
Knowledge Management goes far beyond that by making information digestible and shareable, and not just searchable. It builds connections between information items to create a complex knowledge network. Interweaving content this way often is far too complex and time-consuming for humans, so Project Cortex will use AI to do it.
And here we’ve come a full circle! As the complex knowledge network takes shape, information, which is the foundation of knowledge, needs to be machine-recognizable. Even smart AI-powered bots rely on criteria to identify content, which is why Project Cortex’ AI model needs training.
SharePoint AI models require training
Adding context to SharePoint information with AI
Here is an example of how this will look like in real life. The following screenshot shows a draft of a Corporate News article about a new project called Planet Blue. Since our corporate team has already created a project site and started to work on that project, a bot equipped with Artificial Intelligence can link the News item with the corresponding Planet Blue project site, the members and the associated resources. Readers can hover over the word Planet Blue to automatically get additional insights on a topic card. But there is more. The AI bot not only creates a link between the keyword and the project site but also generates a topic page around the project including a description, a list of project members, resources etc. This topic page will update on ongoing basis.
Topic Cards add context to articles and documents published in SharePoint.
As AI-powered bots create a knowledge network (the previously mentioned interweaving of content), a topic page will also display related items and topics, helping humans understand the context, the scope of information and how the information relates to other information items. You can see an example of that below:
Knowledge Network adds context to documents.
This is just one way to make knowledge more digestible for employees. Project Cortex combines knowledge from many different sources and automatically summarizes it in a way that’s easy to digest by humans. It links related pieces of information and puts it all in context. A single piece of information becomes a part of a complex knowledge network that clearly shows the full context and scope. That’s how companies will be able to turn information into employee knowledge.
Auto-populate document metadata
Project Cortex uses information from different sources to build complex knowledge networks, and one of those sources are documents. Project Cortex will be able to extract information from uploaded documents and fill-in metadata automatically (up to a certain extent). This is what you can see in the screenshots below.
Here’s what a document with no distinct metadata might look like when uploaded:
Recently uploaded document with no metadata
A few seconds later, the smart AI automatically adds metadata:
AI autofills document metadata
While this does not mean that adding metadata (which is an unloved task of many Information Workers) will become obsolete once Project Cortex is available, it shows that even modern technology like Project Cortex still relies on existing technology akin to metadata tags and Content Types.
To achieve this, Project Cortex and the underlying AI technology requires training so that it can recognize new information (for example, which documents belong to a specific project). This is done by creating a model in the new Content Centers, which uses the already existing AI Builder functionality in PowerApps and Power Automate (formerly Microsoft Flow).
AI-powered Content Center
How to prepare for Project Cortex?
The interesting question is, what can organizations do to prepare for Project Cortex launch?
Project Cortex uses AI (Artificial Intelligence) to extract information from documents and to build a knowledge network. However, it can’t work miracles! Under the hood, Project Cortex still relies on metadata and presumably Content Types. Once Project Cortex is available, likely in Q1/2020, I assume that many organizations will import their existing documents to see how Project Cortex can help them with Knowledge Management. And that is exactly where organizations can start today. The better your current information is structured and tagged, the easier it will be to train the AI bots of Project Cortex.
But don’t get me wrong: I’m not suggesting to update your Content Types and metadata terms just to be prepared for the launch of Project Cortex. It is actually the other way around. If you keep your Content Types and metadata terms up to date and tag your documents accordingly, there will be an immediate benefit for your organization and your staff – and your start with Project Cortex will be much smoother.
At the time of writing this (January 2020), there is still too little information available to provide more detailed recommendations. The only thing we know for sure is that Power Automate, PowerApps and AI Builder will be used to do the heavy lifting under the hood. Based on the videos shared during and after Microsoft Ignite, I’m guessing that Project Cortex will take advantage of Managed Metadata as well. And because it is established best practice to use Content Types to ensure consistent usage of metadata, I assume Project Cortex will take advantage of Content Types as well. Hopefully, Microsoft will also use this opportunity to modernize the Term Store and the Content Type hub.
Best Practices regarding metadata and Content Types
Whenever I support customers who are migrating to SharePoint, I conduct at least two (2) workshops at the very beginning of the projects. The first one is around best practices for metadata and the second focuses on Content Types. My recommendation is this: work with your corporate entities and departments to identify the types of documents in use. When compiling your list, consider all document types such as manuals, invoices, reports, contracts, announcements or news. Create a corresponding Content Type for each identified type of document and take advantage of Content Type inheritance. Your corporate Content Types should be stored in the Content Type hub to ensure they can be used throughout the entire organization.
Once you’ve identified all document types, continue working on identifying corporate metadata terms and structure them within Term Sets and Term Groups. Next, assign Term Sets to Content Types and configure your document libraries to use Content Types.
There is one question that comes up each time I deliver my Content Type workshop. I am asked how many Term Sets should be used per Content Type and how many of those should be required. Unfortunately, there is no definite answer to this question. As a rule of thumb, I recommend not to use more than 10 Term Sets per Content Type.
There are also no definitive answers regarding the number of required metadata. I recommend to use only as much metadata as necessary. Ultimately, your metadata needs depend on the types of documents and on how those documents are used. For example, a corporate handbook usually needs less required metadata compared to a contract or an invoice. Also, be mindful of user experience. The more required metadata users need to provide when uploading a document, the more annoying the activity.
Summary
Project Cortex will be an exciting addition to Office 365, and I admit that I can’t wait until it is available. You might know that I am an advocate for Document Management, and I have been promoting Document Management since I became a SharePoint consultant many years ago. Project Cortex will definitely support organizations in transitioning from Information Management to Knowledge Management, but this will come with a price tag. Information still needs to be structured, and AI will require proper and thorough training. The better the training, the better the results, but if the AI model is trained poorly, you can’t expect stunning results.
If you want prepare for the launch of Project Cortex, start by assessing how metadata and Content Types are used in your organization. If you encounter gaps or areas in need of improvement, update your Managed Metadata structure and/or your Content Type structure and configure document libraries to use Content Types!
The DevFacto team is closely monitoring the progress of Project Cortex, and we are already working on guidelines and best practices to support organizations once Project Cortex is available. If you have questions or want prepare your organization for the Project Cortex, launch, get in touch with us.
References
Project Cortex – Your knowledge network in Microsoft 365
Mastering the Art of SharePoint Document Management implementation