Search, Content Tagging and Pages with search results

Every web application that is being built these days has some sort of search functionality. I have seen different implementations. Some of them are simple and easy to maintain, some not so much so. I would like to share with you, readers of my blog, how I have implemented such functionality. I am not saying only my implementation is correct, but served me well in last three applications I have worked on.

Assumptions

1. Lets assume you have content that might belong to one or more categories
2. Second assumption is that you have category landing pages that have listing modules that display content that is tagged to this category.
3. You might have listing/search results modules on other pages of your website.

Key Concepts

To be able to produce search results based on category, all articles, news and press release page items as well as any other content needs to be "tagged" to this category.

To accomplish that I have implemented a base template with  tree list "Category" field. The source of this category needs to point to a location in the Sitecore tree that represents the list of categories. All templates that can be tagged to category should inherit this template. So now, if an articles is tagged to a category or categories, it would show up in search results when you specify selected category value. Nothing remarkable so far.

The next part is landing pages. Lets say I would like to have three modules on my Category One landing page: news, articles, and press releases. Each of these modules should only display the content that is tagged to the Category One. There are two options - either implement a rendering data source that would have the category selection specified or have the same base template with Category field inherited by the landing page. When a value is selected in Category field on Category One landing page, it can be used to produce search results for all modules.

This approach might not be perfect, but seems to be pretty simple for editors to understand and for developers to implement. It allows tagging content to multiple categories/tags, reusing the same template for tagging and producing consistent results.

Serving non-Sitecore data in Sitecore. Part 2: Configuring urls for netFORUM entities.

The next step in the implementation was producing urls for netFORUM entities withing Sitecore application. Solr index is perfect for that. I've extended UrlLink computed field to call custom LinkProvider and produce url for earch of netFORUM entities, and as a result each of the entities should be served from Sitecore solution, get urllink_s value in index.

There were a few different types of entities in netFORUM, and each type had to have it's own url, a parent item in sitecore, and an item that would be Sitecore.Context.Item one when netFORUM entities is rendered. To achieve that I've added custom configuration to the sitecore config.

<Custom>

      <urls>

        <productTypes>

          <books name="books" displayName="Books" productTypeId="[GUID]" site="mywebsite" parentSiteItem="[GUID]" item="[GUID]" />

          <merchandise  name="apparel" displayName="Apparel" productTypeId="[GUID]" site="mywebsite" parentSiteItem="[GUID]" item="[GUID]" />          

        </productTypes>

        <urlExclusions urlPrefix="api|sitecore|~/media|temp|WebResource|~/icon|~/media|~media|ScriptResource|static|speak|.js|.css|.html|.htm|.jpg|.axd|.ahx|.gif|.png" />

      </urls>

</Custom>

<linkManager>
      <providers>
        <add name="sitecore">
          <patch:attribute name="type">Custom.Business.Links.LinkProvider, Custom.Business</patch:attribute>
        </add>
      </providers>
    </linkManager>

A new method was added to LinkProvider that would produce a product url for an IIndexable item:

            internal string GetProductUrl(Sitecore.ContentSearch.IIndexable indexable)
            {
                Assert.ArgumentNotNull(indexable, "indexable");

                if (this.Options.ShortenUrls)
                {
                    var obj = indexable as IndexableProductEntity;

                    var productType = UrlConfiguration.Instance(_productDatabaseName).ProductTypes.FirstOrDefault(p => p.ProductTypeId == obj.ProductTypeId);
                    if (productType != null && productType.ParentSiteItem != null && productType.Item != null)
                    {
                        var siteInfo = Sitecore.Configuration.Settings.Sites.FirstOrDefault(s => s.Name == productType.Site);
                        if (siteInfo == null)
                        {
                            return string.Empty;
                        }

                        var name = ItemUtil.ProposeValidItemName(obj.Name);
                        var itemPath = string.Empty;
                        var urlFormat = Settings.GetSetting("ProductUrlFormat", "{0}/{1}/{2}/{3}");

                        if (!string.IsNullOrWhiteSpace(productType.ProductCategory) && !string.IsNullOrWhiteSpace(productType.ProductSubCategory))
                        {
                            urlFormat = Settings.GetSetting("ProductWithSubCategoryUrlFormat", "{0}/{1}/{2}");
                            itemPath = string.Format(urlFormat, productType.ParentSiteItem.Paths.FullPath.Replace(base.GetRootPath(siteInfo, Sitecore.Context.Language, Factory.GetDatabase(_productDatabaseName)), string.Empty), obj.ProductCode, name);
                        }
                        else
                        {
                            itemPath = string.Format(urlFormat, productType.ParentSiteItem.Paths.FullPath.Replace(base.GetRootPath(siteInfo, Sitecore.Context.Language, Factory.GetDatabase(_productDatabaseName)), string.Empty), productType.Name, obj.ProductCode, name);
                        }
                        if (!string.IsNullOrWhiteSpace(itemPath))
                        {
                            var url = base.BuildItemUrl(string.Empty, itemPath);
                            return this.Options.LowercaseUrls ? url.ToLowerInvariant() : url;
                        }
                        return string.Empty;
                    }
                }

                return string.Empty;
            }

The same method is used to produce product urls within the solution.

Now as I had the urls in index, it was time to implement resolving of incoming urls into entities. So a new processor for httpRequestBegin pipeline was created and added after Sitecore.Pipelines.HttpRequest.ItemResolver, Sitecore.Kernel.

 public class CustomLinkResolver : HttpRequestProcessor
    {
        /// 

        /// A list of lowercase site names to enable this processor for
        /// 
        protected List m_siteNames = null;

        public CustomLinkResolver()
        {
            m_siteNames = new List();
        }

        public override void Process(HttpRequestArgs args)
        {
            if (Context.Item != null)
                return;

            if (!m_siteNames.Contains(Context.Site.Name.ToLower()))
                return;

            if (UrlConfigurations == null)
            {
                return;
            }
            if (UrlConfigurations.UrlExclusions != null && UrlConfigurations.UrlExclusions.Any(u => args.Url.FilePath.Contains(u)))
            {
                return;
            }

            var urlSegments = args.Url.FilePath.Split('/');

            // ensure we have at minimum 2 segments for content type and name
            if (urlSegments.Length < 2)
                return;

            // Second last segment in Url is the content type
            var contentTypeName = urlSegments[urlSegments.Length - 2];

            // Locate the content type item in the Url configuration
            var rootItem = (from ct in UrlConfigurations.ContentTypes
                            where string.Compare(ct.Name, contentTypeName, true) == 0
                            select ct.Item).FirstOrDefault();

            if (rootItem == null)
            {
                if (urlSegments.Length == 5)
                {
                    contentTypeName = urlSegments[urlSegments.Length - 3];
                }
                else if (urlSegments.Length == 3)
                {
                    contentTypeName = urlSegments[urlSegments.Length - 1];
                }
                rootItem = (from ct in UrlConfigurations.ProductTypes
                            where string.Compare(ct.Name, contentTypeName, true) == 0
                            select ct.Item).FirstOrDefault();
            }

            var databaseName = rootItem != null ? rootItem.Database.Name : DatabaseName;
            var indexName = string.Format(BusinessConstants.Search.GlobalSearchIndexNameFormat, databaseName.ToLowerInvariant());

            // Locate the item by urllink
            using (var searchContext = ContentSearchManager.GetIndex(indexName).CreateSearchContext())
            {
                var predicate = PredicateBuilder.True();
                predicate = predicate.And(i => i.Url == args.Url.FilePath);
                var query = searchContext.GetQueryable();
                var result = query.First(searchContext, predicate, false);

                if (result != null)
                {
                    if (result.DatabaseName == BusinessConstants.Search.NetForumDatabaseName)
                    {
                        Context.Item = rootItem;
                        args.Context.Items.Add("product", result);
                    }
                    else if (result.TemplateId == ID.Parse(ShortUrlItem.SitecoreItemTemplateId))
                    {
                        var contextItem = result.GetItem();
                        if (contextItem != null)
                        {
                            var customItem = contextItem.Convert();
                            if (customItem != null)
                            {
                                Context.Item = customItem.BaseItem;
                                args.Context.Items.Add("customItem", customItem);
                            }                            
                        }
                    }
                    else
                    {
                        var contextItem = result.GetItem();
                        if (contextItem != null)
                        {
                            if (!IsValidItemVersion(contextItem))
                            {
                                contextItem = contextItem.Database.GetItem(contextItem.ID);
                            }
                            Context.Item = contextItem;
                        }
                    }
                }
                else
                {
                    return;
                }
            }
        }

        private static bool IsValidItemVersion(Item item)
        {
            if (item == null)
            {
                return false;
            }
            if (item.Versions.IsLatestVersion())
            {
                return true;
            }
            return false;
        }
        /// 

        /// Adds a site by name to the list of site the processor is active for. Called by Sitecore configuration utility
        /// 
        /// The name of the site to add
        public void AddSite(string siteName)
        {
            var lowerName = siteName.ToLower();
            if (!m_siteNames.Contains(lowerName))
                m_siteNames.Add(lowerName);
        }

        public string DatabaseName
        {
            get
            {
                var db = Sitecore.Context.Database ?? Sitecore.Context.ContentDatabase;
                return db.Name;
            }
        }

        private UrlConfiguration UrlConfigurations
        {
            get
            {
                return UrlConfiguration.Instance(Context.Database.Name);
            }
        }

What this processor does is running a query to Solr sitecore_globalsearch_master_index or sitecore_globalsearch_web_index depending on the context and if any of the index documents have matching url in urllink_s field, it saves the object into Sitecore.Context.Items and sets Sitecore.Context.Item to the item property of corresponding productType.

In renderings I can access the value of saved in Sitecore.Context.Items object like this:

var product = (ProductSearchEntity)Context.Items["product"];

Next: Part 3: Implementing cross-core search