Product synchronization between backend and frontend

The data model in Sitecore Commerce Connect is based on  artifacts and product information. All data is stored within Sitecore as Sitecore Items, which means that the solution will use the same components as the rest of Sitecore for providing the standard features for search, scalability, API and the possibility to extend the model by use of inheritance. To populate this tree, there is a solution in Commerce Connect that will provide pre defined pipelines in which to run processors to deal with the different types of information required to keep the data up to date.

commerce-connect-data-model

The tree consists of the data that are used to populate the product information and the product information itself. The folders under Product repository that are yellow are what are called “Artifacts” and are owned by the underlying backend systems but synchronized into Sitecore. The products are stored in item buckets and can be searched just as any item.

Product Synchronization

To be able to benefit from all the features in Sitecore, the product data is synchronized into the data model. This is done by using the concept of pipelines, where there are a couple of steps.

synchronize-all-products

Each step consists of a pipeline itself, but this gives an overview of how the product synchronization works.

Synchronize artifacts

The artifacts are the fundamentals in the logical data model. This includes the different manufacturers/brands, divisions, classifications, attributes, product types. They are by default fetched from the external systems before products are synchronized to ensure that all data to refer to from the products are already present.

parametrics-storm

In the screen shot above, we can see the parametrics in STORM that are going to be exposed as attributes in Sitecore.  In STORM, the product attributes are named parametrics. Those attributes can be either plain input fields or selections from lists for single or multiple values.

specifications-sitecore

When synchronized into Sitecore it is imported as specification lookups. This is then used to integrate with the product data model in Commerce Connect. A specification value can be either a plain input value or connected to a lookup list of fixed values to provide filtering in the frontend.

Get products list to synchronize

This part of this pipeline is used to decide which products to synchronize. We have connected this list to the ESB and a list of updated products that need to be populated in the frontend system. This is a list of external Ids of products that are to be synchronized. The really neat part here is that if data is fetched from several different system, this list can be concatinated as a total list of all products that need to be synchronized from several source systems such as a PIM system and an ERP system.

It is also possible to set a list of products to delete later on in the pipeline.

This list does not have to be a list of all available products, just the list of the products that need to be synchronized. In our solution, the e-commerce backend system will provide us with a list of the product ids that need to be refreshed in the frontend.

Evaluate which products to synchronize

This stage is to eliminate possible data that is not going to be read from the source system and stored back into the target system. This stage may contain calculations such as if the modification date is changed or based on a checksum. If a product is not to be synchronized, it is removed from the list created by the previous step.

Synchronize product

For each product identified in the steps below for being synchronized, this step is used to read the actual product from the source system. This pipeline is invoked once per product from the list that comes out of the steps before.

Several systems may be the source for the same product such as in the case where the ERP holds part of the information and the PIM systems provides additional information and assets.

This pipeline will also update the indexes with product information.

Delete products

The last step is to use the list provided by the previous processors in the pipeline for product synchronization to delete old products that are no longer to be available. This is normally only used for products that has not yet reach the status of sellable since it is not recommended from a SEO perspective to remove old products from the store. It is often better to preserve the SEO value and instead provide a replacement either by automation or by showing that on the product.

Flexibility in the pipelines

As all parts of Sitecore, the provided pipelines are just an example of how things are done. It is a good thing to try to stick to the intended way and just add and replace parts of the processors in the pipelines in order to keep the solution as standardized and upgradable as possible. We will look into the details in the different parts in the upcoming posts.

The following two tabs change content below.

Fredrik Gustavsson

CEO, IT-consultant at Jolix AB
Software and integration magician with an interest in how to improve commerce in a multi channel environment. Owns his own IT-consultancy business and runs an e-commerce store that has a physical store. Will write blog posts on how to take his own store into the future.