Adding Coveo for Sitecore module to MCC

In previous blog post I shared the changes that I had to make to deploy a Next.js solution to Sitecore MCC and Vercel. I added JSS and SPE modules to my images and deployed them to my Azure MCC instance. In addition to JSS and SPE I had to add Coveo for Sitecore to my images which I am going to talk about in this blog post.

Coveo Documentation and Coveo for Sitecore Docker images

Coveo doesn’t have official support for running Coveo for Sitecore on containers, but they have provided sufficient information on how it can be done.

You can find high level instructions in Coveo documentation at https://docs.coveo.com/en/lb8f0136/coveo-for-sitecore-v5/install-coveo-for-sitecore-with-sitecore-in-docker

In this documentation you’ll find a reference to a fork that contains the Coveo for Sitecore assets as well as Coveo for Sitecore SXA images. I specifically was interested in Coveo for Sitecore assets for Sitecore 10.2. At the time this blog post was written there was 10.2 version of Docker images, hence I had to create my own fork with configurations for 10.2. (https://github.com/jcore/sitecore-docker-images)

The images that were created I pushed the project registry. I can’t share the registry link here, but you can use any registry that is available to you.

Solution Changes

Now I have a Coveo for Sitecore Docker for Sitecore 10.2 image that I can use in my solution. Coveo provided a fork that I used in my solution as well (https://docs.coveo.com/en/lb8f0136/coveo-for-sitecore-v5/install-coveo-for-sitecore-with-sitecore-in-docker#step-2-launch-sitecore-docker-instances-with-coveo-for-sitecore-pre-installed).  You can find at https://github.com/coveooss/sitecore-docker-examples/tree/develop/custom-images

In my solution I made the following changes:

.env

# Coveo env section

COVEO_INIT_IMAGE=[You registry url where the Docker image you created in previous step was deployed. Ex. [registry url]/custom-coveo5011101-assets:10.2.0-20220328.1]

COVEO_VERSION=[Coveo version without dots. Ex. 5011101]

COVEO_API_KEY=[API Key created in Sitecore (see Coveo API Keys section of this blog post.)]

COVEO_SEARCH_API_KEY=[API Key created in Sitecore (see Coveo API Keys section of this blog post.)]

COVEO_ORG_ID=[Coveo organization ID that you can find in Coveo Cloud Portal]

COVEO_USER=[Sitecore user that Coveo for Sitecore module will use.]

COVEO_PASSWORD=[Sitecore Coveo account password]

COVEO_FARMNAME=[Coveo farm name]

 

Adding COVEO_API_KEY in Coveo

Create an API key in Coveo Cloud portal. Copy the guid and add it to your .env file for the COVEO_API_KEY variable.

Go back to the Coveo Cloud portal and double click on the key that you just added and make sure that Privileges are set to be as follows:

If privileges are not set correctly, the activation script that runs during the application pipeline execution will fail.

 

Adding COVEO_SEARCH_API_KEY in Coveo

Add the second key to Coveo API keys for the search. Add the guid for the key to your .env file.

Go back to the Coveo Cloud portal and double click on the key that you just added and make sure that Privileges are set to be as follows:

Copying COVEO_ORG_ID from Coveo

Go to Organization section in the navigation on the left and click on Settings. In the modal window select Organization tab and click on Information menu item. Here you’ll find the Organization ID value. Copy and paste it into your .env file.

Other variables

Create a user in Sitecore for Coveo to use the same way you would do it when you install Coveo for Sitecore module in a non-container instance.

Coveo farm name can be anything or even empty for local environment.

 

Updating docker-compose.yml

  coveo-init:

    isolation: ${ISOLATION}

    image: ${REGISTRY}${COMPOSE_PROJECT_NAME}-coveo-init:${COVEO_VERSION}-${SITECORE_VERSION}

    environment:

      COVEO_API_KEY: ${COVEO_API_KEY}

      COVEO_SEARCH_API_KEY: ${COVEO_SEARCH_API_KEY}

      COVEO_ORG_ID: ${COVEO_ORG_ID}

      COVEO_USER: sitecore\${COVEO_USER}

      COVEO_PASSWORD: ${COVEO_PASSWORD}

      COVEO_HOST: http://cm

      SITECORE_ADMIN_USER: sitecore\${SITECORE_ADMIN_USER}

      SITECORE_ADMIN_PASSWORD: ${SITECORE_ADMIN_PASSWORD}

      COVEO_FARMNAME: ${COVEO_FARMNAME}

    depends_on:

      cm:

        condition: service_healthy

      mssql:

        condition: service_healthy      

 

Updating docker-compose.override.yml

  coveo-init:

    image: ${REGISTRY}${COMPOSE_PROJECT_NAME}-coveo-init:${VERSION:-latest}

    build:

      context: ./docker/build/coveo-init

      args:

        BASE_IMAGE: mcr.microsoft.com/powershell:nanoserver-1809

 

Adding coveo-init image

Take the code for this image from Coveo fork. I didn’t make any changes to these files.

CM, CD, and MSSQL-Init image changes

In docker-compose.override.yml add the following line to cm, cd and mssql-init images under args:

COVEO_ASSET: ${COVEO_INIT_IMAGE}

 

Like so:

  mssql-init:

    image: ${REGISTRY}${COMPOSE_PROJECT_NAME}-xp1-mssql-init:${VERSION:-latest}

    build:

      context: ./docker/build/mssql-init

      args:

        BASE_IMAGE: ${SITECORE_DOCKER_REGISTRY}sitecore-xp1-mssql-init:${SITECORE_VERSION}

        SPE_IMAGE: ${SITECORE_MODULE_REGISTRY}sitecore-spe-assets:${SPE_VERSION}

        HEADLESS_SERVICES_IMAGE: ${HEADLESS_SERVICES_IMAGE}

        COVEO_ASSET: ${COVEO_INIT_IMAGE} # No buildable 10.2 version tags yet.

 

Now try running “docker-compose build” command. If it succeeds, you can run docker-compose up -d to make sure the Coveo for Sitecore module is showing up in CM and is also activated.

If everything works as expected, you can try deploying the new images to MCC.

 

Deploying to MCC

Push new cm, cd, mssql-init and coveo-init images to your Azure Registry.

In application repository for MCC the following changes had to be made:

1.       Update the image references for cm, cd, and mssql-init in .

2.       Add a new image reference called coveoinit.

3.       Under roles/sitecore-xp/templates add a new file called coveo-init.yaml with the following content:

.  ---

5.  apiVersion: batch/v1

6.  kind: Job

7.  metadata:

8.    name: coveo-init

9.  spec:

10.   template:

11.     spec:

12.       nodeSelector:

13.         kubernetes.io/os: windows

14.       imagePullSecrets:

15.         - name: sitecore-docker-registry

16.       containers:

17.       - name: coveo-init

18.         image: "{{ docker_images.sitecore.coveoinit }}"

19.         env:

20.         - name: COVEO_API_KEY

21.           valueFrom:

22.             secretKeyRef:

23.               name: sitecore-coveo-api-key

24.               key: sitecore-coveo-api-key.txt

25.         - name: COVEO_SEARCH_API_KEY

26.           valueFrom:

27.             secretKeyRef:

28.               name: sitecore-coveo-search-api-key

29.               key: sitecore-coveo-search-api-key.txt

30.         - name: COVEO_ORG_ID

31.           valueFrom:

32.             secretKeyRef:

33.               name: sitecore-coveo-org-id

34.               key: sitecore-coveo-org-id.txt

35.         - name: COVEO_USER

36.           valueFrom:

37.             secretKeyRef:

38.               name: sitecore-coveo-user

39.               key: sitecore-coveo-user.txt

40.         - name: COVEO_PASSWORD

41.           valueFrom:

42.             secretKeyRef:

43.               name: sitecore-coveo-password

44.               key: sitecore-coveo-password.txt

45.         - name: COVEO_HOST

46.           valueFrom:

47.             secretKeyRef:

48.               key: cm      

49.               name: sitecore-hostname

50.         - name: COVEO_FARMNAME

51.           valueFrom:

52.             secretKeyRef:

53.               key: sitecore-coveo-farmname.txt      

54.               name: sitecore-coveo-farmname              

55.         - name: SITECORE_ADMIN_USER

56.           valueFrom:

57.             secretKeyRef:

58.               name: sitecore-admin-domain-username

59.               key: sitecore-admin-domain-username.txt  

60.         - name: SITECORE_ADMIN_PASSWORD

61.           valueFrom:

62.             secretKeyRef:

63.               name: sitecore-admin

64.               key: sitecore-adminpassword.txt            

65.       restartPolicy: Never

66.   backoffLimit: 5

67.  

4. Add secret keys for all new variable in the file above.

5. In the same folder open secrets.yaml file and add the following section right before “parameters”.

  - data:

    - key: sitecore-coveo-api-key.txt

      objectName: sitecore-coveo-api-key

    secretName: sitecore-coveo-api-key

    type: Opaque    

  - data:

    - key: sitecore-coveo-search-api-key.txt

      objectName: sitecore-coveo-search-api-key

    secretName: sitecore-coveo-search-api-key

    type: Opaque    

  - data:

    - key: sitecore-coveo-org-id.txt

      objectName: sitecore-coveo-org-id

    secretName: sitecore-coveo-org-id

    type: Opaque        

  - data:

    - key: sitecore-coveo-user.txt

      objectName: sitecore-coveo-user

    secretName: sitecore-coveo-user

    type: Opaque    

  - data:

    - key: sitecore-coveo-password.txt

      objectName: sitecore-coveo-password

    secretName: sitecore-coveo-password

    type: Opaque  

  - data:

    - key: sitecore-coveo-farmname.txt

      objectName: sitecore-coveo-farmname

    secretName: sitecore-coveo-farmname

    type: Opaque      

  - data:

    - key: sitecore-admin-domain-username.txt

      objectName: sitecore-admin-domain-username

    secretName: sitecore-admin-domain-username

    type: Opaque  

At the end of objects array in parameters section of the same file add the following section at the very end of the string:

        - |

          objectName: sitecore-coveo-api-key  

          objectType: secret

        - |

          objectName: sitecore-coveo-search-api-key  

          objectType: secret            

        - |

          objectName: sitecore-coveo-org-id  

          objectType: secret        

        - |

          objectName: sitecore-coveo-user  

          objectType: secret        

        - |

          objectName: sitecore-coveo-password  

          objectType: secret  

        - |

          objectName: sitecore-coveo-farmname  

          objectType: secret        

        - |

          objectName: sitecore-admin-domain-username  

          objectType: secret    

6. At the end of main.yaml file in roles/sitecore-xp/tasks folder add the following:

 

- name: Execute post-init containers

  include_tasks: post-init.yaml

  when: hadr_resources.isExecuteInitContainer|bool==true      

7. In the same folder add a new post-init.yaml file with the following content:

---

- name: 'Wait - cm running'

  k8s_info:

    kind: Pod

    label_selectors:

      - app = cm

    namespace: "{{ solution_id }}"

    wait: true

    wait_sleep: 30

    wait_timeout: 180

    wait_condition:

      type: Ready

      status: "True"

 

- name: Execute coveo-init job

  k8s:

    apply: true

    namespace: "{{ solution_id }}"

    state: present

    definition: "{{ lookup('template', 'coveo-init.yaml') }}"

 

- name: 'Wait - coveo-init job'

  k8s_info:

    kind: Job

    name: coveo-init

    namespace: "{{ solution_id }}"

  register: coveo_init_result

  until: (coveo_init_result.resources[0].status.conditions[0].type | default('')) == 'Complete'

  retries: 30

  delay: 30

 

- name: Get all coveo-init completed pods

  k8s_info:

    kind: Pod

    namespace: "{{ solution_id }}"

    label_selectors:

      - job-name = coveo-init

  no_log: true

  register: coveo_pod_list

 

- name: Remove coveo-init job's pods

  k8s:

    kind: Pod

    name: "{{ item.metadata.name }}"

    namespace: "{{ solution_id }}"

    state: absent

  no_log: true

  with_items: "{{ coveo_pod_list.resources }}"

 

- name: Remove coveo-init job

  k8s:

    kind: Job

    name: coveo-init

    namespace: "{{ solution_id }}"

    state: absent

  no_log: true  

This is the Ansible instructions for running of coveo-init job. The job will spin off a new pod as well. To make sure that Coveo activation happens every time you do a deployment, the Ansible script deletes the pod and the job.

 

Check in the changes, create a pull request and merge the branch with your changes to the master branch. That should trigger the application pipeline execution.

If everything runs without error, you see Coveo for Sitecore module installed in your CM instance, and it is enabled, congratulations! If you are having issues, you can remove the deletion of the job and the pod temporarily to check if you see errors there. Often secret keys are not configured correctly, that would cause the job to fail.