Azure devops for CI/CD

Target machine setup

1. Create a new user (devops) with the following sudo permissions to access hasper deployment user (say scrib):

   sudo useradd -m -s /bin/bash devops
   

   Make sure that the user can run sudo commands without password. root also has to be updated since it overriders the devops and forces a password request:

   devops ALL=(scrib) NOPASSWD: ALL
   devops ALL=(ALL) NOPASSWD: /home/devops/myagent/svc.sh
   
   # if necessary. First will enable no-passwd as the default and the
   # second will ensure that sudo-roup users dont need the password
   root   ALL=(ALL:ALL) NOPASSWD: ALL
   %sudo ALL=(ALL:ALL) NOPASSWD: ALL
   

   devops user should be able to run:

   ls /home/scrib/deploy/
   sudo -i -u scrib /home/scrib/deploy/.../run-deploy.sh
   

2. Each target (Prod, QA etc) should be in a separate pool

3. There should be permissions to upload secure files. Three default files that will be uploaded include:

    <prefix>.<pool>.config.json
    <prefix>.<pool>.siteconf.jsonx
    <prefix>.<pool>.django-env
   

4. In the library, we will add the username that the application runs under (e.g., scrib)

    Name          Value
    deploy_user   scrib
   

CI-CD Code

1. Typically all customer-specific code is in enrich- in their git repository

2. The deploy directory within the customer repo has the CI-CD code including build:

    enrich-<customer>
       ├── deploy/
           ├── azure-ado
           │   ├── anico-prod
           │   ├── common
           │   └── scribble-build
           │   └── <customer-yamls>
           └── scripts
               ├── deploy
               └── run-deploy.sh
   

Create a new organization, project

1. Create new organization in Azure DevOps. You can use existing organization if you have one.

   1. Go to https://dev.azure.com/ and click on New Organization from left navigation pane.

   2. Mention name of the organization, region for hosting the project and click on Continue.

   3. Next screen will show progress message.

2. Create new project under the organization. You can use the existing project if you have one.

   1. Mention name of the project, visibility and click on Create.

      Click on Create project button.

   2. After project is created, you will be redirected to project home page.

Create agent pool for staging

1. Navigate to the organization settings.
   1. Click on Organization settings from bottom left corner.
   2. Click on Agent pools under Pipelines section.
   3. Right pane will show existing agent pools including Default, Azure pipelines.
   4. Click on Add pool button. Mention the details in create agent pool form. Click on Create button.
   5. Mention name of the pool and click on Create.
   6. List of available agent pools will be shown after agent pool is created.
   7. Click on the newly created agent pool to add agent to the pool. Click on the agent tab. Click on New agent button.
   8. Steps for agent configuration will be displayed.

Configuring agent for the staging agent pool

1. Obtain a PAT (personal access token) with a long enough duration to turn a VM into an agent. Store it somewhere. You will need it later on. It is unclear if the PAT is for once time registration or if the PAT has to be valid through the lifetime of the agent.

2. Note: You would need sudo permissions with privilage to run systemctl commands for configuring agents on the VM.

3. SSH to staging machine.
4. Refer to the steps given in the agent configuration steps.
5. Download the agent package.
6. Change to the directory where you want to install the agent.
7. Extract the agent package from the location where it is downloaded. Default location given in the instructions might have to be changed from ~/Downloads to the actual location.
8. Run the config.sh script.
9. Enter the details as asked in the script.
10. Enter the server URL.:https://dev.azure.com/organization_name
11. Enter the PAT token.
12. Enter the agent pool name.:staging
13. Enter the agent name.:staging
14. Start the agent.
15. Install agent service using

     sudo ./svc.sh install
    

16. Start agent as a service.

         sudo ./svc.sh start
    

17. Check the status of the agent.

         sudo ./svc.sh status
    

18. Verify the agent is running in the agent pool.

    1. Go to the agent pool in Azure DevOps.
    2. Click on the agent pool.
    3. Click on the agents tab.
    4. Newly added agent will be shown in the list of agents.

19. We are running the self-hosted agent on the staging machine. This agent will be used to deploy the application to the staging environment. Required scripts to deploy, upgrade the application are already available on the staging VM.

Create agent pool for production

1. Process is similar to creating agent pool for staging. As mentioned in Create agent pool for staging section
2. Use following values while creating agent pool for production.
   1. Name: production
   2. Description: Production agent pool
   3. Click on Create button.
3. SSH to production machine for configuring agent.
4. Use following values while configuring agent for the production agent pool.
   1. agent name: production
   2. pool: production

Creating deployment pipeline

1. Make sure you have admin privilage for the git repo to connect the git repo to the Azure DevOps.
2. Go to the project home page.
3. Click on Pipelines from left navigation pane.
4. It will show list of existing pipelines. Click on Create pipeline button.
5. Select the source of the code. Click on GitHub.
6. It will display list of available github repos. Search for the desired repo which contains the source code for the yaml spec for the pipeline. If desired repo is not listed, click on Connection link. From you may use specific connection message at the end of the list.
7. Use existing connection if desired account/organization is listed in the connection. Else click on Install github app button.
8. You will be redirected to github to install ADO app. Click on Configure button.
9. Select repo permissions and click update access button.
10. You will be redirected to the Azure sign-in
11. Selected repos from github will be listed in ADO.
12. Select the repo containing the yaml spec for the pipeline.
13. ADO will auto-detect the yaml spec azure-pipelines.yml file in the repo. Click on Save and run button.
14. Pipeline will be available under Pipelines for this project.

Running the deployment pipeline

1. Go the project home page.
2. Click on Pipelines from left navigation pane.
3. Click on the deployment which you want to run.
4. Click on the Run pipeline button.
5. It will ask for parameter for target environment.
6. Select the target environment and click on Run button.
7. Run details are displayed in the pipeline run page.
8. Logs for the pipeline run are displayed in the pipeline run page.