Skip to content

Azure devops for CI/CD

Target machine setup

  1. Create a new user (devops) with the following sudo permissions to access hasper deployment user (say scrib):

    sudo useradd -m -s /bin/bash devops
    

    Make sure that the user can run sudo commands without password. root also has to be updated since it overriders the devops and forces a password request:

    devops ALL=(scrib) NOPASSWD: ALL
    devops ALL=(ALL) NOPASSWD: /home/devops/myagent/svc.sh
    
    # if necessary. First will enable no-passwd as the default and the
    # second will ensure that sudo-roup users dont need the password
    root   ALL=(ALL:ALL) NOPASSWD: ALL
    %sudo ALL=(ALL:ALL) NOPASSWD: ALL
    

    devops user should be able to run:

    ls /home/scrib/deploy/
    sudo -i -u scrib /home/scrib/deploy/.../run-deploy.sh
    
  2. Each target (Prod, QA etc) should be in a separate pool

  3. There should be permissions to upload secure files. Three default files that will be uploaded include:

     <prefix>.<pool>.config.json
     <prefix>.<pool>.siteconf.jsonx
     <prefix>.<pool>.django-env
    
  4. In the library, we will add the username that the application runs under (e.g., scrib)

     Name          Value
     deploy_user   scrib
    

CI-CD Code

  1. Typically all customer-specific code is in enrich- in their git repository

  2. The deploy directory within the customer repo has the CI-CD code including build:

     enrich-<customer>
        ├── deploy/
            ├── azure-ado
            │   ├── anico-prod
            │   ├── common
            │   └── scribble-build
            │   └── <customer-yamls>
            └── scripts
                ├── deploy
                └── run-deploy.sh
    

Create a new organization, project

  1. Create new organization in Azure DevOps. You can use existing organization if you have one.

    1. Go to https://dev.azure.com/ and click on New Organization from left navigation pane. New Organization

    2. Mention name of the organization, region for hosting the project and click on Continue. Org Details

    3. Next screen will show progress message. Org creation progress
  2. Create new project under the organization. You can use the existing project if you have one.

    1. Mention name of the project, visibility and click on Create. Project Details

      Click on Create project button.

    2. After project is created, you will be redirected to project home page. Project home page

Create agent pool for staging

  1. Navigate to the organization settings.
    1. Click on Organization settings from bottom left corner. Organization settings
    2. Click on Agent pools under Pipelines section. Org settings left pane
    3. Right pane will show existing agent pools including Default, Azure pipelines. Agent pools
    4. Click on Add pool button. Mention the details in create agent pool form. Add agent pool Click on Create button.
    5. Mention name of the pool and click on Create. Pool details
    6. List of available agent pools will be shown after agent pool is created. Agent pools
    7. Click on the newly created agent pool to add agent to the pool. Click on the agent tab. Click on New agent button. Agent tab
    8. Steps for agent configuration will be displayed. Agent configuration steps

Configuring agent for the staging agent pool

  1. Obtain a PAT (personal access token) with a long enough duration to turn a VM into an agent. Store it somewhere. You will need it later on. It is unclear if the PAT is for once time registration or if the PAT has to be valid through the lifetime of the agent. Obtain PAT for agent activation

  2. Note: You would need sudo permissions with privilage to run systemctl commands for configuring agents on the VM.

  3. SSH to staging machine.
  4. Refer to the steps given in the agent configuration steps.
  5. Download the agent package.
  6. Change to the directory where you want to install the agent.
  7. Extract the agent package from the location where it is downloaded. Default location given in the instructions might have to be changed from ~/Downloads to the actual location.
  8. Run the config.sh script.
  9. Enter the details as asked in the script.
  10. Enter the server URL.:https://dev.azure.com/organization_name
  11. Enter the PAT token.
  12. Enter the agent pool name.:staging
  13. Enter the agent name.:staging
  14. Start the agent.
  15. Install agent service using
     sudo ./svc.sh install
    
  16. Start agent as a service.
         sudo ./svc.sh start
    
  17. Check the status of the agent.
         sudo ./svc.sh status
    
  18. Verify the agent is running in the agent pool.

    1. Go to the agent pool in Azure DevOps.
    2. Click on the agent pool.
    3. Click on the agents tab.
    4. Newly added agent will be shown in the list of agents. Agent list
  19. We are running the self-hosted agent on the staging machine. This agent will be used to deploy the application to the staging environment. Required scripts to deploy, upgrade the application are already available on the staging VM.

Create agent pool for production

  1. Process is similar to creating agent pool for staging. As mentioned in Create agent pool for staging section
  2. Use following values while creating agent pool for production.
    1. Name: production
    2. Description: Production agent pool
    3. Click on Create button.
  3. SSH to production machine for configuring agent.
  4. Use following values while configuring agent for the production agent pool.
    1. agent name: production
    2. pool: production

Creating deployment pipeline

  1. Make sure you have admin privilage for the git repo to connect the git repo to the Azure DevOps.
  2. Go to the project home page.
  3. Click on Pipelines from left navigation pane. Pipelines
  4. It will show list of existing pipelines. Click on Create pipeline button. Create pipeline
  5. Select the source of the code. Click on GitHub. Select source
  6. It will display list of available github repos. Search for the desired repo which contains the source code for the yaml spec for the pipeline. Repo search If desired repo is not listed, click on Connection link. From you may use specific connection message at the end of the list.
  7. Use existing connection if desired account/organization is listed in the connection. Else click on Install github app button. Install github app
  8. You will be redirected to github to install ADO app. Click on Configure button. Configure ADO app
  9. Select repo permissions and click update access button. Update access
  10. You will be redirected to the Azure sign-in Azure sign-in
  11. Selected repos from github will be listed in ADO.
  12. Select the repo containing the yaml spec for the pipeline.
  13. ADO will auto-detect the yaml spec azure-pipelines.yml file in the repo. Click on Save and run button. Save and run
  14. Pipeline will be available under Pipelines for this project. Pipeline list

Running the deployment pipeline

  1. Go the project home page.
  2. Click on Pipelines from left navigation pane.
  3. Click on the deployment which you want to run.
  4. Click on the Run pipeline button.
  5. It will ask for parameter for target environment. Run pipeline
  6. Select the target environment and click on Run button.
  7. Run details are displayed in the pipeline run page. Pipeline run
  8. Logs for the pipeline run are displayed in the pipeline run page. Pipeline logs