# Deploying and Managing Solutions - DX

# How Do I Deploy A New Project?

Given that your project will be deployed to Kubernetes, you should prepare a Docker image for each service. When possible, we use Terraform for provisioning infrastructure managed by third parties (e.g., prefer Cloud SQL rather than running a PostgreSQL instance inside Kubernetes).

At this moment:

  • [ ] List the services you want to deploy
    • CKAN
    • Solr
    • PostgreSQL
  • [ ] Fork experiments/dx-helm-national-grid and use it as a template
  • [ ] Delete unused files in /templates
  • [ ] Get access to the Google Cloud project datopian-dx (ask Irio or Irakli)
  • [ ] Connect kubectl with your Google Cloud credentials
  • [ ] Create Argo CD project (either via UI or CLI)
    • [ ] Install argocd cli locally
    • [ ] Run argocd proj create [project name] pointing to remote argo cd
  • [ ] Push dx-helm-national-grid fork to Gitlab/GitHub
  • [ ] Create Argo CD application inside this new project (pointing to the Git repo)

# Detailed


We are in the process of merging this with the shorter steps above.

Currently, we can separate the process of deploying new applications in two parts:

  1. Create repository for the project’s Helm Chart.
  2. Create application in Argo CD.

Argo CD will then continously syncronize the state of the cluster’s deployment with the Git repository. From that moment on, you should be able to make all the changes by only touching application code (e.g., your Flask server, your Python libraries) or deployment specification (i.e., your new Helm Chart).

Let’s start by first creating the repository.

  1. Create a new Git repository based on the company’s Helm Chart template:
    git clone https://gitlab.com/datopian/experiments/dx-helm-template
    cd dx-helm-template
    git remote rename origin upstream
  2. Now, create a new GitLab repository for this project’s deployment.
  3. Customize the template Helm Chart to your needs.
  4. Push the local repository to GitLab.
    git remote add origin [URL_OF_THE_NEW_GITLAB_REPOSITORY]
    git push -u --all origin

Now, you must tell Argo CD to start managing a deployment for this new Helm Chart.

  1. Create a read-only Deploy Token (Settings > Repository > Deploy Tokens) to give Argo CD access to the GitLab repository. Copy both username and password generated by GitLab.
  2. From your local machine, install the Argo CD CLI.
  3. Give Argo CD CLI the credentials for the accounts you used for signing in via the UI.
    argocd login localhost:8080
  4. Create a new project in Argo CD. Inside this project, you will deploy the different environments you need. Since each environment stays in a different Kubernetes namespace (to prevent one environment conflict to another), you need to tell Argo CD that this project may deploy to different namespaces. In this example, they are nationalgrid-dev and nationalgrid-staging. Replace nationalgrid with the name of your new project.
    argocd proj create nationalgrid \
        --dest https://kubernetes.default.svc,nationalgrid-dev \
        --dest https://kubernetes.default.svc,nationalgrid-staging \
        --src https://USERNAME:[email protected]/datopian/experiments/dx-helm-national-grid.git
  5. Create a Kubernetes namespaces for each of the environments.
    kubectl create namespace nationalgrid-dev
    kubectl create namespace nationalgrid-staging
  6. Create the application in Argo CD. Replace nationalgrid, repo, and path parameters with proper values for the new project. In the repo parameter, you should use the credentials from the GitLab Deploy Token. The values parameters will contain the values specific for this project/environment:
    argocd app create nationalgrid-staging \
      --repo https://USERNAME:[email protected]/datopian/experiments/dx-helm-national-grid.git \
      --path nationalgrid \
      --project nationalgrid \
      --values values.yaml \
      --values values.staging.yaml \
      --sync-policy automated \
      --auto-prune \
      --dest-server https://kubernetes.default.svc \
      --dest-namespace nationalgrid-staging \
      --server localhost:8080
  7. Add a DNS entry for the project (and the ingress defined in values.yaml or values.[env].yaml) in Cloudflare.

Now, access the newly deployed application in the address defined in Cloudflare. – ask Irio to define an address.

After you have your new app address defined in Cloudflare, you have to create certificate.
For dev/test purposes you can access your new app at localhost:3000 by port-forward:
kubectl port-forward service/aircan-dev-dx-ckan-aircan -n aircan-dev 3000:80 from the cli.

# How Do I Manage Multiple Environments For The Same Application?

You should follow the example given in dx-helm-national-grid, where we have an Argo CD “project” called National Grid with multiple “applications,” one for each environment.

# How Do I Debug

You can use Argo CD’s UI, or kubectl. In Argo CD, you can click in a Pod, select a container, and see the logs in real time. With kubectl, you can run the following:

kubectl get pods --all-namespaces
kubectl logs [pod name]
kubectl logs [pod name] --previous

kubectl logs shows the logs in realtime. The --previous flag is useful for when the container keeps restarting (usually due to a failed health check).

# How Do I Monitor

For now, please see the item related to debugging. At Datopian, we also use StatusCake for monitoring uptime.

# How Do I Rollback

For rolling back changes made to either Helm Chart or application code, you should write and push a new commit. Due to implementation details of Kubernetes, you might have to manually restart services (you can do that via Argo CD). For instance, if your application reads environment variables only on startup, it won’t matter if Kubernetes replaced them while it’s running. You should kill the container and see Kubernetes recreating it automatically.