CI/CD for solutions

Use Instabase APIs to implement a continuous integration (CI) and continuous delivery (CD) approach to solution development, also known as continuous development.

Solution promotion workflow

Solution development typically occurs across three environments (also called deployments), such as DEV, UAT, and PROD. Solutions are developed in DEV, tested in UAT, and executed in PROD. While it’s possible to use the Instabase UI to export and import solutions across environments, a programmatic approach enables continuous development.

Assuming you have DEV, UAT, and PROD environments, the typical solution promotion process involves development, system integration testing, and production stages. Many of the steps in each stage can be strung together and automated using scripting.

• During the development stage in the DEV environment:

  1. Develop the solution and create any necessary supporting files, such as a package.json file that describes the solution and an icon.png file.

  2. Compile the flow.

  3. Either deploy the solution or publish the solution to Marketplace.

  4. Copy the solution files and any dependencies from the DEV environment to the UAT environment.

• During the system integration testing stage in the UAT environment:

  1. Either deploy the solution or publish the solution to Marketplace.

  2. Perform solution testing to validate functionality.

  3. Copy the solution files and any dependencies from the UAT environment to the PROD environment.

• During the production stage in the PROD environment:

  1. Either deploy the solution or publish the solution to Marketplace.

  2. Execute the solution.

CI/CD Toolkit

CI/CD Toolkit is an Instabase tool that uses Python scripting and Instabase APIs to automate compiling and copying a solution from a source environment to a target environment. CI/CD Toolkit can be launched from a CI/CD pipeline and includes samples for integration with GitHub Actions.

CI/CD Toolkit includes scripts for promoting solutions built with or without using Solution Builder. The promote_solution.py script, designed for solutions not built with Solution Builder, automates the majority of the steps in the solution promotion process, with the exception of development, testing, and execution in production. To promote a solution built in Solution Builder, use the promote_sb_solution.py script to automate those steps. The following diagram provides an overview of how CI/CD Toolkit can be used to help automate solution promotion between environments.

Environment variables

The following environment variables are referenced by the promote_solution.py script.

The following environment variables are referenced by the promote_sb_solution.py script.

For example, for a remote source flow, set these environment variables before running promote_solution.py:

export SOURCE_IB_API_TOKEN="OAUTH_TOKEN"
export TARGET_IB_API_TOKEN="OAUTH_TOKEN"

export SOURCE_IB_HOST="https://dev.domain.tld"
export SOURCE_WORKING_DIR="space/subspace/fs/Instabase Drive/Temp"
export SOURCE_SOLUTION_DIR="space/subspace/fs/Instabase Drive/Solution"
export REL_FLOW_PATH="Flows/Flow.ibflow"
export SOURCE_COMPILED_SOLUTIONS_PATH="space/subspace/fs/Instabase Drive"

export LOCAL_SOLUTION_DIR=""

export TARGET_IB_HOST="https://uat.domain.tld"
export TARGET_IB_PATH="space/subspace/fs/Instabase Drive/Solution"

Flags

The CI/CD Toolkit promote_solution.py script calls Instabase APIs to execute the steps required for automated solution promotion between environments. When executing the script, you can use flags to run a series of pre-selected subroutines or to select specific steps to run. For example, you can specify optional steps like compiling the .ibflow file to an .ibsolution file, or downloading the compiled .ibsolution file for use with environment pipelines and artifact repositories.

Workflow flags

Workflow flags run all required promotion steps for a local or remote source.

Promotion flags

Promotion flags execute specific steps in the promotion process. Workflow flags let you execute multiple steps.

The following flags are used with the promote_solution.py script.

The following flags are used with the promote_sb_solution.py script when promoting Solution Builder projects.

Source flags

Source flags can be used along with promotion flags to indicate the source location of the solution: a local repository or a remote Instabase environment.

Integration with common workflows

There are many ways to integrate CI/CD Toolkit into your software development lifecycle. These are some common examples.

Workflow between remote Instabase clusters

Use this flow to migrate a solution from one Instabase cluster to another, such as from DEV to UAT.

1. Develop your solution on Instabase.

2. Save the package.json file at the root level of your solution.

   Tip

   On subsequent promotions, be sure to update the solution’s version number in the package.json file.

3. Save the icon.png file at the root of your solution.

4. Configure your environment variables according to the provided template. Don’t include a LOCAL_SOLUTION_DIR variable.

5. Execute the following script:

    python cicd/promote_solution.py --remote-flow
   

6. Save the downloaded .ibsolution file as an artifact in your artifact repository.

Workflow between a local repository and Instabase cluster using GitHub Actions

You can use GitHub Actions to leverage a local GitHub repository and migrate the solution from that repo to an Instabase cluster.

1. In GitHub, set the environment variables according to the provided template. Make sure to include a LOCAL_SOLUTION_DIR environment variable.

2. Copy the local-github-actions-workflow.yml from CI/CD Toolkit to a .GitHub/workflows folder at the root level of your solution.

3. Copy the promote_solution.py script from CI/CD Toolkit to the root level of your solution.

4. Develop your solution on Instabase.

5. Create a new feature branch and copy the solution directory to the local repo folder.

6. Save the package.json file at the root level of your solution.

7. Save the icon.png file at the root level of your solution.

8. Push the solution to a feature branch of your GitHub repository.

9. Create a pull request to your main branch.

10. Merge the pull request.

When merged, the GitHub Action automatically performs these steps:

• Runs the command promote_solution.py --local_flow --download_ibsolution --set_github_actions_env_var.

• Creates a GitHub release tag with the version specified in the package.json file.

• Uploads the .ibsolution file to the release artifacts.

Workflow between remote Instabase clusters promoting a Solution Builder project

1. Develop your solution on Instabase.

2. Configure your environment variables according to the provided template. Don’t include a LOCAL_SOLUTION_DIR variable.

3. Execute the following script:

    python cicd/promote_sb_solution.py --remote_flow --download_ibsolution
   

4. Save the downloaded .ibsolution file as an artifact in your artifact repository.

Sample package.json

Solution packages require a package.json file that describes the solution. You can reference the examples here when creating your own. Packages used for continuous development have additional requirements beyond typical solution packages, including:

• A "dependencies" object which lists any models or packages that are linked to the solution. If there are no dependencies, you must still include the "dependencies" object and its fields, but with empty arrays as the values.

• The fields "owner" and "visibility", if the solution is deployed and copied to the Solution Dashboard (Deployed Solutions) rather than being published to Marketplace. Set "owner" to "IB_DEPLOYED" and "visibility" to "PUBLIC".

Example 1: A Form W-2 solution, with dependencies, that’s deployed and copied to the Solution Dashboard.

{
  "name": "Form W-2",
  "version": "0.0.2",
  "short_description": "Automatically process Form W-2",
  "long_description": "Extract key fields from Tax Form W-2 in the United States",
  "authors": [
    "Instabase"
  ],
  "solution_type": "ibflowbin",
  "accelerator_type": [
    "solution"
  ],
  "industry": [
    "Financial Services"
  ],
  "business_vertical": [
    "Retail & Consumer Banking"
  ],
  "use_case": [
    "Account Opening"
  ],
  "icon_file": "icon.png",
  "beta": true,
  "encryption_config": {
    "encryption_type": "v1"
  },
  "dependencies": {
    "models": ["model_for_cicd==0.0.1"],
    "dev_exchange_packages": ["package_for_cicd==0.0.1"]
  },
  "owner": "IB_DEPLOYED",
  "visibility": "PUBLIC"
}

Example 2: A Form W-2 solution, without dependencies, that’s published to Marketplace.

{
  "name": "Form W-2",
  "version": "0.0.2",
  "short_description": "Automatically process Form W-2",
  "long_description": "Extract key fields from Tax Form W-2 in the United States",
  "authors": [
    "Instabase"
  ],
  "solution_type": "ibflowbin",
  "accelerator_type": [
    "solution"
  ],
  "industry": [
    "Financial Services"
  ],
  "business_vertical": [
    "Retail & Consumer Banking"
  ],
  "use_case": [
    "Account Opening"
  ],
  "icon_file": "icon.png",
  "beta": true,
  "encryption_config": {
    "encryption_type": "v1"
  },
  "dependencies": {
    "models": [],
    "dev_exchange_packages": []
  }
}