Git integration

Since of 2.10 Figaf Tool supports integration with Git for CPI and API Management Agent systems. It includes:

  1. Synchronization of all IFlows(for CPI)/API Proxies(for API Management) with configured Git repository. All found changes will be saved in the repository during CTT synchronization. It’s also possible to initialize/reinitialize repository by pushing all objects through manual operation. See this section to learn more about configuration options.

  2. IRT Gradle plugin for running Figaf Tool test suites remotely through Gradle task. It helps to build flexible development workflow with unit and integration tests.

It’s needed to run the command git config --system core.longpaths true as an admin in command prompt if you use OS Windows in order to prevent a problem with long file names.

1. Git integration configuration

Open Configuration → Agents page, click Edit on needed Agent. If you don’t have Agent configured read this section at first. Enable checkbox Enable Git Integration. Then you will see the following fields:

agent git integration config

  1. Skip Synchronization Of IFlows In Git (CPI) or Skip Synchronization Of API Proxies In Git (Api Management) - if true, IFlows or API Proxies won’t be added to git repository (only folders will be initialized).

  2. Update build.gradle automatically - if true, build.gradle will be updated automatically after synchronization. Enable this option if you don’t need to change build.gradle manually. If this option is disabled, you have to update build.gradle file when version of gradle plugin is upgraded.

  3. Update settings.gradle automatically - if true, settings.gradle will be updated automatically after synchronization. Enable this option if you don’t need to change settings.gradle manually. If this option is disabled, you have to update settings.gradle file when objects, stored in Git, are changed on remote system.

  4. Update .gitignore automatically - if true, .gitignore will be updated automatically after synchronization. Enable this option if you don’t need to change .gitignore manually.

  5. Add scenario documentation (only for CPI) - if true, scenario documentation (without the PNG) is generated and added to local folder (documentation isn’t added to repository). The option is enabled only if Skip Synchronization Of IFlows In Git isn’t enabled.

  6. Integrate with Git Pipelines (only for CPI) - if true, then Figaf Tool enables integration with git pipeline to validate updated IFlows. The option is enabled only if Skip Synchronization Of IFlows In Git isn’t enabled. See Integration with Git Pipelines for more details.

  7. Git Remote Url - url to remote repository.

  8. Local Path To Repo (only for on-premise version) - local folder, where repository will be cloned. If it’s not defined, it will use /<origin repository name> as a local path relatively to irt.jar. You can define either absolute or relative paths here, a relative path will be calculated from the same folder where Figaf Tool is executed.

  9. Branch Name - branch used by Figaf Tool. If it doesn’t exist, it will be created automatically. We recommend to define a special branch for Figaf Tool and then merge it with your working branch when it needs. Changes are committed to this branch during synchronization operation. But you can also initialize/reinitialize a full state of all objects at one time manually.

  10. Git Username - username for accessing remote Git repository. Figaf Tool will make commits on behalf specified user, but with its own name in commits.

  11. Git Password - password related to specified Git user.

  12. Ignore Files - paths used when you are downloading/uploading IFlow/ApiProxy through cpi-plugin or api-management-plugin. All matched files and folders won’t be added to bundled archive. Define only relative paths, the root is IFlow/ApiProxy bundled folder. By default it ignores src/test, build.gradle, settings.gradle, gradle.properties. The main reason to have some project files excluded is that they are not a part of bundled model. But some of them (like src/test, where you can keep your unit tests for groovy scripts) makes CPI IFlow non-operational, i.e. upload will be successful, but IFlow won’t work in runtime.

The following packages on CPI won’t be added to the repository during synchronization:

  1. Packages created by Figaf Tool through transport to virtual landscape item (their name starts with Z_<landscape item label>|, for example Z_QA|).

  2. Packages created by Figaf Tool for regression testing (their name starts with Z_Figaf|)

If you want to add lib files of some iflow to the index, put the following line into .gitignore:

!<package folder>/<iflow folder>/src/main/resources/lib/*.jar

Once you configure integration, save the agent, then click on dots of the related agent and then Init/Reinit Git repository button. It will start synchronization and init/reinit the repository.

2. Integration with Git Pipelines

Git Pipelines work only for repositories in GitHub.

The following configuration should be done to get the integration working:

  1. Set up a workflow yourself in Git repository:

    1. Configure figaf-validation.yml file:

      figaf-validation.yml 
      name: Figaf Pipeline

on:
  push:
    branches:
      - '**'

jobs:
  figaf-validations:
    runs-on: ubuntu-latest
    steps:
      - name: Checkout repository
        uses: actions/checkout@v2
        with:
          fetch-depth: 0
          token: ${{ secrets.GITHUB_TOKEN }}

      - name: Set up Java 11
        uses: actions/setup-java@v2
        with:
          distribution: 'adopt'
          java-version: '11'

      - name: Download and extract CPI Lint
        run: |
          curl -L -o cpilint.zip https://github.com/mwittrock/cpilint/releases/download/v1.0.4/cpilint-1.0.4.zip
          unzip -o cpilint.zip
          rm cpilint.zip
          chmod +x cpilint-1.0.4/bin/cpilint

      - name: Setup Gradle
        uses: gradle/gradle-build-action@v2
        with:
          gradle-version: 7.4

      - name: Get changed IFlow directories
        run: |
          changedIflowDirs=$(git diff --name-only --diff-filter=d HEAD^ | grep -E '.*/IntegrationFlow/[^/]+' | awk -F/ '{print $1"/"$2"/"$3}' | sort | uniq)
          echo "Changed IFlow Directories: $changedIflowDirs"

          EOF=$(dd if=/dev/urandom bs=15 count=1 status=none | base64)
          echo "changedIflowDirs<<$EOF" >> ${GITHUB_ENV}
          echo "${changedIflowDirs}" >> ${GITHUB_ENV}
          echo "$EOF" >> ${GITHUB_ENV}

      - name: CPI Lint validation
        run: |
          if [ -z "${{ env.changedIflowDirs }}" ]; then
            echo "No IFlow changes detected, skipping CPI Lint validation"
            exit 0
          fi

          readarray -t changedIflowDirs <<< "${{ env.changedIflowDirs }}"

          for integrationFlowDir in "${changedIflowDirs[@]}"
          do
            echo "integrationFlowDir: $integrationFlowDir"
            iflowName="$(basename "$integrationFlowDir")"
            packageName="$(basename "$(dirname "$(dirname "$integrationFlowDir")")")"
            (cd "$integrationFlowDir" && zip -r "../$iflowName.zip" .)

            cpilintCommand="${{ github.workspace }}/cpilint-1.0.4/bin/cpilint -rules=${{ github.workspace }}/.github/workflows/cpilint.xml -files=\"../$iflowName.zip\""
            echo "Executing cpilint: $cpilintCommand"
            cpilintStatus=0
            output=$(cd "$integrationFlowDir" && eval $cpilintCommand 2>&1) || cpilintStatus=$?

            if [ -z "$cpilintStatus" ]; then
              cpilintStatus=0
            fi

            echo "CPI Lint output: $output"
            echo "CPI Lint exit code: $cpilintStatus"

            echo "Validation name: CPI Lint" > "$integrationFlowDir/pipeline-validation.txt"
            echo "Execution time: $(date +'%Y-%m-%d %H:%M:%S')" >> "$integrationFlowDir/pipeline-validation.txt"
            echo "Exit code: $cpilintStatus" >> "$integrationFlowDir/pipeline-validation.txt"
            echo -e "Output: $output\n" >> "$integrationFlowDir/pipeline-validation.txt"

            rm -f "$integrationFlowDir.zip"
          done

      - name: Run unit tests
        run: |
          if [ -z "${{ env.changedIflowDirs }}" ]; then
            echo "No IFlow changes detected, skipping Unit tests validation"
            exit 0
          fi

          readarray -t changedIflowDirs <<< "${{ env.changedIflowDirs }}"

          for integrationFlowDir in "${changedIflowDirs[@]}"
          do
            iflowName="$(basename "$integrationFlowDir")"
            packageName="$(basename "$(dirname "$(dirname "$integrationFlowDir")")")"
            runTestsCommand="gradle :$packageName:iflow-$iflowName:test"
            echo "Executing runTestsCommand: $runTestsCommand"
            runTestsStatus=0
            output=$(eval $runTestsCommand 2>&1) || runTestsStatus=$?
            if [ -z "$runTestsStatus" ]; then
              runTestsStatus=0
            fi
            echo "Unit tests output: $output"
            echo "Unit tests exit code: $runTestsStatus"

            echo "Validation name: Unit tests" >> "$integrationFlowDir/pipeline-validation.txt"
            echo "Execution time: $(date +'%Y-%m-%d %H:%M:%S')" >> "$integrationFlowDir/pipeline-validation.txt"
            echo "Exit code: $runTestsStatus" >> "$integrationFlowDir/pipeline-validation.txt"
            echo -e "Output: $output\n" >> "$integrationFlowDir/pipeline-validation.txt"
          done

      - name: Commit and push changes
        run: |
          if [[ $(git status --porcelain) ]]; then
            git config --local user.email "[email protected]"
            git config --local user.name "Figaf"
            git add .
            git commit -m "Update pipeline-validation.txt"
            git push
          else
            echo "No changes to commit"
          fi

    2. Configure cpilint.xml:

      cpilint.xml 
      <?xml version="1.0"?>
<cpilint>
<rules>
<iflow-description-required/>
<disallowed-scripting-languages>
<disallow>javascript</disallow>
</disallowed-scripting-languages>
</rules>
</cpilint>
    Push the files to remote repository.

  2. Go to agent for which you want to configure integration with Git Pipeline and enable Integrate with Git Pipelines.

  3. Init/Reinit Git repository for the agent.

Now each updated IFlow will be validated automatically and the result will be saved into pipeline-validation.txt file in the IFlow folder in the repository. Then you can check Git Pipeline Validation Status on Tracked object page and this validation will be integrated into validation of transports.

Gradle version used locally and in Git pipeline should be the same. In our example version 7.4 is used.

3. Groovy scripts unit testing

Figaf Tool provides a possibility to generate unit tests for groovy scripts using the state of the recorded messages in CPI test cases. To use that feature you should open a Messages tab on the Test Case page. It has a button Generate groovy scripts test data. There are two options:

  1. Add to git repository. If git integration is configured and enabled, Figaf Tool will build test data files and commit them to repository.

  2. Download archive. All needed files will be archived and downloaded to your computer.

There are several files which are generated for testing:

  1. Files in common folder. These files are the basis for unit testing of groovy scripts.
    GroovyTestData and MessageTestData are a model.
    MessageImpl is a simple implementation of SAP Message interface.
    AbstractGroovyTest is an abstract class which does all the testing and has several methods which can be extended.

  2. Files in <iflow_package_name>/<iflow_name> folder.
    GroovyScriptsTest.groovy is a test which is ready to be run. It has all the needed links and methods.
    Files in resources folder are the test data for GroovyScriptsTest.groovy. Each file has input and expected output data. Each file contains an integer in the end of the name. If you select Download archive these numbers just start from 1. If you select Add to git repository Figaf Tool will not overwrite the existing test files. Instead of it Figaf Tool will find the maximum number among the existing files and increment this number.

You need to manually download CPI client jars and put them to the libs folder at root project level. If you want to change folder name, type your folder in Gradle dependency definition in common/build.gradle:

compile fileTree(dir: '../libs', include: '*.jar')

You can extend testing and write custom tests using AbstractGroovyTest.groovy. It’s not recommended to edit GroovyScriptsTest.groovy because it can be overwritten next time by Figaf Tool. It’s better to write custom tests in a separate class which should extend AbstractGroovyTest. The following methods can be overridden:

  1. processMessageData(String groovyScriptPath, String testDataFilePath, String methodName) - it calls methodName of the groovy script for test data and returns messageDataExpected and messageDataActual.

  2. assertMessages(MessageTestData messageDataExpected, MessageTestData messageDataActual, List<String> ignoredKeysPrefixes, List<String> ignoredKeys) - assert messageDataExpected and messageDataActual excluding the keys presented in ignoredKeysPrefixes and ignoredKeys.

  3. basicGroovyScriptTest(String groovyScriptPath, String testDataFilePath, String methodName, List<String> ignoredKeysPrefixes, List<String> ignoredKeys) - aggregate two previous methods.

  4. getIgnoredKeysPrefixes() - if a key starts from this list’s value, this key will be excluded from assertion in assertMessages method.

  5. getIgnoredKeys() - if a key is presented in this list, this key will be excluded from assertion in assertMessages method.

For example if you want to use processMessageData but have your own assertions you can easily do it.

import org.junit.jupiter.api.Test

import static org.assertj.core.api.Assertions.assertThat
import static org.mockito.BDDMockito.then

class SampleTest extends AbstractGroovyTest {

    @Test
    void customTest() {

        // given
        initMessageLogFactoryMocks()

        String groovyScriptPath = "src/main/resources/script/script1.groovy"
        String testDataFilePath = "src/test/resources/test-data-files/script1/processData/test-data-1.json"

        // when
        def (MessageTestData messageDataExpected, MessageTestData messageDataActual) = processMessageData(groovyScriptPath, testDataFilePath, "processData")

        // then

        // assert custom messageLog interaction
        then(messageLog).should().addAttachmentAsString("error.xml", messageDataExpected.getBody() as String, "application/xml")
        then(messageLog).should().addAttachmentAsString("responseError", "daniel test attachment", "text/plain")

        // assert properties/headers/body
        def propertyKey = "newError3"
        String actualPropertyValue = messageDataActual.getProperties().get(propertyKey)

        assertThat(actualPropertyValue).
                overridingErrorMessage("Property with key '%s' must be not null", propertyKey).
                isNotNull()
        assertThat(actualPropertyValue).
                overridingErrorMessage("Property with key '%s' must end with 'Test3', but actual value was '%s'", propertyKey, actualPropertyValue).
                endsWith("Test3")

    }
}