Posts

Creating unit tests for SAP CPI

Cross-Post: This blog has also been posted on blogs.sap.com

The best way to be able to fix any of your integrations is to be able to run tests really fast. If you are doing SAP CPI development in ie. Groovy it can be a little challenging to set up test cases for your SAP CPI scripts.

There are a number of different blogs on how to make it possible. I think Eng Swee Yeoh has created a number of blogs on how to test SAP CPI.

You will need some boilerplate code to set up test cases and run and do your verifications. We have now added a way to automate the process.

Last week we showed how we could expose your SAP CPI and API management as a Git repository. In the Git repository, we have added mock services for all the necessary implementations, so it is possible to run the scripts without connecting to the services.

Since it is mock some parts of the running may not be 100% if you use some exotic functions. That is why you still should be testing in real CPI.

In one of the coming releases, we will also be adding mesageFactory to test the attachments that you have created.

The only thing you need to do is to add the SAP CPI jars you can download here.

Testing with Figaf

Last week we released a version of Figaf IRT that enabled you to expose your SAP CPI content via Git. With this integration, we now have another nice option. We can create unit tests based on your scripts.

If you have created test cases for an iflow in Figaf IRT then you have the option to “Generate Groovy Scripts Test data”. This will create test cases for all the scripts that you have in that iflow.

The program will create all input and output messages in a Json format like the following. This will allow Figaf IRT to create automated assertions of all the parameters in the message.

{
  "input" : {
    "headers" : {
      "SAP_MplCorrelationId" : "AF1eZ3x-sdvOIJSgilALr8Lldude",
      "CamelHttpResponseCode" : "200",
      "apikey" : "124234523",
 //  ....
    },
    "properties" : {
      "CamelStreamCacheUnitOfWork" : "DefaultUnitOfWork",
      "CamelSplitComplete" : "false",
      "CamelSplitIndex" : "1",
      "CamelCorrelationId" : "ID-vsa6859459-36905-1565339707446-5304-2",
      "CamelMessageHistory" : "[DefaultMessageHistory[routeId=Process_1, node=CallActivity_27_1566467962530], DefaultMessageHistory[routeId=Process_17, node=CallActivity_39_1566467962450], DefaultMessageHistory[routeId=Process_17, node=MessageFlow_34_1566467962454], DefaultMessageHistory[routeId=Process_17, node=setHeader13227], DefaultMessageHistory[routeId=Process_17, node=setHeader13228], DefaultMessageHistory[routeId=Process_17, node=to14489], DefaultMessageHistory[routeId=Process_17, node=removeHeader3781], DefaultMessageHistory[routeId=Process_17, node=removeHeader3782], DefaultMessageHistory[routeId=Process_17, node=removeHeader3783], DefaultMessageHistory[routeId=Process_17, node=to14490], DefaultMessageHistory[routeId=Process_17, node=CallActivity_41_1566467962463]]",
      ".hasMoreRecords" : "false",
   // ....
    },
    "body" : "{\r\n\"d\" : {\r\n\"__metadata\": {\r\n\"uri\": \"https://services.odata.org/V2/Northwind/Northwind.svc/\"type\": \"NorthwindModel.Customer\"\r\n}, \"CustomerID\": \"TOMSP\", \"CompanyName\": \"Toms Spezialit\\u00e4ten\", \"ContactName\": \"Karin Josephs\", \"ContactTitle\": \"Marketing Manager\", \"Address\": \"Luisenstr. 48\", \"City\": \"M\\u00fcnster\", \"Region\": null, \"PostalCode\": \"44087\", \"Country\": \"Germany\", \"Phone\": \"0251-031259\", \"Fax\": \"0251-035695\", \"Orders\": {\r\n\"__deferred\": {\r\n\"uri\": \"https://services.odata.org/V2/Northwind/Northwind.svc/Customers('TOMSP')/Orders\"\r\n}\r\n}, \"CustomerDemographics\": {\r\n\"__deferred\": {\r\n\"uri\": \"https://services.odata.org/V2/Northwind/Northwind.svc/Customers('TOMSP')/CustomerDemographics\"\r\n}\r\n}\r\n}\r\n}"
  },
  "output" : {
    
    //same as input
    
     }
}

It will also create a testing method like the following that contains all the scripts in the flow and link to all the test messages that you have in the repository. You can remove all the scripts that you do not want to be testing with.

Every time you run the generate test case it will update the file and add more test data with an increased number. So you can run generate as many times as you want it would be wise only to add each test case once, otherwise, you are not adding any value to the testing.

The generated test case looks like the following. And contains testing of all your scripts.

package com.figaf

import org.junit.jupiter.params.ParameterizedTest
import org.junit.jupiter.params.provider.ValueSource

class GroovyScriptsTest extends AbstractGroovyTest {

    @ParameterizedTest
    @ValueSource(strings = [
            "src/test/resources/test-data-files/SetHeaders2/processData/test-data-1.json"
    ])
    void test_SetHeaders2Groovy(String testDataFile) {
        String groovyScriptPath = "src/main/resources/script/SetHeaders2.groovy"
        basicGroovyScriptTest(groovyScriptPath, testDataFile, "processData", getIgnoredKeysPrefixes(), getIgnoredKeys())
    }

    @ParameterizedTest
    @ValueSource(strings = [
            "src/test/resources/test-data-files/setHeaders/processData/test-data-2.json",
            "src/test/resources/test-data-files/setHeaders/processData/test-data-3.json"
    ])
    void test_setHeadersGroovy(String testDataFile) {
        String groovyScriptPath = "src/main/resources/script/setHeaders.groovy"
        basicGroovyScriptTest(groovyScriptPath, testDataFile, "processData", getIgnoredKeysPrefixes(), getIgnoredKeys())
    }


    @Override
    List<String> getIgnoredKeys() {
        List<String> keys = super.getIgnoredKeys()
        keys.addAll(Arrays.asList())
        return keys
    }

}

I think this approach will make it a lot easier to set up testing and run a set of assertions. As you don’t need to do anything and all required information is added to the environment. If there are parameters that you want to exclude from your script you can add it to the getIgnoredKeys.

Custom testing

The standard way will only allow you to test input with the expected output. It may not be what you want to do in all cases. There may be test cases you are creating that need to be evaluated differently.

You then have the option to just send a message to the processing and then insert your assertions to the processing. An example of such a testing approach is the following. So here you can do all your custom assertions and get the errors if anything does not match.

An example of such a custom validation could look like the following.

package com.figaf

import org.assertj.core.api.Assertions
import org.assertj.core.api.SoftAssertions
import org.junit.jupiter.api.Test

class SetHeadersTest extends AbstractGroovyTest {

    @Test
    void customTest() {
        String groovyScriptPath = "src/main/resources/script/setHeaders.groovy"
        String testDataFilePath = "src/test/resources/test-data-files/setHeaders/processData/test-data-1.json"
        def (MessageTestData messageDataExpected, MessageTestData messageDataActual) =
        processMessageData(groovyScriptPath, testDataFilePath, "processData")
        String actualModeValue = messageDataActual.getProperties().get("newError3")
        Assertions.assertThat(actualModeValue).isNotNull()
        Assertions.assertThat(actualModeValue)
                .endsWith("Test3")
    }

    @Test
    void customTestSoftAssertions() {
        String groovyScriptPath = "src/main/resources/script/setHeaders.groovy"
        String testDataFilePath = "src/test/resources/test-data-files/setHeaders/processData/test-data-1.json"
        def (MessageTestData messageDataExpected, MessageTestData messageDataActual) =
        processMessageData(groovyScriptPath, testDataFilePath, "processData")
        String actualPropValue = messageDataActual.getProperties().get("newError3")
        String expectedPropValue = messageDataExpected.getProperties().get("newError3")

        SoftAssertions softly = new SoftAssertions()

        softly.assertThat(actualPropValue).isNotNull()
        softly.assertThat(actualPropValue).endsWith("Test3")
        softly.assertThat(actualPropValue).isEqualTo(expectedPropValue)

        softly.assertAll()
    }
}

In the examples you can see different ways to run the assertion.

Try Figaf IRT

There is a community version of Figaf IRT that allows you to run the application in single-user and with some limitations, but all of this should be working. It is all free.

I do hope that you see the functions and can see how much it can improve your usage of SAP CPI and want to buy the full package.

We are improving the functionality over time, so it will be easier to manage your SAP CPI system.

Test EDIFACT/X12 on SAP PI/PO

Yesterday I had a demostration with a client of the Seeburger Migration tool, that was doing a migration of their EDIFACT/X12 from Message mappings from Seeburger to B2B Add-on on the format. I wanted to show how it was possible to make it possible to test that the mappings were correctly migrated and converted.

I do think that this also could be useful for other SAP PI/PO customers, that are either considering a migration project or just want better control of what is happening to their EDI messages.

It is crucial to test all your EDI documents because of the volume of the business going that way. Some of the customers I have worked the EDI part was 20-50% of their turnover, so it is the part that is too big to fail. It is pretty difficult to test it normally because you need to have a good understanding of what can change in your mappings. Both when it comes to upgrades and also when it comes to changes done by developers.

As you can see of the video it is pretty simple to setup a test case based on data from your productive system. Because it is where you have the most reliable amount of data and variance. It is just to complex to test everything based manual test case creation, that is why we believe it should be automated.

We will set up both EDI document in both directions so you can see validation of EDI document and also that it works with the EDISeperator. We only show EDIFACT but X12 and EANCOM works the same way.

We do not currently have an integration with dual stack systems. We did implement it in the old release of IRT, so it will be possible to update it to the current version. We currently do no have access to a dual stack system, so if this is something you want. We hope you will help working with us to make it possible.

How about change management

So Figaf IRT is a lot more than just a testing application. We started doing testing of SAP PI/PO but have since expanded the service to handle the full change management. Just link the CTS transport with the Business request and you are able to do document all changes. And also test the changes that you make. It reuses all your EDI test cases for the process.

Try IRT

You can try out Figaf IRT tool. It just takes 15 minutes to set up on your laptop to see how easy it is and what it can do for your SAP PI/PO and CPI development. The solution can scale to run for your full enterprise.

Testing SAP CPI with Mock data service

If you start making any changes to your SAP CPI system you will end up with problems where you need to be able to test your integration flows, if you make modifications to your scripts. In most cases, it will require that you have access to the same backed system and create the same requests.

Figaf have been supporting testing of SAP CPI for about 1 year. We was able to do it in the following way.

  1. Switch the integration flow into trace mode.
  2. Let the user send some messages
  3. Figaf IRT could then pick up all the trace data and use it as a baseline.

To test an integration flow it did the following.

  1. Switch the integration flow into trace mode.
  2. Figaf would send the input data
  3. Figaf IRT could then pick up all the trace data
  4. Compare the data with the original data.

It did provide an easy way to test your iflows with and enabled users to compare the data. To see if anything had changed.

Why we needed a new way of testing

The old way is good if the flows you don’t have many backed services you are connecting with and they are able to reply the same data. If you have SuccessFactors it may be difficult to create or fire the same employ multiply times. Or if your interface is expecting a delta of data, then you will not without making modification be able to fetch the correct data.

And then we want to be able to run the test more often, so we can be sure nothing is impacted. It could be our new approach about adding shared resources to SAP CPI, so if you make a modification in a script it will update all the other iflows.

Or if possible be able to run it on the next release of SAP CPI to see that nothing is affected.

How testing SAP CPI with mock data work

The testing work a little different than if you have any receiver adapters. Figaf will create a copy of the iFlow, and replace all Receiver channels with a HTTP call to the Figaf IRT server. It will send information about the current test case and what part of the process it is in. The API exposed at the Figaf server will then return the HTTP headers that have been changed in the process and also the body.

As a user the only thing you need to change is how to select that the test should be performed with mock data.

If there is a change in the iflow then Figaf would automatically synchronize the testing copy, and deploy it.

You can see the demo of how the service works.

As you can see it works pretty simple to get to work. Just one tick mark on the Testing Template. The solution does work with Cloud Connector if you have CPI on your local installation of IRT, you will just need some settings to specify URL and cloud connector information when configuring the iflows.

What are the limits of mock data

With the mock data there is some limits. Some we can probably find a way to solve with more development and feedback from customers.

  • You will not be able to test your adapter.
  • You be able to test any binary objects in the headers, it is not possible to recover them.
  • It does not support Enrich scenarios at the moment.

Try it your self

You can download Figaf tool and run it in your own landscape or you can try it out in our cloud deployment.

SAP PI/PO Migration

There many companies working on migration or planning a migration of their SAP XI/PI to SAP PI/PO 7.5 systems. Over the years I have created a number of resources that make a lot of sense if you are in the process of considering a migration. Here is one place that you can find all my best resources on the topic.

Timeline

One of the drivers for migration is that the support of everything but 7.5 will end in December 2020. That is currently in 20 months time. Not a long time if you need to plan a project, get funding and running the upgrade. I would assume that the minimum time for a migration is 6 months once you got people working on the project. If you need to get the funding you know your companies budget process, so you better get started.

As I read the note 1648480 extended support will not be possible for the Java stack.

Options for upgrading

There are a number of options that you can use when upgrading your landscape. Product manager of PI/PO Alexander Bundschuh has written a list of the different options (13) that you have depending on where you are at the moment. What options for Upgrading SAP PI/PO

My recommendation is to go for a single stack system because the dual stack is not developed anymore. It will also improve your next upgrade because of a simpler architecture. You will have to covert your ABAP Mappings.

Your different options

There is also the option to migrate to SAP CPI, which is the cloud-based tool for Process Integration. It has some areas where it can replace a PI and BPM solution, but there are patterns that are not fully optimal yet. It will no doubt be the place where SAP is focusing their development, so you will see a number of improvements there.

If you are not planning to use PO functions like B2B Add-on, BPM/BRM then it is probably best to keep at current license but move to an AEX, Java only version of your system. The only reason they would be to run CPI content on your PO system. If you have a few ccBPM probably consider moving them to CPI or Cloud Workflow.

If you are using B2B Add-on or want to run CPI content locally the PO is the way to go.

There is the concept Stack split that I have heard some customers use. The idea is to upgrade your 7.31 dual stack system to 7.5 with dual stack. In 7.5 they cannot share a system ID so one of the systems will get a new System ID. You will then be able to delete the ABAP system. So you only have a single stack system. It does require that you are using ICO’s for all your integration. I’m not sure if this is a supported process and how it works. Maybe your consultants know how it works.

Outlook

SAP PI/PO 7.5 will be in maintenance until 2024. SAP has announced there will be a new version of SAP PI/PO in 2022. The goal of this version is to support a newer Java version and latest Databases. The release will depend on when drivers for databases and Java is availabe from vendors.

There is a roadmap for the SAP PI/PO you need to search for Process Orchestration and then login to see the roadmap.

From what I can see on the roadmap it does not look like BPM and BRM function will be improved. You probably need to find another solution for BPM process. It could be the SAP Cloud Platform Workflow or CPI that will allow you to run it. The place where you will see most development is on running CPI content on SAP PO.

Process

The process for migration is pretty simple. It is just something about planning what to do. It does make a lot of sense to get an understanding of how you want to migrate so you can get started and know how it will work. I think it will give you some information to start the migration from. It can be improved and need to be adapted to your model. There are two areas that you need to focus on.

  • Planning: In this, you try to understand what you should be doing with regards to migration. This is where you need to figure out how to upgrade certain elements. This should be considered before you scale up the team. In this phase, you will also be able to see how you automate the solution.
  • Execution: In this part is about making the migration as fast as possible and keep the risk low. Here is where you make a migration of a single interface, test it and then move it to production.

You can read the full post and see all slides and the replay at the link below.

Automation of the process

We have been making a lot of tools for the SAP Platform and they do make a lot of sense to apply in any migration project. The more you can automate the easier the process will be. Here are some of the things you can consider for your project.

Understanding your current integration

To understand what part of your current integration and message mappings, we have created a free tool that allows you to see what is going on. It can give you information about which message mappings you have used and how many times they have been executed in the last month. It can also give you information about which modules you have and how they are used. It is a free part of the Figaf IRT application so it is just to try it out, so you can plan your migration.

Seeburger Migration

If you have Seeburger and are considering what options you have for migration then we have a path for it. We have created an automated tool that can take your Seeburger Message mapping and convert it to a B2B Add-on mapping with a few click. You can then save many hours in the migration process. You just select which mapping you want to convert and which B2B structure you want to use. Then the application will take care of the rest. It will cost you a fixed fee per mapping.

You can read more on the application at Seeburger Migration tool or at how the seeburger migration works.

Testing your migration

If you are doing any upgrade or change of your landscape it is really useful to perform test to validate that it works correctly. I have in some cases seen that after an upgrade something on the mappings had changed behavior. It is, therefore, a good idea to be able to test that nothing is affected.

You have the following options to Test SAP PI.

  • Manual testing requires a lot of resources
  • SAP PIT a free tool from SAP to test from 7.5 sp14.
  • Figaf IRT a tool build to make it a lot easier to test SAP PI/PO. Is our tool designed to make upgrade testing faster
  • INT4 Iftt a ABAP tool made for testing interfaces. It is good for testing result in the ABAP tables

Comparison of the two testing tools, I would recommend the Figaf tool because it makes the testing a lot faster to create and allow you to test your first upgrade. It also contains more patterns for testing and is able to test modules which is important with B2B Add-on.

Automation of the migration process

SAP have a tool that allows you to migrate channels and ICOs. It is a part of the PI system and you just link the two systems together and can then migrate scenarios or channels.

Figaf is currently working on a tool to automate the full migration process. The goal is to make the process a lot easier and enable users to see if they have tested and performed configuration in the full landscape.

Anything missing

Is there something missing let me know.

If you any questions on your migration, sent a email to info@figaf.com.

Run Figaf IRT in your own Cloud

We have added the Figaf Cloud. as a way to allow users to run our tool to manage the SAP CPI and API management. It is a great way to scale and deploy your application.

We have it hosted in Google Cloud Platform in a Kubernetes cluster. It i pretty easy to do in GCP and it allow scaling of the application pretty easy. It did take some time to understand how to build the app and secure the application.

For customers, it is a really good way to get started using Figaf IRT because they don’t even need to install anything. And all their partners can get access to the network and run applications. So a customer can run all the application to perform DevOps on the SAP CPI and API management systems. The DevOps include

  • Monitoring and alerting
  • Change Tracking or Versioning, so you can link business requirement with a change
  • Documentation
  • Testing
  • Transport of CPI/API flows

With all the focus on data protection, it may prove to be difficult to control all your data that exists in the Figaf IRT solution. You want to have control of where it is located.

We have an ompremish installation of the Figaf IRT tool and it contains the same features as the cloud, though there can be a bit of delay with it because of the release cycles of the applications. One problem that we often hear is so where should we install the IRT application. It can sometime be deficult to find a database and a Virtual server where the Java application can run.

Cloud

For some customers it is much easier to go buy/start a server in the cloud from GCP, AWS or Azure. You can have a direct connection from the cloud enviorment to your on-premish system and thereby get access. The integration department can then get the server self and run it there.

Our installation is pretty simple you can basicly just start the application with the click of a button. It does require a bit more effort if you want to run it with specific databases and setup SSL. It take 30 minutes to install IRT if you have installed the database. The good thing about the cloud enviorments is they all have the PostgreSQL server that we use as a managed. It will then eliminate some of the Normal DB operations that you are doing and just handle it with a few click in the web configuration.

It will also be possible to provide upgrades fairly simple. We have made a docker image that you can use.

We have added a Docker image to DockerHub, so you can run IRT pretty simple to get started with using it. You will need to make a few configurations to get it running. We have a guide on running it on Azure with the hosted database.

The container for running IRT on a Docker image in one of the places starts at 200 USD a month, depending on how you want to scale and have it to perform.

Try it

You can also try Figaf IRT out on your own laptop fast.