Biggest challenges for SAP Integration in 2020

/0 Comments/in , , /by

This blog contains some of the biggest trends in SAP Integration in 2020. Hopefully, you have already progressed with some of the areas, so you will have more time to focus on the other topics.

The topics covered are:

If you liked the blog and content please help share it with your team, Linked, Twitter or your favorite social network

Migration

Ever since the dawn of software age, there has always been a task to perform migrations. They just happen to ensure customers has the newest software and are able to achieve a lower TCO for their integration, and have support for their tools. The same is true for the SAP Integration portfolio it also undergoes changes. 2020 is the last year that 7.11, 7.3, 7.31 and 7.4 is supported, so in 2021 all customers must be on SAP PI/PO 7.5.

There will still be customers that have not managed to upgrade before then, but they must then accept their system is out of support. So if something bad happens they can move the interfaces to a newer platform.

In some cases, you can go for an upgrade to PI 7.5  with a stack split and then delete the ABAP stack. It is a bit risky and you will need to have good testing in place. It requires that you have ICO’s only.

Migration projects are a technical exercise that adds little value to the business, except they must be able to process it. The business therefore seldom cares about it, it is just some technical thing that must be completed. A migration project can take many months for you to perform, so you need to plan it. It is probably one of the tasks that you want outside help to perform because it will not be a core competency of your team to run migration projects. 

At Figaf we have created a set of tools that allow you to automate your SAP PI/CPI process, and they can also be used for migration. Here they can help you test everything is correct and identical to what was seen before. That way you know everything works as seen before.

We have also the only tool on the market that allows you to manage your migration process. You will be able to see how fast your migration is progressing and it does help with configuration and transports. I have a recording of the migration tool here.

If you a large investment in Seeburger Message Maps then the tool will enable you to automate the migration. In 2 minutes you can take the old message mapping and update it to a B2B Add-on structure with the Figaf Seeburger Migration Tool. That is a lot less than the 4-8 hours others have reported for migration of on message mapping.

The migration is not a straight path and can be challenging, but depending on your scenario there can cost savings in the operations aspect. 

I have created a list of my best resources for an SAP PI/PO migration here.

SAP Cloud Platform Integration CPI

SAP CPI has already been mainstream and the goto tool for creating new cloud-based integration. For customers using cloud services, it is the best tool compared to SAP PI/PO. 

It makes it much easier to create integrations because everything can be managed in just one iflow and modularity.

The amount of redelivered content is growing and it becomes a lot easier to make modifications to it. So you can create your own special processes that can be reused in other places. 

There is also the Cloud Foundry process that is going on. Currently, there is a large gap between NEO and CF. I would expect this gap to close since CPI is available on AWS, Azure, and Alicloud and Google is not on the road map. This should mean there are more resources to close the gaps. Once it is done I would expect a CF would be the main place SAP is delivering new functions on SAP CPI, to get customers to move to the platform. 

Regarding SAP PI/PO there are still some scenarios that are much better to perform on CPI with regards to handling point to point messages, restarting and design simplifications. 

I do see there is a need in the platform for automation to make the processing easier of managing SAP CPI and I believe the Figaf tools can help you in the process. It will handle testing, governance, transports, documentation, and support. It will also allow you to develop faster with Git integration, so you can develop Groovy scripts easier. 

If you are new to SAP CPI the process of developing in SAP CPI is different than you are used to in SAP PI/PO. With greater power comes great responsibility. It is therefore recommended to learn how the tool works and how to leverage it. I have created a list of different concepts that you must master before creating your first SAP CPI iflow. 

Migration to CPI

Some SAP PI/PO customers have already migrated to SAP CPI and achieved some great savings in regards to operations. I do see that there will be a lot more customers moving to CPI. 

Some have already completed the migration and are seeing big returns in cost savings because the cloud platform is cheaper to run and manage. 

There are some tools that allow you to reuse message mappings but there is still some limitation for the process. I hope that we at Figaf can take our current migration tool from SAP PI/PO and leverage the same process for a CPI migration. We still have some steps to be completed before it is possible. 

Automate the delivery process for SAP Integration

The way SAP PI/PO has been transported and configured has not changed the last 10-15 years. It is basically happening in the same way. You will need to create some CTS+ transports and once they have been applied on target systems users must configure things manually. Testing was also done manually.

This is much different than the rest of the world now is thinking of delivering software much different. Where ideas as DevOps is the new norm most rapid development projects. It allows developers to release code much faster because the processes can be automated. 

I do see customers talking about how they are able to automate the delivery process of SAP PI or SAP CPI, so they can live up to a requirement to deliver integration faster.

There are a few aspects that the automation gives. It makes it easier to get other people to take over the code and manage it, this means you will not be as reliant on the same developers, so they can change jobs without affecting you. 

Figaf has a solution that allows you to automate the full delivery process so you don’t need to spend months creating your own tool for it. 

API Strategy

I have talked with a number of organizations. Most of them have some projects about exposing APIs either from the SAP systems or from other business systems, so their customers would be able to use it. 

It is quite a fascinating journey to start on. Just as when people started with an SOA journey in 2005. I guess the API is even more critical now and will help you build more applications. The business may also be more interested in exposing services, so the process may be faster. 

I guess you can get a lot of good ideas and inspiration for it and plan it well. In the development process, you probably have to be more agile and experiment. You need to understand that you cannot make correct APIs the first time. 

I do like the SAP API mgt platform, it is pretty easy to get started with. The great benefit is that you can expose your first Odata service on the internet really easy from your SAP backend systems. 

It may have some limitations to other better platforms. It may be a good place to get started and understand what APIs you have and how to expose them. If you find some great need or challenge you can migrate away from SAP API mgt or combine 2 platforms. 

I do think that it is something you will need to learn how to work on in your organization. There are some good books on the market that will make sense to read up on. 

I will recommend going through some training or material on the topics, so you will be able to judge how you want to start out. I would start with the SAP API mgt book but will probably also find a vendor neutral book. can

Citizen integration

Citizen integration is a hot topic. It is about allowing the business to self make new integration without having to contact IT for it. It will make it a lot easier to create the correct integrations, but also add a lot of complexity to the landscape. 

A good Citizen integration will require

  • A good way to govern access business users only see the data they are allowed to
  • Monitor so users can see what is going on
  • Scalable to more users can use it
  • The flexibility that users can find different ways to enrich or modify the content
  • Move from Business to IT without re-engineer everything

For BI reporting there already have been some tools like PowerBI alike. Here users can create their own reports one some predefined reports. It does require a good management process because the report can consume a lot of CPU and memory. I did read a good blog on the topic for BI where it has been possible for some time. I would assume that much of the same learning can apply to Integration. 

In SAP SuccessFactors there is a place where business experts can create their own reports and files based on some data. I’m not sure where they can send the files and share them. I do like the approach because nobody understands the data better than the business users do. 

As a normal user, I do like Zapier and use it for some of my integrations. It is pretty simple to distribute data between systems and map fields, so it would be something that many users will be able to configure. They do have some enterprise accounts but I’m not really sure it will scale to Fortune 500 companies with regards to simplicity, governance, and flexibility. 

In the SAP world, there are Open Connectors/Cloud Elements which also could provide some option to make it easier for businesses to exchange data in predefined formats. 

I’m looking forward to seeing what will be used and how tools will work in that perspective.

S/4 HANA migration and integration projects

Many SAP customers are considering how and if they want to upgrade to S4/HANA. Some have already upgraded, but most still have that journey ahead. Though some are considering if they want to or not. Let’s see how the support changes. 

If you “just” perform an upgrade you can mostly still use all your current integrations. You will need to test to see if everything still works. Some tables have been moved and some programs bay need to be changed.

If you start from scratch there are a lot of new interfaces, so no more need for using IDOCs. So it will instead be SOAP messages or events.

There is also the option to use AIF (Application Interface Framework) to manage . 

It will be something that you will be learning from so good luck on that process.

Help share the article

I hope you found some useful information in the blog. Please help share the page.

Simplifying your SAP Integration support

/0 Comments/in , /by

On our development SAP PI system, we a currently getting a lot of alerts. Some cache errors, some bad configurations, and some strange errors.

In many organizations, they just use the standard mail job to send all alerts happening on the PI system. You can then end up with a mailbox that looks completely full. Nobody knows how to process it and what has been handled. What alerts do not matter, and which is important. What do we have a solution for? 

In the AlertConsumerJob2 you can write texts on what to do for each job. But it will require quite a bit of management on the PI system to handle it. You will need steps like create new Alerts Rules, write a new text and schedule a new job. Not really optimal. 

I have seen this type of full inboxes at a number of clients and wanted to make a tool that could handle all the processing. So I created the Figaf Support tool. It was a standalone product, that we now have integrated into the Figaf suite. 

Figaf will receive and order all alerts from SAP PI/PO, SAP API Management, SAP CPI. The main feature is that it enables you to: 

  • Integrated action log, to view what needs to be processed
  • Simplify setup rules for each different alert setup with Xpath
  • Send an email 
  • Webhook to Jira, Slack or an SMS provider
  • Resent or Cancel messages
  • Mute alerts for some minutes
  • Create Ticket direct to fix the issue

You can see a demo of this in the video

  • For SAP PI it relies on CMBA (Standard alerts) enriched with PI message payload
  • For SAP CPI it has a filter to select which messages to download and treat as alert like Status = Failed, or Receiver = Log. Resent is possible with a script is applied to the flow
  • For SAP API management IRT will add some logging scripts that will use the KVM store to save alerts. 

The benefit is that you just have one way to handle all alerts for your SAP Integration. You will fast be able to set up alert processing and handling. 

You can get started now. It only takes 15 minutes to download and set up your first alert at that time. 

Download now at https://figaf.com/irt and get 30 days trial

Or try Figaf Cloud to manage CPI and API mgt

Monitoring of SAP CPI

/0 Comments/in , /by

One challenge about making an application that runs well is to be able to monitor it with less effort. And being proactive about errors.

SAP CPI comes with some limited screens for monitoring what is processed and active alerting.

SAP is providing the OData API, so you can make your own monitoring application if you want more than the standard monitor. You can develop the monitor by a number of ways from Analytics Cloud to an SAP UI5 application. They will all work but it is some work you will need to do. You can also use Solman, though I’m not sure it will give much other information. There is also some more advanced monitoring tools from SAP Focused Run and a cloud version, but they will probably require a large investment and implementation.

That is why we have added it as part of the Figaf IRT application. It gives you three essential views:

  1. Active alerting if a message is not being processed correctly, then it will allow you to set up rules. A rule could be to notify a specific person or to restart the message. For the restart to work, you will need to include a script that saves payload as an attachment in case of errors.
  2. Monitoring for processes. The Figaf Monitor allows users to set up a group of iFlows to be viewed together. You can then give access to Finance interface for the Finance people, and they will not be able to see anything from HR. The view also gives a better overview of Sender, Receiver, Application ID from the Table view. This will allow you to find the correct message faster.
  3. System status page. We have a page that logs the status of the CPI system every 5 minutes. You then get a nice graph over how the system performs. It also has actions os if latency is above 500ms you can send an email.

Here is a list of screenshots to show that it is possible to monitor SAP CPI better without needing to code anything.

  • Configuration of Rule for alert
  • System performance
  • Overview of the system flows
  • List of current alerts
  • Monitor of multiply specific iflows

I have recorded a video of this you can see it here.

If you want to try it out you can get started for just 150 EUR/month for our cloud version of Figaf Tool. You can try it out for free.

Try Figaf IRT to monitor your SAP CPI

Creating unit tests for SAP CPI

/0 Comments/in , /by

Cross-Post: This blog has also been posted on blogs.sap.com

The best way to be able to fix any of your integrations is to be able to run tests really fast. If you are doing SAP CPI development in ie. Groovy it can be a little challenging to set up test cases for your SAP CPI scripts.

There are a number of different blogs on how to make it possible. I think Eng Swee Yeoh has created a number of blogs on how to test SAP CPI.

You will need some boilerplate code to set up test cases and run and do your verifications. We have now added a way to automate the process.

Last week we showed how we could expose your SAP CPI and API management as a Git repository. In the Git repository, we have added mock services for all the necessary implementations, so it is possible to run the scripts without connecting to the services.

Since it is mock some parts of the running may not be 100% if you use some exotic functions. That is why you still should be testing in real CPI.

In one of the coming releases, we will also be adding mesageFactory to test the attachments that you have created.

The only thing you need to do is to add the SAP CPI jars you can download here.

Testing with Figaf

Last week we released a version of Figaf IRT that enabled you to expose your SAP CPI content via Git. With this integration, we now have another nice option. We can create unit tests based on your scripts.

If you have created test cases for an iflow in Figaf IRT then you have the option to “Generate Groovy Scripts Test data”. This will create test cases for all the scripts that you have in that iflow.

The program will create all input and output messages in a Json format like the following. This will allow Figaf IRT to create automated assertions of all the parameters in the message.

{
  "input" : {
    "headers" : {
      "SAP_MplCorrelationId" : "AF1eZ3x-sdvOIJSgilALr8Lldude",
      "CamelHttpResponseCode" : "200",
      "apikey" : "124234523",
 //  ....
    },
    "properties" : {
      "CamelStreamCacheUnitOfWork" : "DefaultUnitOfWork",
      "CamelSplitComplete" : "false",
      "CamelSplitIndex" : "1",
      "CamelCorrelationId" : "ID-vsa6859459-36905-1565339707446-5304-2",
      "CamelMessageHistory" : "[DefaultMessageHistory[routeId=Process_1, node=CallActivity_27_1566467962530], DefaultMessageHistory[routeId=Process_17, node=CallActivity_39_1566467962450], DefaultMessageHistory[routeId=Process_17, node=MessageFlow_34_1566467962454], DefaultMessageHistory[routeId=Process_17, node=setHeader13227], DefaultMessageHistory[routeId=Process_17, node=setHeader13228], DefaultMessageHistory[routeId=Process_17, node=to14489], DefaultMessageHistory[routeId=Process_17, node=removeHeader3781], DefaultMessageHistory[routeId=Process_17, node=removeHeader3782], DefaultMessageHistory[routeId=Process_17, node=removeHeader3783], DefaultMessageHistory[routeId=Process_17, node=to14490], DefaultMessageHistory[routeId=Process_17, node=CallActivity_41_1566467962463]]",
      ".hasMoreRecords" : "false",
   // ....
    },
    "body" : "{\r\n\"d\" : {\r\n\"__metadata\": {\r\n\"uri\": \"https://services.odata.org/V2/Northwind/Northwind.svc/\"type\": \"NorthwindModel.Customer\"\r\n}, \"CustomerID\": \"TOMSP\", \"CompanyName\": \"Toms Spezialit\\u00e4ten\", \"ContactName\": \"Karin Josephs\", \"ContactTitle\": \"Marketing Manager\", \"Address\": \"Luisenstr. 48\", \"City\": \"M\\u00fcnster\", \"Region\": null, \"PostalCode\": \"44087\", \"Country\": \"Germany\", \"Phone\": \"0251-031259\", \"Fax\": \"0251-035695\", \"Orders\": {\r\n\"__deferred\": {\r\n\"uri\": \"https://services.odata.org/V2/Northwind/Northwind.svc/Customers('TOMSP')/Orders\"\r\n}\r\n}, \"CustomerDemographics\": {\r\n\"__deferred\": {\r\n\"uri\": \"https://services.odata.org/V2/Northwind/Northwind.svc/Customers('TOMSP')/CustomerDemographics\"\r\n}\r\n}\r\n}\r\n}"
  },
  "output" : {
    
    //same as input
    
     }
}

It will also create a testing method like the following that contains all the scripts in the flow and link to all the test messages that you have in the repository. You can remove all the scripts that you do not want to be testing with.

Every time you run the generate test case it will update the file and add more test data with an increased number. So you can run generate as many times as you want it would be wise only to add each test case once, otherwise, you are not adding any value to the testing.

The generated test case looks like the following. And contains testing of all your scripts.

package com.figaf

import org.junit.jupiter.params.ParameterizedTest
import org.junit.jupiter.params.provider.ValueSource

class GroovyScriptsTest extends AbstractGroovyTest {

    @ParameterizedTest
    @ValueSource(strings = [
            "src/test/resources/test-data-files/SetHeaders2/processData/test-data-1.json"
    ])
    void test_SetHeaders2Groovy(String testDataFile) {
        String groovyScriptPath = "src/main/resources/script/SetHeaders2.groovy"
        basicGroovyScriptTest(groovyScriptPath, testDataFile, "processData", getIgnoredKeysPrefixes(), getIgnoredKeys())
    }

    @ParameterizedTest
    @ValueSource(strings = [
            "src/test/resources/test-data-files/setHeaders/processData/test-data-2.json",
            "src/test/resources/test-data-files/setHeaders/processData/test-data-3.json"
    ])
    void test_setHeadersGroovy(String testDataFile) {
        String groovyScriptPath = "src/main/resources/script/setHeaders.groovy"
        basicGroovyScriptTest(groovyScriptPath, testDataFile, "processData", getIgnoredKeysPrefixes(), getIgnoredKeys())
    }


    @Override
    List<String> getIgnoredKeys() {
        List<String> keys = super.getIgnoredKeys()
        keys.addAll(Arrays.asList())
        return keys
    }

}

I think this approach will make it a lot easier to set up testing and run a set of assertions. As you don’t need to do anything and all required information is added to the environment. If there are parameters that you want to exclude from your script you can add it to the getIgnoredKeys.

Custom testing

The standard way will only allow you to test input with the expected output. It may not be what you want to do in all cases. There may be test cases you are creating that need to be evaluated differently.

You then have the option to just send a message to the processing and then insert your assertions to the processing. An example of such a testing approach is the following. So here you can do all your custom assertions and get the errors if anything does not match.

An example of such a custom validation could look like the following.

package com.figaf

import org.assertj.core.api.Assertions
import org.assertj.core.api.SoftAssertions
import org.junit.jupiter.api.Test

class SetHeadersTest extends AbstractGroovyTest {

    @Test
    void customTest() {
        String groovyScriptPath = "src/main/resources/script/setHeaders.groovy"
        String testDataFilePath = "src/test/resources/test-data-files/setHeaders/processData/test-data-1.json"
        def (MessageTestData messageDataExpected, MessageTestData messageDataActual) =
        processMessageData(groovyScriptPath, testDataFilePath, "processData")
        String actualModeValue = messageDataActual.getProperties().get("newError3")
        Assertions.assertThat(actualModeValue).isNotNull()
        Assertions.assertThat(actualModeValue)
                .endsWith("Test3")
    }

    @Test
    void customTestSoftAssertions() {
        String groovyScriptPath = "src/main/resources/script/setHeaders.groovy"
        String testDataFilePath = "src/test/resources/test-data-files/setHeaders/processData/test-data-1.json"
        def (MessageTestData messageDataExpected, MessageTestData messageDataActual) =
        processMessageData(groovyScriptPath, testDataFilePath, "processData")
        String actualPropValue = messageDataActual.getProperties().get("newError3")
        String expectedPropValue = messageDataExpected.getProperties().get("newError3")

        SoftAssertions softly = new SoftAssertions()

        softly.assertThat(actualPropValue).isNotNull()
        softly.assertThat(actualPropValue).endsWith("Test3")
        softly.assertThat(actualPropValue).isEqualTo(expectedPropValue)

        softly.assertAll()
    }
}

In the examples you can see different ways to run the assertion.

Try Figaf IRT

There is a community version of Figaf IRT that allows you to run the application in single-user and with some limitations, but all of this should be working. It is all free.

I do hope that you see the functions and can see how much it can improve your usage of SAP CPI and want to buy the full package.

We are improving the functionality over time, so it will be easier to manage your SAP CPI system.

Speed up your SAP CPI development process

/0 Comments/in , /by

Thursday, I hosted the webinar “How to speed up SAP CPI development”. It is an important topic because developers need to find a way to learn how to create what they can do to improve the speed of the integration service.

I have been talking about the topic easier but this time we have added an important component. That is Figaf IRT now can make your development process much simpler. So now we really can speed up the development and also change management.

There are two areas where the application allows users to create faster integration.

  • Development in your favorite IDE, this is where we have improved a lot lately.
  • Change management, this is where you document changes and transport individual iflows.

You can see the presentation here.

Development process

When you are working with SAP CPI you will often end up having to edit an XSLT or a Groovy script. It is normally not really easy to do in the build-in editor. Sure there may be simple changes where you can do it, but if you want to make any complicated logic you need to create

We have been adding functions to make it a lot easier over the last few months. The biggest impact has been our Git integration.

We have also added a method to share scripts and other resources between iflows. The challenge with sharing of scripts between flows is to keep it synchronized and ensure a change does not break other iflows.

Git integration

The Git integration allows you to take your SAP CPI iflow and just expose it to all content to a Git repository. We have also added configuration files templates so you just can use them. With this, you get plugins that will allow users to deploy scripts from their IDE.

We have also added a unit testing component that allows users to test with real payloads. This speeds up the processing about ensure that you have the correct to start your groovy script. From your IDE you also have the option to test how the created iflows works.

Change management

We have build a solution that allows users to make it possible to document what they have done. I believe that all changes in an Integration environment must be because of business reasons like Service Request, Ticket or what is called in your method.

We have made it simple to link Service Request and the changes that happen. This speed up the development process.

It is also possible to transport individual iflows across your landscape. You can even use the virtual environment on your CPI system. This means that you are able to develop and test on the same tenant, so you can save an instance. Figaf also allows you to manage configuration a lot easier across your environment.

Testing is also a part of your release. You can run tests really simple on all your iflow. This makes it possible to see if your changes have a negative impact on your process. It is also possible to test with mock services, so you can test integrations even if they require you to have access to third-party apps, that do not support calling with the same data multiple times. Then it will enable you to test your flows, scripts are working the same way also after an upgrade.

Try IRT self

You can try out all the functions with Git or change management with our free community edition. If you enjoy it you can upgrade to any of our solutions that allow a lot more integration development options.

Download IRT now or run it in our cloud

Download Figaf IRT for free