Posts

Comparing Figaf IRT with SAP PIT

SAP has recently published SAP PIT, which is a testing tool for SAP PI/PO. Costumers have asked me about the difference between SAP PIT and Figaf IRT. Does it actually matter which tool you use? The answer is: Yes, it does matter. In this blog post, I will give you an overview to help you to understand the differences.

The Figaf tool is a lot faster to use for creating and running test cases on. It also have a lot better integration with a DevOps/Continues Delivery process so you can deliver your integrations lot faster.

Figaf  IRTSAP PIT
Test the followingSender Modules, Routing, Mappings, Receiver ModuleRouting, Mappings
Test case creationAutomatic process multiple recordings at one timeSetup one recording of time
System requirement recordingAll 7.31, 7.4 and 7.5 add a FigafModule or use SAP logging no patches required7.31, 7.4,7.5 patched after February 2019. Fetch data from dual-stack systems.
PI version to run test onAll7.5 SP14
Location of testing application or dataSeparate java server and databaseSAP PI system
Can test your upgrade to sp14YesNo
Running test casesRun full test suites automatedRun one interface at the time, Then run validation
ComparisonXML, JSON, Text, Edifact, X12, BinaryXML, Text
Patterns supportedAsync, Sync, EDISeperator, Bridges: sync-async, async-syncAsync, Sync
Visual display of differenceYesYes (with SP15)
Data anonymizationYes No
Export of test data to PITYesNo
Integrated with Release management and change management toolYesNo
Mapping of business systemsAutomaticAutomatic (with SP16)
Add test cases from failed messagesYesNo
Manual test case creationYesNo
Releases per year123-4 per year
Price10.000+ EUR/year Included in your license

This is based on the 7.5 SP16. SAP has a roadmap to improve the tool but I do not know it.

Biggest difference

In general, the testing of the two tools works the same. Figaf is faster at creating many test cases at once and be able to run validations. It supports more patterns for testing. It will save time for testing of affected changes.

The place where Figaf really stands out is when you look at the automation for running the full SAP PI/PO environment. It will enable you to run the test cases that are affected by change/transport. You can automate the delivery process including documentation of change, transport, configuration, governance, and of cause testing.

If you want to see how the full governance model looks for SAP PI/PO including testing of a change then watch this video.

Do you want to know more about Figaf IRT? Then visit www.figaf.com/irt.

What is the test coverage of your SAP PI message mappings

On a customer demo I got asked what is the test coverage of the tool. We do show in the UI how many ICO’s you have test for but how about message mappings and also modules used in your landscape and how many times they was run.

We already build a report that showed how many times a message mapping was run in a given period based on the data in the PI monitor. So it was just about combining the two sources to give users a good view of what is going on.

For each integration flow we added the number of test cases created with the IRT tool. Then number if then propaged down to each message mapping so we can show how many message mappings is tested and more important which is not.

We also added a tab with the modules used in your landscape and how they performed. Probably we also need to add the IRT test cases to this. But it also depend on how people want to with our with our our modules.

Check out the demo and then try it for free on your own system

You can try it out on your own system. Just download the Figaf IRT tool run it on your laptop and you will be able to see the data after the landscape have been downloaded.

Automated SAP PIT Test case creation

A month ago SAP released a testing tool for SAP PI/PO. One thing I have been contemplating a lot is how to test the first upgrade. One challenge with the SAP PIT testing tool is to be able to run a test on your first upgrade with the tool. I do see a problem with the first two upgrades. If you accept the premise that we need to test all upgrades for SAP PI/PO and therefore need a tool for it. Then it has some challenges for the beginning.

The first upgrade

The system needs you to implement OSS Note 2650265 which is an upgrade to MESSAGING SYSTEM SERVICE released February 2019 and applies to many services packs for 7.31 and above. If you want to implement this upgrade you will need to patch all components because they are linked in SAP Netweaver Java. This means that for this to work you need to implement quite a number of changes for it to work. I see the process as the following:

  1. Implement the patch according to the note on your development system.
  2. Since you don’t know if any thing is affected you will need to run a full test with your SAP PI
  3. Implement the note in production
  4. Once the note is in production you can upgrade your development system to 7.5 SP 14.
  5. Then you can start to create test cases on the 7.5 SP14 and validate that nothing has changed. You will then compare production data with data on SP14 so you cannot compare how the upgrade works.
  6. Once you are done with creating the test cases you can implement SP 14 in your landscape
  7. Next upgrade can be made a bit easier because you have the test data

For me it does sound like a lot of extra manual tests and changes to your landscape limiting how much you can do in the process.

Figaf IRT

The Figaf tools allow you to test all SAP PI 7.31, 7.4 and 7.5 system without installing new support packs. We have a number of options to record messages. Either as SAP is doing with looking in the log messages, except we can use a patch that is 2 years old to enrich your monitoring web services. Or we even have a web scraping option that is even older than it. But we do have a better solution which is to add a module to your processing chain. That way you can test much faster on your system with a lower impact on them.

Screenshot of the new button to export data to PIT

SAP Customers prefer to use standard tools, in the places where it makes sense and they can find enough value. We, therefore, are working on a way to enable you to export your Figaf IRT test cases to SAP PIT, so you can stop using the Figaf tools for testing. This way you can use Figaf to help test the first upgrades and then use SAP PIT for the future

We do hope that we are proving enough value together with ways to make your SAP testing better and faster, and also allow you tigth itnegration to test the interfaces that you are changing.

How how does the migration work

It is a pretty simple just process. Once you have a Testing Template in Figaf IRT and have run it you will see the Export to PIT button. The messages then exist on your SAP PI system. Figaf IRT will create a Test Suite in PIT with the same messages. IRT know which messages should be added to the test cases so it will request. Then it is just up to PIT to fetch the messages.

It is a licensed feature of the Figaf IRT so you will need to purchase a license, it can save you some time for your testing.

You can see a demo of it here.

There will be a few more changes to make for the process to work, like:

  • Set a name for the template
  • Send ignoring elements over to the test case
  • Update it the process if PITs API is changing since it is not published.

There may also come future development in the PIT tool, we can use to run this process better. Lets see in the next support packs. It is possible to add running of tests cases and integrate it with our DevOps appoach so you can test the interfaces that is changed by a mapping.

If you want to see how fast you can create test cases in Figaf and run your tests in it then download the tool for free and get started in an hour. We do also have many extra features that you will not find in SAP PIT.

Webinar replay: Why you should test SAP PI/CPI

Do you want to learn how to test your SAP PI/PO system for upgrades in a smarter way? That was the main focus on the webinar, I held Thursday 28., March 2019. At the webinar, I covered the following topics:

– Why you should be testing your solution
– How the SAP PI testing solution (PIT) works and its limitations. Like it is pretty difficult to test the first upgrade because there is a need to make patch uploads.
– What a tool for testing your interfaces should be able to
– How a test tool will improve your sap pi upgrade and patches

I do show a short demo of you can set up test cases with the Figaf tool and can create multiple test cases in just one go. Anonymize the test data so you can run a test with even sensitive data.

Here you can watch the full replay (47 minutes):

Please also see the slideshow

I have also made a slideshow, which I hope you will like:

So, are ready to take your workflow to the next level with Figaf Integration Regression Tool? Then push the button below.

The testing solution from SAP to test PI/PO called PIT

Friday the 8 of March was the day that SAP finally released SP 14 of SAP PI/PO 7.5. The big promise was the new testing solution that allowed users to test SAP PI message process. It is called PIT.

“TL;DR the testing solution from SAP has some good features for testing. If you are serious with your testing and want full coverage then try out the Figaf IRT tool. “

As a vendor with a similar product, I have been interested in knowing what was going on with it and how it was working. I had seen some initial working of it but did not know where it would end. If it was going to be a killer product then it would remove part of my market. No customers can see the solution and judge if it will be useful for their use cases.

The key takes away is that testing is an important part of your it process where it is upgrading/patching, development or change of your interfaces. It is something that customer have expected and would like to see improved.

It is also good to know that SAP is taking the same approach as we have for fetching and running test cases.

Let’s start with the good part.

The Good part of PIT

  • It is a part of your license so you do not have to pay separately for it.
  • It removes the excuse that you cannot run any tests because you can always set it up
  • It supports fetching test data from dual-stack systems so you can test when you are performing upgrades for the systems. You will need to apply some ABAP notes to allow the recording.
  • You don’t need to install extra systems to run it the testing application.
  • It has a shared repository of data that all developers with the right role can access.
  • You can configure comparison of attachments
  • There are batch jobs running message processing and validation.
  • It is standard software following SAP delivery and support method.
  • Messages can be stopped before reaching the adapter and thereby not send it to the receiver.
  • That no changes need to be made to your landscape except you need to switch logging on for a period of time for the interfaces you want to collect data from.
  • It gives a good overview over how many messages that are affected by a change in one element
One overview i like. In just one screen you can see where the differences are structure across the testing result

I have not seen any roadmaps of the tool and how much resources and new features will be delivered but it looks like there will be more effort in it. So this is all based on the initial release.

Check out the video that I recorded that show my first look at what the tool can do.

My first experience with the tool and what is possible.

I believe that PIT will probably run more tests that my tool because the availability of the tool and customers it is simple to install the system.

What are the requirements

The test system must be installed on a 7.5 sp 14 or above. It can only run a test on the system with that SP level.

PIT  is able to collect test data from AEX system 7.31+ that have been updated within the last 2 years (It requires this note 2650265, which seems to have been backported to all service packs on 7.31+).

PIT can also collect messages from ABAP based systems here you need to apply some more support packages to allow the system to read data.

Read more on

I still see some areas where we have advantages over the SAP tool. (Okay, I may be a little biased, but if you try it out you will hopefully too.)

Why you still need to buy my tool

  • It enables you to test your modules that way you are ensuring that it is also covered as a part of the test.
  • We have a number of options for testing including using the SAP logging, but unlike SAP we du support all 7.31+ releases because we have web scraping to get messages. It is not an ideal solution but allows you to create test data on your old system without having to patch them. Because then you will have to do testing.
  • The speed of recording multiple test cases is a lot faster with the IRT tool because it can handle a lot of configuration needed. You can select multiple iflows and record them at the same time.
  • We do have a graphical overview to view differences and add ignoring of certain elements like time stamps that you do not want to cover.
  • Figaf support patterns not covered by PIT like Async-sync, Sync-async, EDISeperator and pattern is determinated automatically
  • Comparison not only in XML but also JSON, X12, EDIFACT, Text, and Binary
  • Simple data anonymization. You can select your input payload and then with a few clicks analyze everything and insert random names or number on different locations. This allows you to test with sensitive data and comply with GDPR.
  • Allow you to run multiple jobs and test cases when you want or scheduled
  • Excel report of all results so you can show how it worked
  • Connection with our DevOps solution so you can ensure that you are testing all interfaces that are affected by your change. It also allows you to document what was changed.
  • For customers the ability to get new features or functions delivered in days.
  • And so on… I could probably find a few more areas that we cover better.

Will we integrate the Figaf IRT tool with PIT

Since SAP will have a lot of test cases running in the tool we may be able to add some additional benefits of running tests with it. There may be something that can be implemented. We can leverage the way SAP is processing messages if you don’t want to get the test messages into the adapter. We could also use the ability to send a message directly to the message processing without creating an XI30 message.

We cloud probably also leverage the SAP part about downloading messages from dual-stack systems, and use them in our landscape.

IT will probably depend on when customers are adopting 7.5 sp14.

Your take away

If you have a plan to apply 7.5 sp 14 in the near future to your landscape do evaluate it. And see how it fits your plan and idea for working with the tools.

If you want to tool that can help you set up test cases faster, give you full tracking of what is being developed then try out the Figaf IRT tool? And we do support CPI also in the same approach.

You can check out the tool and see if it will suit your testing capabilities better. Check it out at Figaf.com/IRT