Test Suite page

Test Suite page includes several tabs with various data related to the test suite:

On each tab you are able to edit the test suite data (to start editing click on edit button):

  • Test Suite Title.

    Test Suite Title must be unique.

  • Scheduled - if true, TestSuiteRunner task runs this test suite according to configured cron expression (configure TestSuiteRunner property on Application configuration page).

    Agent Objects used for testing should be licensed, otherwise, scheduled test suites won’t be run.

  • Send report by email - if true, TestSuiteResultNotifier task sends test suite excel report to email configured on Application configuration page (configure TestSuiteResultNotifier, SMTP host, SMTP port, SMTP username, SMTP password, Email Protocol, Email From, Email To properties on Application configuration page).

  • Use default test systems for running enables Run on behavior for scheduled test suite running - test suite will be run on all Agents licensed as test systems excluding marked as production systems.

    You can check licensed Agents on License page. Production systems can be checked on Agent Configuration.

To save changes click on save button, to reset - reset.

To run the test suite click on run or run on buttons:

  • Run uses static links to test objects defined in the test case.

  • Run on resolves these links dynamically at the moment of test run. It is possible to see how these links are resolved through Current Mapping Configuration button - it opens 'Mapping Configuration' dialog box:

    mapping configuration

    There are Agent Objects (parent nodes) and Test Objects (child nodes) in opened dialog box.

Test cases aren’t run on deleted test objects. Such objects are skipped and listed in a warning message box (Test cases will not be run on the following test objects because these objects are deleted: …​), while the other test objects and test cases of the run are processed. A test suite in which no test case has a runnable test object isn’t run at all, and no test suite run is created.
You can’t run test suite without linked test cases.

To create recordings click on Record messages button. It opens the dialog where you can select an integration object then Create recording requests dialog box will be opened.

  1. RECORDINGS tab looks:

    ts recordings

    Here you can:

    1. View Agent Object clicking on Agent Object Title (it opens Integration Object Details page).

    2. Delete recordings (see corresponding actions here).

    3. View recording configuration clicking on dots icon and then on Config button (see settings here).

    4. Copy recording clicking on dots icon and then on Copy button (see settings here).

    5. Poll remote messages for selected recordings.

    6. Create test cases for selected recordings clicking on play with plus.

  2. LINKED TEST CASES tab shows test cases linked to current test suite and looks:

    ts linked test cases

    Here you can:

    1. View Agent Object clicking on Agent Object Title (it opens Integration Object Details page).

    2. Add test cases clicking on plus button. It opens 'Add Test Cases to the test suite' dialog box:

      ts add test cases dialog

      Just select the Agent, the object and required test cases and click on 'OK'.

    3. Select test case(s) and do actions:

      1. Detach test case(s) from test suite clicking on red trash button. It deletes the link between current test suite and selected test case(s).

    4. View test case details clicking on details button. It opens Test Case Details page.

  3. LAST RESULT tab shows last test suite run result and looks:

    ts last result

    On this tab you can:

    1. View Agent and Test Objects clicking on Agent Object Title or Test Object Title. It opens Integration Object Details page.

    2. Poll remote results clicking on Poll remote results.

      Since Figaf Tool 2.15 automatic polling is added for SAP CPI Agents. Once the messages have been run successfully, automatic polling is started.

      Since Figaf Tool 2.15.2 active polling statistics can be viewed. If there is polling in progress for the integration objects and you open another poll remote data dialog, then

      • recordings/test runs which are in both polling requests will be marked as SKIPPED_AS_DUPLICATE and statistics will be copied from original item and shown on UI in the new one.

      • recordings/test runs which are not in the original polling request will be SKIPPED in the new one.

      Since Figaf Tool 2305 polling will not be started if there is already a started polling requests with same objects in progress or polling has been already completed for all selected objects.

      If you have polled almost all messages and you just need to poll small number of messages to finish testing, you can enable option Ignore messages cache during polling on Application configuration page. When this option is enabled, already polled messages won’t be skipped. It affects performance when the amount of messages is large and you just need to poll small number to finish testing.

    3. View last polling result clicking on Open Last Polling Result.

      Open Last Polling Result button is available after polling remote data and until page is refreshed.

    4. Export as a test case to PIT (only for PRO) clicking on Export as a Test Case to PIT.

      Export as a Test Case to PIT is available only for licensed version after the test suite running.

    5. Compare testing results and build reports clicking on check button.

      The option reprocesses comparison and report generation asynchronously. It makes sense to execute the option once you have configured a new "ignore rule".

    6. Do the following actions clicking on menu and choosing corresponding option:

      1. Rerun reruns required test case.

      2. View testcase opens 'Test Case Details' page.

      3. Download report.

        Report type can be one of 2 types:

        1. xlsx report:

          • Contains information about test run/test suite run results.

          • By default only failed results are added to xlsx report. If you want to change this behavior, go to Configuration → Application page and enable Show successful messages in the test suite’s report property.

        2. zip archive with 3 csv reports:

          • test-run-metadata.csv - contains common information about test run/test suite run (e.g. test object, counters for success/error/unexpected/not/compared/unfinished test run/test suite run results).

          • diff-report.csv - contains diff report for test run/test suite run.

          • processed-messages-report.csv - contains processed messages report for test run/test suite run.

        The report type is defined from Use bundled CSV report generation strategy property configured on Application configuration page. If it’s true, the report type is zip, otherwise, it’s xlsx.

      4. View results opens Test Run Details page.

  4. RESULTS HISTORY tab shows results of all test suite runs and looks:

    ts results history

    On this tab you can:

    1. Configure Running Date filter to hide some test suite run results.

    2. Select the result and do actions:

      1. Compare testing results and build reports clicking on check button.

        The option reprocesses comparison and report generation asynchronously. It makes sense to execute the option once you have configured a new "ignore rule".

      2. Poll remote results for unfinished runs clicking on Poll remote results.

        Since Figaf Tool 2.15 automatic polling is added for SAP CPI Agents. Once the messages have been run successfully, automatic polling is started.

        Since Figaf Tool 2.15.2 active polling statistics can be viewed. If there is polling in progress for the integration objects and you open another poll remote data dialog, then

        • recordings/test runs which are in both polling requests will be marked as SKIPPED_AS_DUPLICATE and statistics will be copied from original item and shown on UI in the new one.

        • recordings/test runs which are not in the original polling request will be SKIPPED in the new one.

        Since Figaf Tool 2305 polling will not be started if there is already a started polling requests with same objects in progress or polling has been already completed for all selected objects.

        If you have polled almost all messages and you just need to poll small number of messages to finish testing, you can enable option Ignore messages cache during polling on Application configuration page. When this option is enabled, already polled messages won’t be skipped. It affects performance when the amount of messages is large and you just need to poll small number to finish testing.

      3. View last polling result clicking on Open Last Polling Result.

        Open Last Polling Result button is available after polling remote data and until page is refreshed.

      4. Export as a test case to PIT (only for PRO) clicking on Export as a Test Case to PIT.

      5. Delete testing results clicking on red trash button.

    3. Download report of the test suite run result clicking on download button in the table row.

      Report type can be one of 2 types:

      1. xlsx report:

        • Contains information about test run/test suite run results.

        • By default only failed results are added to xlsx report. If you want to change this behavior, go to Configuration → Application page and enable Show successful messages in the test suite’s report property.

      2. zip archive with 3 csv reports:

        • test-run-metadata.csv - contains common information about test run/test suite run (e.g. test object, counters for success/error/unexpected/not/compared/unfinished test run/test suite run results).

        • diff-report.csv - contains diff report for test run/test suite run.

        • processed-messages-report.csv - contains processed messages report for test run/test suite run.

      The report type is defined from Use bundled CSV report generation strategy property configured on Application configuration page. If it’s true, the report type is zip, otherwise, it’s xlsx.

    4. View Test Suite Run details clicking on details. It opens Test Suite Run page.