Whenever you talk about testing SAP, many people will shy away as they think that is boring. How wrong they are we proved at the #StugHH panel discussion “SAP Test Tool Slam – Successful Test of the S/4 Conversion”. One could learn so much from the panelists and the discussion with up to 40 participants. The panelists joined me coming from very different backgrounds:
Sebastian Geissler, as SAP Solman expert, claims that SAP Solution Manager is an indispensable tool for S/4 conversion projects.
Alexander Ertl from Tricentis points out that Tosca from Tricentis is supporting the project from impact analysis to automated testing thus allowing to quickly implement S/4 Hana in an integrated way.
Last but not least, Dennis Kallies emphasizes that the tools will not help you, if you do not think about a proper approach for the whole project and setup.
When testing the S/4 Conversion, you will always encounter some obstacles: on top of the “normal” complexity of SAP Systems, there are also some additional topics to be considered to tame this beast. However, as usual, there are many “heroes” around who come to your rescue.
During the panel discussion we talked about a few potential threads of the test of the S/4 conversion that might be new to the experienced ERP SAP test manager.
Project Size
This time, the whole system might be changed. Do you know what everything is? Is there anybody who does? As Sebastian pointed out, this time you might have to choose a decentralized approach to collect the information needed, so that it can be evaluated. For this you will need a tool, using which you cannot only collect the information, but also plan and lead the conversion project. Alexander added that the tool also should be able to execute automatic impact analysis, supporting the evaluation of the collected information. On top, all test cases might have to be evaluated for adaptation.
Focus on Test Risks
Dennis suggested a risk-based approach to test the conversion, in combination with an proper approach to identify the coverage of the changes. To identify such a streamlined scope you might need a change of methodology: how to identify the optimal test set for such a large project? Some of the increased risks result from more changes of functionality, changes of the UI and the technology used for it. Other risks relate to possible extensions into the cloud or to interfaces to other systems.
Documentation
Derived from the project size, there will be a lot of documentation – and the test cases are only a part of the documentation. You will need to setup an approach to identify what needs to be documented and how to do version control for it. You need to identify, what can be taken from the SAP best practices, what is new. SAP Solman gives you the opportunity to build upon the S/4 best practices, and tool providers like TOSCA provide accelerators to test those. But you also need a proper approach to extend those to your needs. And other than for smaller ERP adaptation projects, you might have to think about documenting everything.
Tools
What is the right tool for you – depends on what you need. It might be an option to use the SAP Solman for the documentation, and some test tool for the test and test management – consider using the best of two worlds. If you have less money or no experiences with test tools, you might think about using SAP Solman (as of Version 7.2) and other SAP tools to do everything. If you do not want wo invest in SAP Solman setup, think about adopting to the “almost agile” pre-build customization “Focused Build” (I personally think this is as agile, as a big steamboat can get… ). For test automation, the SAP build in tool CBTA most likely is not as powerful as TOSCA test automation, but it is a start.
One very important fact to consider when choosing the tools is the overall system landscape: do you have extensions in the cloud? Then you would have to document those manually in SAP Solman. Do you use S/4 in the cloud? Then the upcoming SAP Cloud ALM might be the tool of your choice. Are you worried about integrations with other systems that use for example Selenium tests? Then you might need an interface to those. Or you have a tool that integrates all different kinds of tests…
Whatever tool you choose – the beast SAP will need proper tools for automated impact analysis so that you can always adjust your test scope according to the changes. SAP suggests using TBOMs which in turn enable the Business Process Change Analyzer (BPCA) to determine whether or not an executable unit is affected by a change. This knowledge can be used to pick the right test cases – if connected. Setting up TBOMs takes some time and a proper approach, so it is not done very often. But if done, you lay the foundation for later savings.
And now?
The discussion in the panel had many more details, but also, we just touched on a few topics that might need a closer look: how should your system landscape look? 3 or 4 or even more systems? How do you handle dual maintenance between the still productive and the migrated S/4 system? How does the role of the test manager change? And more details to how we use SAP Solman in an optimal way? There are many more topics, it remains exciting to identify and master the challenges of testing the S/4 Migration. More to come here? Do you have questions? Please feel free to ask in the comments, and I will find people to answer them.
P.S.: You wonder who is the beauty guarded by the beast? Sorry to say, it is not you, who needs rescue. In my eyes the beauty of testing SAP S/4 conversion lies in all those impressive tools and approaches that reduce the complexity and help you on your way.