The German SAP users’ organization just had their yearly event #DSAGJK19, using the analogy to movies as leitmotif through the conference. Visiting the conference with a testing mindset, the movie I had to think of was “A rebel without a cause” (You know, the one where James Dean had a red leather jacket…). With me in the role of Jim or Judy or any of the other youth in the movie and the SAP users as the adults… Kindof like the idea of being a rebel… loving quality in a world of business users….
You know the movie? If not – have a quick look at Wikipedia. Basically, it is about a rebellious young man with a troubled past who comes to a new town, finding friends and enemies. Or watch the trailer – from where I also have borrowed the pictures.
My parents (business and managers) would not understand me. I would just honestly try to do the right thing (test), but end up getting into fights between my parents (“You are tearing me apart”) when they discuss about what should be important for me… (make the business tester happy or automate) or on how much I should drink (my testing should cost…). And I still have to find my position in the gangs out there, battling happily along (the SAP testing tool providers). And with the gangs we fight for the right approach for testing SAP: method against tools… Who will chicken out (as Jim did in the movie, thus saving his life)?
Not to stress that analogy too much, let me summarize my experience at the conference: there are – except for the tool people – not too many participants and speakers at the DSAGJK19, who talk about testing. Looking at the agenda, there was (except for the talks of the tool people) exactly one talk with “test management” in the title – and that was about how to best formulate the business processes to be tested so that test management can thrive on that. However, the tool people could give some very interesting insights of typically perceived potential issues.
What is the Cause for Rebellion?
Some of you, who read my articles do not know SAP from the inside. So here a short list of some major differences in testing SAP vs. your homegrown application:
- SAP provides a huge pre-integrated standard system that already went through a lot of testing. It is all about business processes!
- SAP also provides many tools for monitoring this system
- You “only” need to test your customization plus its integration – it is just a “Nightmare on Elm street” – äh, I mean “in the Z-room”
- SAP will urge an upgrade schedule upon you
- You most likely will not have educated testers but business users who are told to help you testing
- There is something like an “old SAP” (ECC) and a “new SAP” S/4 Hana and many questions between those…
- And the “new SAP” is also available as a cloud only solution in the SAP Hana Enterprise Cloud (HEC), together with other cloud only applications like “Success Factors”, C/4 Hana, …
- And actually, SAP also has introduced a complete new cloud thinking approach, they call it SCP (SAP Cloud platform) including AI, IoT and other fancy stuff. This is also to be used for extensions – which is a pretty new paradigm as compared to the former approaches for customizations.
- Don’t say “deployment”, they call it ”transport”
I perceive as the biggest challenge the sheer size of the system, where it is just not possible to have an overview of the functionality nor the dependencies nor the customizations. How would you test that?
What is the Answer of SAP?
SAP actually provides many answers, ranging from unit testing (via QUnit, ABAPUint) via component testing frameworks (OPA5 for testing of UI5 modules) to test automation tools like ECatt. They have thought about integration in CI/CD and about interfaces to what I would call “proper testing tools”, … but mainly: they believe in a standardized approach: model your solution properly into nice process diagrams, then you basically have your business process tests already at hand. Add some test data and you are set.
Is this sufficient? What about further parts of the solution whenever you do not follow the standard SAP, for example in case you have customized too much. Customizations as part of the ECC core in the so called Z-Room will not fit to the standard SAP approach! Who will test anything other than happy path? And what if the users did not invest in proper process descriptions? How do you manage the frequent update schedule? How do you do regression test efficiently? And what if your landscape is complex, beyond those many ECC systems extending into SAP cloud applications, apps from the cloud? Where there is no standard business process?
The Gangs’ Perspective
Obviously, the tool people gangs also think that SAP’s answers do not suffice. They give a lot of answers to questions I did not hear at the conference. I still wonder if this is because I was at the wrong places, or if the SAP business user community takes the availability of test as granted. And one gang was not even there (Micro Focus), maybe as they just knew they are the old dog? So here are some of the questions where I found answers:
How can I reduce the effort for the business users?
- Make the regression test fully automated by recording something in production and replaying it on a changed QA system and a copy of the original QA system to see if all differences are wanted (Basis Technologies)
- Make test automation so easy that even business users can do it (SuXXesso)
- Provide SAP accelerator packages with test cases which you also only have to adjust to your customization (SuXXesso, Micro Focus, Tricentis)
- Make sure to reduce testing effort to the bare minimum. For this, use usage statistics and technical dependencies as known by the SAP system (Tricentis, Panaya, SAP Solution Manager, Micro Focus)
How can I test my customizations?
- Write and automate proper testcases (about all of the above)
- Extend given test cases
In a transaction based system, how can I get test data?
- Take a full system copy and anonymize it if needed (Libelle)
- Take consistent slices of data from production and anonymize it (EPI Use)
- Include possibilities to query the currently available test data via predefined SQLs (SuXXesso)
What about loosely coupled systems and other modern stuff?
- Use virtualized interfaces and intelligent mocks for APIs so that they can be included in usual automation approaches. (Tricentis, Microfocus)
- Simulate (mobile) devices (Tricentis, Micro Focus)
- Check speed of the app, the backend, the network (Tricentis, Microfocus, NeoTys)
- Security and governance checks (Onapsis)
- Security services for the complete SAP landscape (Akquinet)
I am sure, I have not covered everything, and I am still a bit overwhelmed and need to read more. But I would expect more methodological guidance answering questions like: what do I need to test in which level? What is tested by SAP and what is my job? How ca I get grips on the test of customization,s, and worse: of their transfer to S/4 Hana? And why do I have to rely on business testers? When would I need educated testers? And what if I seriously work in an agile way? Who chickens out first in the race of method against tool?
I understand, that these would not be the questions of tool people, but I would like to understand the real quality problems of the SAP community to give good answers, as a mix between methods and tool usage.
From the conference I got the impression that the SAP folks think that they don’t have an issue at least with testing the good old ECC system. Whatever they do, they do it since ever, there is not enough pain to change it. But I would like to ask the question:
Given that you already have a working strategy to test your SAP core ERP system (ECC), what is your next testing challenge:
- Improve the testing strategy for ECC
- Test in context of the S/4 Hana migration (brownfield)
- Test in context of a S/4 Hana greenfield approach
- Understand how to test the SAP applications in the Hana Enterprise Cloud
- Test the functionality in the SCP and its integration with other on premise or cloud applications
- Don’t know
I did not get the answers on the DSAGJK19. Need to find a different way to find out.