Reports
Automated tests need good reporting so that the test results are meaningful. For this purpose, we use the well-known Allure framework. We have a short guide on how to set up Allure. To get even more insights into your test results we enriched the reports by additional data. This article is about the additional features we have added and how to use them.
Sometimes it is necessary to know the exact link a step in the automation opened and sometimes it's just convenient to jump into a flow at a certain point. For this we have created a function which adds the currently displayed link as a step to the Allure report.
As soon as it is activated, every time a new page is called, a link to exactly this page is stored in the report. This way you can see if the link is correct and even open the page to have a look at it.
This is activated per default but can be deactivated via the neodymium.report.enableStepLinks property in the Neodymium configuration.
When looking into a failed testcase, it is necessary to get the corresponding test data to be able to replicate the issue. To not be forced to look through a bunch of code just to find the used test data, we have implemented the display of test data in JSON format for each test in the Allure report.
This is activated by default but can be deactivated if the neodymium.report.enableTestDataInReport property in the Neodymium configuration is set to false.
While activated, this feature adds the test data as an attachment with the name always starting with "Testdata" to the Allure report. It is important to mention, that it only applies to test data that is initialized with DataUtils or DataItem, both belonging to Neodymium.
If the test data changes during the test run, it is also possible to add the changes as an additional attachment to the Allure report. To accomplish that, we implemented a helper function addDataAsJsonToReport(String name, Object data) in AllureAddons, belonging to Neodymium. The parameter name is the name of the attachment and the parameter data is the changed testdata.
If you compare different json data and the comparison fails, you either get a huge and confusing error message or a short meaningless one, where you don't see what exactly went wrong. In order to prevent wasting time on looking through those kind of error messages, we have implemented the class JsonAssert, for what we utilized the already existing JSONAssert.
If assertEquals fails, an attachment named "Json Compare" containing the differences of the json data is added to the Allure report. If assertNotEquals fails, an attachment named "Json View" containing the json data once is added to the Allure report.