Is it possible to only fail your run if it falls below a percentage threshold?

[austinmehmet]

What are you trying to achieve?

Wanting to run my end to end tests in my CI and only allow deployment if 70% of my tests pass. We have some unstable downstream services our E2E tests hit. Is it possible to set this threshold or has anyone solved this issue?

[kobenguyent]

@austinmehmet isn't easy to set it in the your CI/CD configuration level?

something like that has been there for Jenkins
https://issues.jenkins-ci.org/secure/attachment/32714/TestNG%20Plugin.jpg

[austinmehmet]

I am using Gitlab CI/CD which I don't think has that same configuration Jenkins uses. Also; how do you generate that results.xml file from codecept?

[kobenguyent]

I am using Gitlab CI/CD which I don't think has that same configuration Jenkins uses. Also; how do you generate that results.xml file from codecept?

@austinmehmet you can use this https://codecept.io/reports#xml to generate the xml report.