[debbugs-tracker] bug#30526: closed (java-testng: non-deterministic test failures)

[GNU bug Tracking System]

Your message dated Tue, 27 Feb 2018 08:19:50 +0100
with message-id <address@hidden>
and subject line Re: bug#30526: java-testng: non-deterministic test failures
has caused the debbugs.gnu.org bug report #30526,
regarding java-testng: non-deterministic test failures
to be marked as done.

(If you believe you have received this mail in error, please contact
address@hidden)


-- 
30526: http://debbugs.gnu.org/cgi/bugreport.cgi?bug=30526
GNU Bug Tracking System
Contact address@hidden with problems
> --- Begin Message ---
>
>
>
> Subject: 
>
> java-testng: non-deterministic test failures
>
>
>
>
> Date: 
>
> Mon, 19 Feb 2018 14:55:04 +0100
>
>
>
>
> User-agent: 
>
> mu4e 1.0-alpha3; emacs 25.3.1
>
>
>
>
> Sometimes I get this error when building java-testng 6.14.2:
>
> --8<---------------cut here---------------start------------->8---
> ===============================================
> TestNG
> Total tests run: 1630, Failures: 1, Skips: 0
> ===============================================
>
> Failures in <suite> :TestNG, <test> :Regression2
> test.testng387.TestNG387.testInvocationCounterIsCorrectForMethodWithDataProvider()
> StackTrace:
>  java.lang.AssertionError: Arrays do not have the same size:3 != 4
>         at 
> test.testng387.TestNG387.testInvocationCounterIsCorrectForMethodWithDataProvider(Unknown
>  Source)
> ... Removed 27 stack frames
>
> phase `check' failed after 297.3 seconds
> --8<---------------cut here---------------end--------------->8---
>
>
> -- 
> Ricardo
>
> GPG: BCA6 89B6 3655 3801 C3C6  2150 197A 5888 235F ACAC
> https://elephly.net
>
>
>
>
> --- End Message ---

--- Begin Message --- Subject: Re: bug#30526: java-testng: non-deterministic test failures Date: Tue, 27 Feb 2018 08:19:50 +0100

2018-02-26 8:23 GMT+01:00 Gábor Boskovits <address@hidden>:

2018-02-25 12:16 GMT+01:00 Julien Lepiller <address@hidden>:

Le Mon, 19 Feb 2018 14:55:04 +0100,
Ricardo Wurmus <address@hidden> a écrit :

> Sometimes I get this error when building java-testng 6.14.2:
>
> --8<---------------cut here---------------start------------->8---
> ===============================================
> TestNG
> Total tests run: 1630, Failures: 1, Skips: 0
> ===============================================
>
> Failures in <suite> :TestNG, <test> :Regression2
> test.testng387.TestNG387.testInvocationCounterIsCorrectForMethodWithDataProvider()
> StackTrace:
>  java.lang.AssertionError: Arrays do not have the same size:3 != 4
>         at
> test.testng387.TestNG387.testInvocationCounterIsCorrectForMethodWithDataProvider(Unknown
> Source) ... Removed 27 stack frames
>
> phase `check' failed after 297.3 seconds
> --8<---------------cut here---------------end--------------->8---
>
>

Hi,

I've updated testng to the latest version (6.14.3) and it seems to
solve the issue. Can you confirm it?

 

Ok, I will check that today. 

It seems, that these are related to some race conditions in the parallel execution of tests. We also have verifyThatTestMethodsRunInParallelThreads failing sometimes. I will report back with results today.

guix build --rounds=30 run to completion with this update, referring to discussion on the IRC I'm closing this bug as fixed.

--- End Message ---