problem-listed and/or excluded from tier1 and put in a later tier. Lets
not remove all SA testing in tier1 just because of a handful of issues.
I understand your concerns. But the original idea of tiered testing is
that tier1 failures are treated as urgent issues and to resolve. 
Here is list of test failures for 1000 runs of tier1 tests in Mach5. (I
am not able to provide a link here) Please note that all SA tests are
excluded on Solaris and MacosX already.
The failures in of 'runtime/handshake/' are relatively caused by
https://bugs.openjdk.java.net/browse/JDK-8214174 but should be also
fixed/excluded. SA tests are also unstable and there are no plans to fix
So it means that we are going to have tier1 tests unstable for a long time.
The possible way to make tier1 more stable would be to run only some
very basic sanity SA tests in tier1. Might be to develop new sanity
test which have some failover for existing SA bugs.
Post by JC Beyler
I agree with David. I am also concerned about us not detecting SA
breakages (which could happen along with hotspot changes) soon enough.
(Which was the primary reason to get these tests in).
Post by JC Beyler
My concern here, if we care about keeping the SA operational, is that
in tier3 these tests will not be covered by the jdk/submit testing
Post by Leonid Mesnik
Could you please review following fix which moves SA tests from
tier1 to tier3. There are some bugs which cause intermittent
failures of any test. SA tests fail intermittently are not stable
enough for tier1.
However failures are not very frequent. Also I don't think that
putting all test in Problemlist.txt is very good idea because it
left SA without any testing at all.
So now all SA tests which are included in hotspot_tier3_runtime group.