RFR (round 1), JDK-8214259: Implementation: JEP 189: Shenandoah: A Low-Pause Garbage Collector

[Roman Kennke]

Hi,

This is the first round of changes for including Shenandoah GC into
mainline.
I divided the review into parts that roughly correspond to the mailing lists
that would normally review it, and I divided it into 'shared' code
changes and
'shenandoah' code changes (actually, mostly additions). The intend is to
eventually
push them as single 'combined' changeset, once reviewed.

JEP:
  https://openjdk.java.net/jeps/189
Bug entry:

 https://bugs.openjdk.java.net/browse/JDK-8214259

Webrevs:
  http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/

For those who want to see the full change, have a look at the
shenandoah-complete
<http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shenandoah-complete/>
directory,
it contains the full combined webrev. Alternatively, there is the file
shenandoah-master.patch
<http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shenandoah-master.patch>,
which is what I intend to commit (and which should be equivalent to the
'shenandoah-complete' webrev).

Sections to review (at this point) are the following:
 *) shenandoah-gc
<http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shenandoah-gc/>
    - Actual Shenandoah implementation, almost completely residing in
gc/shenandoah

 *) shared-gc
<http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shared-gc/>
    - This is mostly boilerplate that is common to any GC
    - referenceProcessor.cpp has a little change to make one assert not
fail (next to CMS and G1)
    - taskqueue.hpp has some small adjustments to enable subclassing

 *) shared-serviceability
<http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shared-serviceability/>
    - The usual code to support another GC

 *) shared-runtime
<http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shared-runtime/>
    - A number of friends declarations to allow Shenandoah iterators to
hook up with,
      e.g. ClassLoaderData, CodeCache, etc
    - Warning and disabling JFR LeakProfiler
    - fieldDescriptor.hpp added is_stable() accessor, for use in
Shenandoah C2 optimizations
    - Locks initialization in mutexLocker.cpp as usual
    - VM operations defines for Shenandoah's VM ops
    - globalDefinitions.hpp added UINT64_FORMAT_HEX_W for use in
Shenandoah's logging
    - The usual macros in macro.hpp

 *) shared-build
<http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shared-build/>
    - Add shenandoah feature, enabled by default, as agreed with
Vladimir K. beforehand
    - Some flags for shenandoah-enabled compilation to get
SUPPORT_BARRIER_ON_PRIMITIVES
      and SUPPORT_NOT_TO_SPACE_INVARIANT which is required for
Shenandoah's barriers
    - --param inline-unit-growth=1000 settings for 2 shenandoah source
files, which is
      useful to get the whole marking loop inlined (observed significant
regression if we
      don't)

 *) shared-tests
<http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shared-tests/>
    - Test infrastructure to support Shenandoah
    - Shenandoah test groups
    - Exclude Shenandoah in various tests that can be run with selected GC
    - Enable/add configure for Shenandoah for tests that make sense to
run with it

 *) shenandoah-tests
<http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shenandoah-tests/>
    - Shenandoah specific tests, most reside in gc/shenandoah subdirectory
    - A couple of tests configurations have been added, e.g.
TestGCBasherWithShenandoah.java

I intentionally left out shared-compiler for now, because we have some
work left to do
there, but if you click around you'll find the patch anyway, in case you
want to take
a peek at it.

We have regular builds on:
  - {Linux} x {x86_64, x86_32, armhf, aarch64, ppc64el, s390x}
  - {Windows} x {x86_64},
  - {MacOS X} x {x86_64}

This also routinely passes:
  - the new Shenandoah tests
  - jcstress with/without aggressive Shenandoah verification
  - specjvm2008 with/without aggressive Shenandoah verification


I'd like to thank my collegues at Red Hat: Christine Flood, she deserves
the credit for being the original inventor of Shenandoah, Aleksey
Shiplëv, Roland Westrelin & Zhengyu Gu for their countless
contributions, everybody else in Red Hat's OpenJDK team for testing,
advice and support, my collegues in Oracle's GC, runtime and compiler
teams for tirelessly helping with and reviewing all the GC interface and
related changes, and of course the many early adopters for reporting
bugs and success stories and feature requests: we wouldn't be here
without any of you!

Best regards,
Roman

[Erik Joelsson]

Build changes look ok to me.

/Erik

Post by Roman Kennke
Hi,
This is the first round of changes for including Shenandoah GC into
mainline.
I divided the review into parts that roughly correspond to the mailing lists
that would normally review it, and I divided it into 'shared' code
changes and
'shenandoah' code changes (actually, mostly additions). The intend is to
eventually
push them as single 'combined' changeset, once reviewed.
  https://openjdk.java.net/jeps/189
 https://bugs.openjdk.java.net/browse/JDK-8214259
  http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/
For those who want to see the full change, have a look at the
shenandoah-complete
<http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shenandoah-complete/>
directory,
it contains the full combined webrev. Alternatively, there is the file
shenandoah-master.patch
<http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shenandoah-master.patch>,
which is what I intend to commit (and which should be equivalent to the
'shenandoah-complete' webrev).
 *) shenandoah-gc
<http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shenandoah-gc/>
    - Actual Shenandoah implementation, almost completely residing in
gc/shenandoah
 *) shared-gc
<http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shared-gc/>
    - This is mostly boilerplate that is common to any GC
    - referenceProcessor.cpp has a little change to make one assert not
fail (next to CMS and G1)
    - taskqueue.hpp has some small adjustments to enable subclassing
 *) shared-serviceability
<http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shared-serviceability/>
    - The usual code to support another GC
 *) shared-runtime
<http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shared-runtime/>
    - A number of friends declarations to allow Shenandoah iterators to
hook up with,
      e.g. ClassLoaderData, CodeCache, etc
    - Warning and disabling JFR LeakProfiler
    - fieldDescriptor.hpp added is_stable() accessor, for use in
Shenandoah C2 optimizations
    - Locks initialization in mutexLocker.cpp as usual
    - VM operations defines for Shenandoah's VM ops
    - globalDefinitions.hpp added UINT64_FORMAT_HEX_W for use in
Shenandoah's logging
    - The usual macros in macro.hpp
 *) shared-build
<http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shared-build/>
    - Add shenandoah feature, enabled by default, as agreed with
Vladimir K. beforehand
    - Some flags for shenandoah-enabled compilation to get
SUPPORT_BARRIER_ON_PRIMITIVES
      and SUPPORT_NOT_TO_SPACE_INVARIANT which is required for
Shenandoah's barriers
    - --param inline-unit-growth=1000 settings for 2 shenandoah source
files, which is
      useful to get the whole marking loop inlined (observed significant
regression if we
      don't)
 *) shared-tests
<http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shared-tests/>
    - Test infrastructure to support Shenandoah
    - Shenandoah test groups
    - Exclude Shenandoah in various tests that can be run with selected GC
    - Enable/add configure for Shenandoah for tests that make sense to
run with it
 *) shenandoah-tests
<http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shenandoah-tests/>
    - Shenandoah specific tests, most reside in gc/shenandoah subdirectory
    - A couple of tests configurations have been added, e.g.
TestGCBasherWithShenandoah.java
I intentionally left out shared-compiler for now, because we have some
work left to do
there, but if you click around you'll find the patch anyway, in case you
want to take
a peek at it.
  - {Linux} x {x86_64, x86_32, armhf, aarch64, ppc64el, s390x}
  - {Windows} x {x86_64},
  - {MacOS X} x {x86_64}
  - the new Shenandoah tests
  - jcstress with/without aggressive Shenandoah verification
  - specjvm2008 with/without aggressive Shenandoah verification
I'd like to thank my collegues at Red Hat: Christine Flood, she deserves
the credit for being the original inventor of Shenandoah, Aleksey
Shiplëv, Roland Westrelin & Zhengyu Gu for their countless
contributions, everybody else in Red Hat's OpenJDK team for testing,
advice and support, my collegues in Oracle's GC, runtime and compiler
teams for tirelessly helping with and reviewing all the GC interface and
related changes, and of course the many early adopters for reporting
bugs and success stories and feature requests: we wouldn't be here
without any of you!
Best regards,
Roman

[Vladimir Kozlov]

Hi Roman,

You need to check if shenandoahgc is disabled first - check DISABLED_JVM_FEATURES list (see code for jvmci).

Do you support 32-bit x86? You have OPENJDK_TARGET_CPU == xx86 check.

Do you support all x86 OSs?

Why you set VM CFLAGS when shenandoahgc is not enabled? It is in JvmFeatures.gmk.

I looked on C2 changes. It has INCLUDE_SHENANDOAHGC, checks and new gc specific nodes. This looks intrusive. I hope you
clean it up.

Thanks,
Vladimir

Post by Erik Joelsson
Build changes look ok to me.
/Erik

Post by Roman Kennke
Hi,
This is the first round of changes for including Shenandoah GC into
mainline.
I divided the review into parts that roughly correspond to the mailing lists
that would normally review it, and I divided it into 'shared' code
changes and
'shenandoah' code changes (actually, mostly additions). The intend is to
eventually
push them as single 'combined' changeset, once reviewed.
   https://openjdk.java.net/jeps/189
  https://bugs.openjdk.java.net/browse/JDK-8214259
   http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/
For those who want to see the full change, have a look at the
shenandoah-complete
<http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shenandoah-complete/>
directory,
it contains the full combined webrev. Alternatively, there is the file
shenandoah-master.patch
<http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shenandoah-master.patch>,
which is what I intend to commit (and which should be equivalent to the
'shenandoah-complete' webrev).
  *) shenandoah-gc
<http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shenandoah-gc/>
     - Actual Shenandoah implementation, almost completely residing in
gc/shenandoah
  *) shared-gc
<http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shared-gc/>
     - This is mostly boilerplate that is common to any GC
     - referenceProcessor.cpp has a little change to make one assert not
fail (next to CMS and G1)
     - taskqueue.hpp has some small adjustments to enable subclassing
  *) shared-serviceability
<http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shared-serviceability/>
     - The usual code to support another GC
  *) shared-runtime
<http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shared-runtime/>
     - A number of friends declarations to allow Shenandoah iterators to
hook up with,
       e.g. ClassLoaderData, CodeCache, etc
     - Warning and disabling JFR LeakProfiler
     - fieldDescriptor.hpp added is_stable() accessor, for use in
Shenandoah C2 optimizations
     - Locks initialization in mutexLocker.cpp as usual
     - VM operations defines for Shenandoah's VM ops
     - globalDefinitions.hpp added UINT64_FORMAT_HEX_W for use in
Shenandoah's logging
     - The usual macros in macro.hpp
  *) shared-build
<http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shared-build/>
     - Add shenandoah feature, enabled by default, as agreed with
Vladimir K. beforehand
     - Some flags for shenandoah-enabled compilation to get
SUPPORT_BARRIER_ON_PRIMITIVES
       and SUPPORT_NOT_TO_SPACE_INVARIANT which is required for
Shenandoah's barriers
     - --param inline-unit-growth=1000 settings for 2 shenandoah source
files, which is
       useful to get the whole marking loop inlined (observed significant
regression if we
       don't)
  *) shared-tests
<http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shared-tests/>
     - Test infrastructure to support Shenandoah
     - Shenandoah test groups
     - Exclude Shenandoah in various tests that can be run with selected GC
     - Enable/add configure for Shenandoah for tests that make sense to
run with it
  *) shenandoah-tests
<http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shenandoah-tests/>
     - Shenandoah specific tests, most reside in gc/shenandoah subdirectory
     - A couple of tests configurations have been added, e.g.
TestGCBasherWithShenandoah.java
I intentionally left out shared-compiler for now, because we have some
work left to do
there, but if you click around you'll find the patch anyway, in case you
want to take
a peek at it.
   - {Linux} x {x86_64, x86_32, armhf, aarch64, ppc64el, s390x}
   - {Windows} x {x86_64},
   - {MacOS X} x {x86_64}
   - the new Shenandoah tests
   - jcstress with/without aggressive Shenandoah verification
   - specjvm2008 with/without aggressive Shenandoah verification
I'd like to thank my collegues at Red Hat: Christine Flood, she deserves
the credit for being the original inventor of Shenandoah, Aleksey
Shiplëv, Roland Westrelin & Zhengyu Gu for their countless
contributions, everybody else in Red Hat's OpenJDK team for testing,
advice and support, my collegues in Oracle's GC, runtime and compiler
teams for tirelessly helping with and reviewing all the GC interface and
related changes, and of course the many early adopters for reporting
bugs and success stories and feature requests: we wouldn't be here
without any of you!
Best regards,
Roman

[Roman Kennke]

Hi Vladimir,

Post by Vladimir Kozlov
You need to check if shenandoahgc is disabled first - check
DISABLED_JVM_FEATURES list (see code for jvmci).

Why? If Shenandoah is disabled, it will be on this list, and therefore
not be built (see JvmFeatures.gmk).

Post by Vladimir Kozlov
Do you support 32-bit x86? You have OPENJDK_TARGET_CPU == xx86 check.
Do you support all x86 OSs?

We can build with 32bit, but it will not run. It seems indeed curious to
enable it. It's probably only there to allow 32bit builds with
Shenandoah enabled, just to verify that we have all the relevant LP64
checks in code in place. I will check it with my collegues.

Post by Vladimir Kozlov
Why you set VM CFLAGS when shenandoahgc is not enabled? It is in JvmFeatures.gmk.

This *disables* and excludes Shenandoah if not enabled.

+ifneq ($(call check-jvm-feature, shenandoahgc), true)
+ JVM_CFLAGS_FEATURES += -DINCLUDE_SHENANDOAHGC=0
+ JVM_EXCLUDE_PATTERNS += gc/shenandoah

pretty much the same pattern as zgc and other features.

and then we add some flags when Shenandoah is enabled:

+else
+ JVM_CFLAGS_FEATURES += -DSUPPORT_BARRIER_ON_PRIMITIVES
-DSUPPORT_NOT_TO_SPACE_INVARIANT
+endif

... which are required for building with Shenandoah enabled.

Post by Vladimir Kozlov
I looked on C2 changes. It has INCLUDE_SHENANDOAHGC, checks and new gc
specific nodes. This looks intrusive. I hope you clean it up.

There are a few places with INCLUDE_SHENANDOAHGC checks where it seemed
excessive to add a whole API just for a tiny, very GC specific check. We
still do have intrusive changes in loop*, which we are working on to
resolve. We declare+define all our GC specific nodes in
shenandoahBarrierSetC2.hpp and related Shenandoah-specific files. The
only things in shared code is the usual declarations in classes.hpp/cpp
and node.hpp to get is_ShenandoahBarrier() an similar queries. This
seems in-line with what other GCs do (look for e.g. LoadBarrier). Please
be specific which changes in opto you'd like to see resolved (and don't
look at loop* source files at this point ;-) ).

Thanks for reviewing!
Roman

Post by Vladimir Kozlov
Thanks,
Vladimir

Post by Erik Joelsson
Build changes look ok to me.
/Erik

Post by Roman Kennke
Hi,
This is the first round of changes for including Shenandoah GC into
mainline.
I divided the review into parts that roughly correspond to the mailing lists
that would normally review it, and I divided it into 'shared' code
changes and
'shenandoah' code changes (actually, mostly additions). The intend is to
eventually
push them as single 'combined' changeset, once reviewed.
   https://openjdk.java.net/jeps/189
  https://bugs.openjdk.java.net/browse/JDK-8214259
   http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/
For those who want to see the full change, have a look at the
shenandoah-complete
<http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shenandoah-complete/>
directory,
it contains the full combined webrev. Alternatively, there is the file
shenandoah-master.patch
<http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shenandoah-master.patch>,
which is what I intend to commit (and which should be equivalent to the
'shenandoah-complete' webrev).
  *) shenandoah-gc
<http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shenandoah-gc/>
     - Actual Shenandoah implementation, almost completely residing in
gc/shenandoah
  *) shared-gc
<http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shared-gc/>
     - This is mostly boilerplate that is common to any GC
     - referenceProcessor.cpp has a little change to make one assert not
fail (next to CMS and G1)
     - taskqueue.hpp has some small adjustments to enable subclassing
  *) shared-serviceability
<http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shared-serviceability/>
     - The usual code to support another GC
  *) shared-runtime
<http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shared-runtime/>
     - A number of friends declarations to allow Shenandoah iterators to
hook up with,
       e.g. ClassLoaderData, CodeCache, etc
     - Warning and disabling JFR LeakProfiler
     - fieldDescriptor.hpp added is_stable() accessor, for use in
Shenandoah C2 optimizations
     - Locks initialization in mutexLocker.cpp as usual
     - VM operations defines for Shenandoah's VM ops
     - globalDefinitions.hpp added UINT64_FORMAT_HEX_W for use in
Shenandoah's logging
     - The usual macros in macro.hpp
  *) shared-build
<http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shared-build/>
     - Add shenandoah feature, enabled by default, as agreed with
Vladimir K. beforehand
     - Some flags for shenandoah-enabled compilation to get
SUPPORT_BARRIER_ON_PRIMITIVES
       and SUPPORT_NOT_TO_SPACE_INVARIANT which is required for
Shenandoah's barriers
     - --param inline-unit-growth=1000 settings for 2 shenandoah source
files, which is
       useful to get the whole marking loop inlined (observed
significant
regression if we
       don't)
  *) shared-tests
<http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shared-tests/>
     - Test infrastructure to support Shenandoah
     - Shenandoah test groups
     - Exclude Shenandoah in various tests that can be run with
selected GC
     - Enable/add configure for Shenandoah for tests that make sense to
run with it
  *) shenandoah-tests
<http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shenandoah-tests/>
     - Shenandoah specific tests, most reside in gc/shenandoah
subdirectory
     - A couple of tests configurations have been added, e.g.
TestGCBasherWithShenandoah.java
I intentionally left out shared-compiler for now, because we have some
work left to do
there, but if you click around you'll find the patch anyway, in case you
want to take
a peek at it.
   - {Linux} x {x86_64, x86_32, armhf, aarch64, ppc64el, s390x}
   - {Windows} x {x86_64},
   - {MacOS X} x {x86_64}
   - the new Shenandoah tests
   - jcstress with/without aggressive Shenandoah verification
   - specjvm2008 with/without aggressive Shenandoah verification
I'd like to thank my collegues at Red Hat: Christine Flood, she deserves
the credit for being the original inventor of Shenandoah, Aleksey
Shiplëv, Roland Westrelin & Zhengyu Gu for their countless
contributions, everybody else in Red Hat's OpenJDK team for testing,
advice and support, my collegues in Oracle's GC, runtime and compiler
teams for tirelessly helping with and reviewing all the GC interface and
related changes, and of course the many early adopters for reporting
bugs and success stories and feature requests: we wouldn't be here
without any of you!
Best regards,
Roman

[Vladimir Kozlov]

Post by Roman Kennke
Hi Vladimir,

Post by Vladimir Kozlov
You need to check if shenandoahgc is disabled first - check
DISABLED_JVM_FEATURES list (see code for jvmci).

Why? If Shenandoah is disabled, it will be on this list, and therefore
not be built (see JvmFeatures.gmk).

Did you test with --with-jvm-variants=-shenandoahgc to make sure it is disabled?

May be I needed explicitly check jvmci because I need it early to check it for enable Graal build. So it is different
from your case.

Post by Roman Kennke

Post by Vladimir Kozlov
Do you support 32-bit x86? You have OPENJDK_TARGET_CPU == xx86 check.
Do you support all x86 OSs?

We can build with 32bit, but it will not run. It seems indeed curious to
enable it. It's probably only there to allow 32bit builds with
Shenandoah enabled, just to verify that we have all the relevant LP64
checks in code in place. I will check it with my collegues.

What about OS? Do you support Windows, MacOS? I did not see any OS specific changes. So may be it is alright.

Post by Roman Kennke

Post by Vladimir Kozlov
Why you set VM CFLAGS when shenandoahgc is not enabled? It is in JvmFeatures.gmk.

This *disables* and excludes Shenandoah if not enabled.
+ifneq ($(call check-jvm-feature, shenandoahgc), true)
+ JVM_CFLAGS_FEATURES += -DINCLUDE_SHENANDOAHGC=0
+ JVM_EXCLUDE_PATTERNS += gc/shenandoah
pretty much the same pattern as zgc and other features.
+else
+ JVM_CFLAGS_FEATURES += -DSUPPORT_BARRIER_ON_PRIMITIVES
-DSUPPORT_NOT_TO_SPACE_INVARIANT
+endif
... which are required for building with Shenandoah enabled.

My bad. Somehow I thought it was reverse. Too much coffee at the morning. :(
Looks good.

Post by Roman Kennke

Post by Vladimir Kozlov
I looked on C2 changes. It has INCLUDE_SHENANDOAHGC, checks and new gc
specific nodes. This looks intrusive. I hope you clean it up.

There are a few places with INCLUDE_SHENANDOAHGC checks where it seemed
excessive to add a whole API just for a tiny, very GC specific check. We
still do have intrusive changes in loop*, which we are working on to
resolve. We declare+define all our GC specific nodes in
shenandoahBarrierSetC2.hpp and related Shenandoah-specific files. The
only things in shared code is the usual declarations in classes.hpp/cpp
and node.hpp to get is_ShenandoahBarrier() an similar queries. This
seems in-line with what other GCs do (look for e.g. LoadBarrier). Please
be specific which changes in opto you'd like to see resolved (and don't
look at loop* source files at this point ;-) ).

Is it possible to reuse LoadBarrier by adding GC specific flag to it?
I am not against adding new nodes if really needed. My concern is about using GC name in node's names.
Yes, I am fine with very few INCLUDE_SHENANDOAHGC.

Thanks,
Vladimir

Post by Roman Kennke
Thanks for reviewing!
Roman

Post by Vladimir Kozlov
Thanks,
Vladimir

Post by Erik Joelsson
Build changes look ok to me.
/Erik

Post by Roman Kennke
Hi,
This is the first round of changes for including Shenandoah GC into
mainline.
I divided the review into parts that roughly correspond to the mailing lists
that would normally review it, and I divided it into 'shared' code
changes and
'shenandoah' code changes (actually, mostly additions). The intend is to
eventually
push them as single 'combined' changeset, once reviewed.
   https://openjdk.java.net/jeps/189
  https://bugs.openjdk.java.net/browse/JDK-8214259
   http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/
For those who want to see the full change, have a look at the
shenandoah-complete
<http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shenandoah-complete/>
directory,
it contains the full combined webrev. Alternatively, there is the file
shenandoah-master.patch
<http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shenandoah-master.patch>,
which is what I intend to commit (and which should be equivalent to the
'shenandoah-complete' webrev).
  *) shenandoah-gc
<http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shenandoah-gc/>
     - Actual Shenandoah implementation, almost completely residing in
gc/shenandoah
  *) shared-gc
<http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shared-gc/>
     - This is mostly boilerplate that is common to any GC
     - referenceProcessor.cpp has a little change to make one assert not
fail (next to CMS and G1)
     - taskqueue.hpp has some small adjustments to enable subclassing
  *) shared-serviceability
<http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shared-serviceability/>
     - The usual code to support another GC
  *) shared-runtime
<http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shared-runtime/>
     - A number of friends declarations to allow Shenandoah iterators to
hook up with,
       e.g. ClassLoaderData, CodeCache, etc
     - Warning and disabling JFR LeakProfiler
     - fieldDescriptor.hpp added is_stable() accessor, for use in
Shenandoah C2 optimizations
     - Locks initialization in mutexLocker.cpp as usual
     - VM operations defines for Shenandoah's VM ops
     - globalDefinitions.hpp added UINT64_FORMAT_HEX_W for use in
Shenandoah's logging
     - The usual macros in macro.hpp
  *) shared-build
<http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shared-build/>
     - Add shenandoah feature, enabled by default, as agreed with
Vladimir K. beforehand
     - Some flags for shenandoah-enabled compilation to get
SUPPORT_BARRIER_ON_PRIMITIVES
       and SUPPORT_NOT_TO_SPACE_INVARIANT which is required for
Shenandoah's barriers
     - --param inline-unit-growth=1000 settings for 2 shenandoah source
files, which is
       useful to get the whole marking loop inlined (observed significant
regression if we
       don't)
  *) shared-tests
<http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shared-tests/>
     - Test infrastructure to support Shenandoah
     - Shenandoah test groups
     - Exclude Shenandoah in various tests that can be run with selected GC
     - Enable/add configure for Shenandoah for tests that make sense to
run with it
  *) shenandoah-tests
<http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shenandoah-tests/>
     - Shenandoah specific tests, most reside in gc/shenandoah subdirectory
     - A couple of tests configurations have been added, e.g.
TestGCBasherWithShenandoah.java
I intentionally left out shared-compiler for now, because we have some
work left to do
there, but if you click around you'll find the patch anyway, in case you
want to take
a peek at it.
   - {Linux} x {x86_64, x86_32, armhf, aarch64, ppc64el, s390x}
   - {Windows} x {x86_64},
   - {MacOS X} x {x86_64}
   - the new Shenandoah tests
   - jcstress with/without aggressive Shenandoah verification
   - specjvm2008 with/without aggressive Shenandoah verification
I'd like to thank my collegues at Red Hat: Christine Flood, she deserves
the credit for being the original inventor of Shenandoah, Aleksey
Shiplëv, Roland Westrelin & Zhengyu Gu for their countless
contributions, everybody else in Red Hat's OpenJDK team for testing,
advice and support, my collegues in Oracle's GC, runtime and compiler
teams for tirelessly helping with and reviewing all the GC interface and
related changes, and of course the many early adopters for reporting
bugs and success stories and feature requests: we wouldn't be here
without any of you!
Best regards,
Roman

[Roman Kennke]

Hi Vladimir,

Post by Vladimir Kozlov

Post by Roman Kennke
Hi Vladimir,

Post by Vladimir Kozlov
You need to check if shenandoahgc is disabled first - check
DISABLED_JVM_FEATURES list (see code for jvmci).

Why? If Shenandoah is disabled, it will be on this list, and therefore
not be built (see JvmFeatures.gmk).

Did you test with --with-jvm-variants=-shenandoahgc to make sure it is disabled?

I tested with --with-jvm-features=-shenandoahgc (and will do so again
right after finishing this email, just to be sure). (Note: shenandoahgc
is a feature, not a variant).

Post by Vladimir Kozlov
May be I needed explicitly check jvmci because I need it early to check
it for enable Graal build. So it is different from your case.

Probably.

Post by Vladimir Kozlov

Post by Roman Kennke

Post by Vladimir Kozlov
Do you support 32-bit x86? You have OPENJDK_TARGET_CPU == xx86 check.
Do you support all x86 OSs?

We can build with 32bit, but it will not run. It seems indeed curious to
enable it. It's probably only there to allow 32bit builds with
Shenandoah enabled, just to verify that we have all the relevant LP64
checks in code in place. I will check it with my collegues.

What about OS? Do you support Windows, MacOS? I did not see any OS
specific changes. So may be it is alright.

Shenandoah is OS agnostic and we compile + run it on Windows
successfully. Adopters tell us it's fine on MacOS too.

Post by Vladimir Kozlov

Post by Roman Kennke

Post by Vladimir Kozlov
Why you set VM CFLAGS when shenandoahgc is not enabled? It is in JvmFeatures.gmk.

This *disables* and excludes Shenandoah if not enabled.
+ifneq ($(call check-jvm-feature, shenandoahgc), true)
+  JVM_CFLAGS_FEATURES += -DINCLUDE_SHENANDOAHGC=0
+  JVM_EXCLUDE_PATTERNS += gc/shenandoah
pretty much the same pattern as zgc and other features.
+else
+  JVM_CFLAGS_FEATURES += -DSUPPORT_BARRIER_ON_PRIMITIVES
-DSUPPORT_NOT_TO_SPACE_INVARIANT
+endif
... which are required for building with Shenandoah enabled.

My bad. Somehow I thought it was reverse. Too much coffee at the morning. :(

Yeah, maybe it's written the wrong way around (double-negation), but
that's how it is for other similar blocks too.

Post by Vladimir Kozlov
Looks good.

Thanks!

Post by Vladimir Kozlov

Post by Roman Kennke

Post by Vladimir Kozlov
I looked on C2 changes. It has INCLUDE_SHENANDOAHGC, checks and new gc
specific nodes. This looks intrusive. I hope you clean it up.

There are a few places with INCLUDE_SHENANDOAHGC checks where it seemed
excessive to add a whole API just for a tiny, very GC specific check. We
still do have intrusive changes in loop*, which we are working on to
resolve. We declare+define all our GC specific nodes in
shenandoahBarrierSetC2.hpp and related Shenandoah-specific files. The
only things in shared code is the usual declarations in classes.hpp/cpp
and node.hpp to get is_ShenandoahBarrier() an similar queries. This
seems in-line with what other GCs do (look for e.g. LoadBarrier). Please
be specific which changes in opto you'd like to see resolved (and don't
look at loop* source files at this point ;-) ).

Is it possible to reuse LoadBarrier by adding GC specific flag to it?
I am not against adding new nodes if really needed. My concern is about
using GC name in node's names.

That would be very weird. Shenandoah's barrier is *not* a load barrier.
GC's names in node name makes sense (to me) because the generated code
is GC specific, and it's used in .ad files to match. I guess we could
give it a more generic names (ReadBarrier, WriteBarrier,
GCCompareAndSwap, ..) and then match them in .ad and call via
BarrierSetAssembler to generate GC specific code. But it seems odd and
hard to read+understand to me.

Post by Vladimir Kozlov
Yes, I am fine with very few INCLUDE_SHENANDOAHGC.

Ok. Let's see how it looks like after Roland's latest changes are in.

Post by Vladimir Kozlov
Thanks,
Vladimir

Thanks for reviewing!

Roman

Post by Vladimir Kozlov

Post by Roman Kennke
Thanks for reviewing!
Roman

Post by Vladimir Kozlov
Thanks,
Vladimir

Post by Erik Joelsson
Build changes look ok to me.
/Erik

Post by Roman Kennke
Hi,
This is the first round of changes for including Shenandoah GC into
mainline.
I divided the review into parts that roughly correspond to the mailing lists
that would normally review it, and I divided it into 'shared' code
changes and
'shenandoah' code changes (actually, mostly additions). The intend is to
eventually
push them as single 'combined' changeset, once reviewed.
    https://openjdk.java.net/jeps/189
   https://bugs.openjdk.java.net/browse/JDK-8214259
    http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/
For those who want to see the full change, have a look at the
shenandoah-complete
<http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shenandoah-complete/>
directory,
it contains the full combined webrev. Alternatively, there is the file
shenandoah-master.patch
<http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shenandoah-master.patch>,
which is what I intend to commit (and which should be equivalent to the
'shenandoah-complete' webrev).
   *) shenandoah-gc
<http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shenandoah-gc/>
      - Actual Shenandoah implementation, almost completely
residing in
gc/shenandoah
   *) shared-gc
<http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shared-gc/>
      - This is mostly boilerplate that is common to any GC
      - referenceProcessor.cpp has a little change to make one
assert not
fail (next to CMS and G1)
      - taskqueue.hpp has some small adjustments to enable subclassing
   *) shared-serviceability
<http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shared-serviceability/>
      - The usual code to support another GC
   *) shared-runtime
<http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shared-runtime/>
      - A number of friends declarations to allow Shenandoah
iterators to
hook up with,
        e.g. ClassLoaderData, CodeCache, etc
      - Warning and disabling JFR LeakProfiler
      - fieldDescriptor.hpp added is_stable() accessor, for use in
Shenandoah C2 optimizations
      - Locks initialization in mutexLocker.cpp as usual
      - VM operations defines for Shenandoah's VM ops
      - globalDefinitions.hpp added UINT64_FORMAT_HEX_W for use in
Shenandoah's logging
      - The usual macros in macro.hpp
   *) shared-build
<http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shared-build/>
      - Add shenandoah feature, enabled by default, as agreed with
Vladimir K. beforehand
      - Some flags for shenandoah-enabled compilation to get
SUPPORT_BARRIER_ON_PRIMITIVES
        and SUPPORT_NOT_TO_SPACE_INVARIANT which is required for
Shenandoah's barriers
      - --param inline-unit-growth=1000 settings for 2 shenandoah
source
files, which is
        useful to get the whole marking loop inlined (observed
significant
regression if we
        don't)
   *) shared-tests
<http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shared-tests/>
      - Test infrastructure to support Shenandoah
      - Shenandoah test groups
      - Exclude Shenandoah in various tests that can be run with
selected GC
      - Enable/add configure for Shenandoah for tests that make
sense to
run with it
   *) shenandoah-tests
<http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shenandoah-tests/>
      - Shenandoah specific tests, most reside in gc/shenandoah
subdirectory
      - A couple of tests configurations have been added, e.g.
TestGCBasherWithShenandoah.java
I intentionally left out shared-compiler for now, because we have some
work left to do
there, but if you click around you'll find the patch anyway, in case you
want to take
a peek at it.
    - {Linux} x {x86_64, x86_32, armhf, aarch64, ppc64el, s390x}
    - {Windows} x {x86_64},
    - {MacOS X} x {x86_64}
    - the new Shenandoah tests
    - jcstress with/without aggressive Shenandoah verification
    - specjvm2008 with/without aggressive Shenandoah verification
I'd like to thank my collegues at Red Hat: Christine Flood, she deserves
the credit for being the original inventor of Shenandoah, Aleksey
Shiplëv, Roland Westrelin & Zhengyu Gu for their countless
contributions, everybody else in Red Hat's OpenJDK team for testing,
advice and support, my collegues in Oracle's GC, runtime and compiler
teams for tirelessly helping with and reviewing all the GC
interface and
related changes, and of course the many early adopters for reporting
bugs and success stories and feature requests: we wouldn't be here
without any of you!
Best regards,
Roman

[Roman Kennke]

Post by Vladimir Kozlov

Post by Roman Kennke

Post by Vladimir Kozlov
You need to check if shenandoahgc is disabled first - check
DISABLED_JVM_FEATURES list (see code for jvmci).

Why? If Shenandoah is disabled, it will be on this list, and therefore
not be built (see JvmFeatures.gmk).

Did you test with --with-jvm-variants=-shenandoahgc to make sure it is disabled?

Test: Built the Shenandoah-patched tree with
--with-jvm-features=-shenandoahgc. Then:

java -XX:+UnlockExperimentalVMOptions -XX:+UseShenandoahGC --version
Error occurred during initialization of VM
Option -XX:+UseShenandoahGC not supported

Also, we regularily build minimal and zero, and those disable Shenandoah
too, that's how we know this works reliably.

Thanks,
Roman

[Vladimir Kozlov]

Good. Thank you for testing. I don't have any more question about build changes.

I will wait C2 changes.

Post by Roman Kennke

Post by Vladimir Kozlov
Is it possible to reuse LoadBarrier by adding GC specific flag to it?
I am not against adding new nodes if really needed. My concern is about
using GC name in node's names.

That would be very weird. Shenandoah's barrier is *not* a load barrier.
GC's names in node name makes sense (to me) because the generated code
is GC specific, and it's used in .ad files to match. I guess we could
give it a more generic names (ReadBarrier, WriteBarrier,
GCCompareAndSwap, ..) and then match them in .ad and call via
BarrierSetAssembler to generate GC specific code. But it seems odd and
hard to read+understand to me.

I got that you can't reuse exisitng nodes.

Sorry, but I would prefer generic names for new nodes. You don't need BarrierSetAssembler if these new nodes are used
only for Shenandoah. Side note, don't use #ifdef in .ad files. The check for nodes generation should be done in C2 code
during ideal graph generation.

Thanks,
Vladimir

Post by Roman Kennke

Post by Vladimir Kozlov

Post by Roman Kennke

Post by Vladimir Kozlov
You need to check if shenandoahgc is disabled first - check
DISABLED_JVM_FEATURES list (see code for jvmci).

Why? If Shenandoah is disabled, it will be on this list, and therefore
not be built (see JvmFeatures.gmk).

Did you test with --with-jvm-variants=-shenandoahgc to make sure it is disabled?

Test: Built the Shenandoah-patched tree with
java -XX:+UnlockExperimentalVMOptions -XX:+UseShenandoahGC --version
Error occurred during initialization of VM
Option -XX:+UseShenandoahGC not supported
Also, we regularily build minimal and zero, and those disable Shenandoah
too, that's how we know this works reliably.
Thanks,
Roman

[Roman Kennke]

Post by Vladimir Kozlov
Do you support 32-bit x86? You have OPENJDK_TARGET_CPU == xx86 check.
Do you support all x86 OSs?

We enable this because we can actually run on such hardware. We fall
back to 'passive' mode though, which means Shenandoah acts more or less
like Parallel GC, and doesn't involve any barriers at all, and doesn't
do any concurrent processing. This has been useful for footprint
experiments.

Roman

Post by Vladimir Kozlov
Why you set VM CFLAGS when shenandoahgc is not enabled? It is in JvmFeatures.gmk.
I looked on C2 changes. It has INCLUDE_SHENANDOAHGC, checks and new gc
specific nodes. This looks intrusive. I hope you clean it up.
Thanks,
Vladimir

Post by Erik Joelsson
Build changes look ok to me.
/Erik

Post by Roman Kennke
Hi,
This is the first round of changes for including Shenandoah GC into
mainline.
I divided the review into parts that roughly correspond to the mailing lists
that would normally review it, and I divided it into 'shared' code
changes and
'shenandoah' code changes (actually, mostly additions). The intend is to
eventually
push them as single 'combined' changeset, once reviewed.
   https://openjdk.java.net/jeps/189
  https://bugs.openjdk.java.net/browse/JDK-8214259
   http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/
For those who want to see the full change, have a look at the
shenandoah-complete
<http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shenandoah-complete/>
directory,
it contains the full combined webrev. Alternatively, there is the file
shenandoah-master.patch
<http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shenandoah-master.patch>,
which is what I intend to commit (and which should be equivalent to the
'shenandoah-complete' webrev).
  *) shenandoah-gc
<http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shenandoah-gc/>
     - Actual Shenandoah implementation, almost completely residing in
gc/shenandoah
  *) shared-gc
<http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shared-gc/>
     - This is mostly boilerplate that is common to any GC
     - referenceProcessor.cpp has a little change to make one assert not
fail (next to CMS and G1)
     - taskqueue.hpp has some small adjustments to enable subclassing
  *) shared-serviceability
<http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shared-serviceability/>
     - The usual code to support another GC
  *) shared-runtime
<http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shared-runtime/>
     - A number of friends declarations to allow Shenandoah iterators to
hook up with,
       e.g. ClassLoaderData, CodeCache, etc
     - Warning and disabling JFR LeakProfiler
     - fieldDescriptor.hpp added is_stable() accessor, for use in
Shenandoah C2 optimizations
     - Locks initialization in mutexLocker.cpp as usual
     - VM operations defines for Shenandoah's VM ops
     - globalDefinitions.hpp added UINT64_FORMAT_HEX_W for use in
Shenandoah's logging
     - The usual macros in macro.hpp
  *) shared-build
<http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shared-build/>
     - Add shenandoah feature, enabled by default, as agreed with
Vladimir K. beforehand
     - Some flags for shenandoah-enabled compilation to get
SUPPORT_BARRIER_ON_PRIMITIVES
       and SUPPORT_NOT_TO_SPACE_INVARIANT which is required for
Shenandoah's barriers
     - --param inline-unit-growth=1000 settings for 2 shenandoah source
files, which is
       useful to get the whole marking loop inlined (observed
significant
regression if we
       don't)
  *) shared-tests
<http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shared-tests/>
     - Test infrastructure to support Shenandoah
     - Shenandoah test groups
     - Exclude Shenandoah in various tests that can be run with
selected GC
     - Enable/add configure for Shenandoah for tests that make sense to
run with it
  *) shenandoah-tests
<http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shenandoah-tests/>
     - Shenandoah specific tests, most reside in gc/shenandoah
subdirectory
     - A couple of tests configurations have been added, e.g.
TestGCBasherWithShenandoah.java
I intentionally left out shared-compiler for now, because we have some
work left to do
there, but if you click around you'll find the patch anyway, in case you
want to take
a peek at it.
   - {Linux} x {x86_64, x86_32, armhf, aarch64, ppc64el, s390x}
   - {Windows} x {x86_64},
   - {MacOS X} x {x86_64}
   - the new Shenandoah tests
   - jcstress with/without aggressive Shenandoah verification
   - specjvm2008 with/without aggressive Shenandoah verification
I'd like to thank my collegues at Red Hat: Christine Flood, she deserves
the credit for being the original inventor of Shenandoah, Aleksey
Shiplëv, Roland Westrelin & Zhengyu Gu for their countless
contributions, everybody else in Red Hat's OpenJDK team for testing,
advice and support, my collegues in Oracle's GC, runtime and compiler
teams for tirelessly helping with and reviewing all the GC interface and
related changes, and of course the many early adopters for reporting
bugs and success stories and feature requests: we wouldn't be here
without any of you!
Best regards,
Roman

[Roman Kennke]

Thanks for reviewing, Erik!

Roman

Post by Erik Joelsson
Build changes look ok to me.
/Erik

Post by Roman Kennke
Hi,
This is the first round of changes for including Shenandoah GC into
mainline.
I divided the review into parts that roughly correspond to the mailing lists
that would normally review it, and I divided it into 'shared' code
changes and
'shenandoah' code changes (actually, mostly additions). The intend is to
eventually
push them as single 'combined' changeset, once reviewed.
   https://openjdk.java.net/jeps/189
  https://bugs.openjdk.java.net/browse/JDK-8214259
   http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/
For those who want to see the full change, have a look at the
shenandoah-complete
<http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shenandoah-complete/>
directory,
it contains the full combined webrev. Alternatively, there is the file
shenandoah-master.patch
<http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shenandoah-master.patch>,
which is what I intend to commit (and which should be equivalent to the
'shenandoah-complete' webrev).
  *) shenandoah-gc
<http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shenandoah-gc/>
     - Actual Shenandoah implementation, almost completely residing in
gc/shenandoah
  *) shared-gc
<http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shared-gc/>
     - This is mostly boilerplate that is common to any GC
     - referenceProcessor.cpp has a little change to make one assert not
fail (next to CMS and G1)
     - taskqueue.hpp has some small adjustments to enable subclassing
  *) shared-serviceability
<http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shared-serviceability/>
     - The usual code to support another GC
  *) shared-runtime
<http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shared-runtime/>
     - A number of friends declarations to allow Shenandoah iterators to
hook up with,
       e.g. ClassLoaderData, CodeCache, etc
     - Warning and disabling JFR LeakProfiler
     - fieldDescriptor.hpp added is_stable() accessor, for use in
Shenandoah C2 optimizations
     - Locks initialization in mutexLocker.cpp as usual
     - VM operations defines for Shenandoah's VM ops
     - globalDefinitions.hpp added UINT64_FORMAT_HEX_W for use in
Shenandoah's logging
     - The usual macros in macro.hpp
  *) shared-build
<http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shared-build/>
     - Add shenandoah feature, enabled by default, as agreed with
Vladimir K. beforehand
     - Some flags for shenandoah-enabled compilation to get
SUPPORT_BARRIER_ON_PRIMITIVES
       and SUPPORT_NOT_TO_SPACE_INVARIANT which is required for
Shenandoah's barriers
     - --param inline-unit-growth=1000 settings for 2 shenandoah source
files, which is
       useful to get the whole marking loop inlined (observed significant
regression if we
       don't)
  *) shared-tests
<http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shared-tests/>
     - Test infrastructure to support Shenandoah
     - Shenandoah test groups
     - Exclude Shenandoah in various tests that can be run with
selected GC
     - Enable/add configure for Shenandoah for tests that make sense to
run with it
  *) shenandoah-tests
<http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shenandoah-tests/>
     - Shenandoah specific tests, most reside in gc/shenandoah
subdirectory
     - A couple of tests configurations have been added, e.g.
TestGCBasherWithShenandoah.java
I intentionally left out shared-compiler for now, because we have some
work left to do
there, but if you click around you'll find the patch anyway, in case you
want to take
a peek at it.
   - {Linux} x {x86_64, x86_32, armhf, aarch64, ppc64el, s390x}
   - {Windows} x {x86_64},
   - {MacOS X} x {x86_64}
   - the new Shenandoah tests
   - jcstress with/without aggressive Shenandoah verification
   - specjvm2008 with/without aggressive Shenandoah verification
I'd like to thank my collegues at Red Hat: Christine Flood, she deserves
the credit for being the original inventor of Shenandoah, Aleksey
Shiplëv, Roland Westrelin & Zhengyu Gu for their countless
contributions, everybody else in Red Hat's OpenJDK team for testing,
advice and support, my collegues in Oracle's GC, runtime and compiler
teams for tirelessly helping with and reviewing all the GC interface and
related changes, and of course the many early adopters for reporting
bugs and success stories and feature requests: we wouldn't be here
without any of you!
Best regards,
Roman

[Magnus Ihse Bursie]

Post by Erik Joelsson
Build changes look ok to me.

I agree, but I'd like to point out a (possible) style improvement. In
hotspot.m4,

if test "x$OPENJDK_TARGET_CPU" = "xx86_64" || test
"x$OPENJDK_TARGET_CPU" = "xaarch64" || test "x$OPENJDK_TARGET_CPU" ==
"xx86"; then

can be more succinctly expressed as

if test "x$OPENJDK_TARGET_CPU_ARCH" = "xx86" || test
"x$OPENJDK_TARGET_CPU" = "xaarch64" ; then


But it's a matter of preference, if you do not want to change. Otherwise
it looks good!

/Magnus

Post by Erik Joelsson
/Erik

Post by Roman Kennke
Hi,
This is the first round of changes for including Shenandoah GC into
mainline.
I divided the review into parts that roughly correspond to the mailing lists
that would normally review it, and I divided it into 'shared' code
changes and
'shenandoah' code changes (actually, mostly additions). The intend is to
eventually
push them as single 'combined' changeset, once reviewed.
https://openjdk.java.net/jeps/189
https://bugs.openjdk.java.net/browse/JDK-8214259
http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/
For those who want to see the full change, have a look at the
shenandoah-complete
<http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shenandoah-complete/>
directory,
it contains the full combined webrev. Alternatively, there is the file
shenandoah-master.patch
<http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shenandoah-master.patch>,
which is what I intend to commit (and which should be equivalent to the
'shenandoah-complete' webrev).
*) shenandoah-gc
<http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shenandoah-gc/>
- Actual Shenandoah implementation, almost completely residing in
gc/shenandoah
*) shared-gc
<http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shared-gc/>
- This is mostly boilerplate that is common to any GC
- referenceProcessor.cpp has a little change to make one assert not
fail (next to CMS and G1)
- taskqueue.hpp has some small adjustments to enable subclassing
*) shared-serviceability
<http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shared-serviceability/>
- The usual code to support another GC
*) shared-runtime
<http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shared-runtime/>
- A number of friends declarations to allow Shenandoah iterators to
hook up with,
e.g. ClassLoaderData, CodeCache, etc
- Warning and disabling JFR LeakProfiler
- fieldDescriptor.hpp added is_stable() accessor, for use in
Shenandoah C2 optimizations
- Locks initialization in mutexLocker.cpp as usual
- VM operations defines for Shenandoah's VM ops
- globalDefinitions.hpp added UINT64_FORMAT_HEX_W for use in
Shenandoah's logging
- The usual macros in macro.hpp
*) shared-build
<http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shared-build/>
- Add shenandoah feature, enabled by default, as agreed with
Vladimir K. beforehand
- Some flags for shenandoah-enabled compilation to get
SUPPORT_BARRIER_ON_PRIMITIVES
and SUPPORT_NOT_TO_SPACE_INVARIANT which is required for
Shenandoah's barriers
- --param inline-unit-growth=1000 settings for 2 shenandoah source
files, which is
useful to get the whole marking loop inlined (observed
significant
regression if we
don't)
*) shared-tests
<http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shared-tests/>
- Test infrastructure to support Shenandoah
- Shenandoah test groups
- Exclude Shenandoah in various tests that can be run with
selected GC
- Enable/add configure for Shenandoah for tests that make sense to
run with it
*) shenandoah-tests
<http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shenandoah-tests/>
- Shenandoah specific tests, most reside in gc/shenandoah
subdirectory
- A couple of tests configurations have been added, e.g.
TestGCBasherWithShenandoah.java
I intentionally left out shared-compiler for now, because we have some
work left to do
there, but if you click around you'll find the patch anyway, in case you
want to take
a peek at it.
- {Linux} x {x86_64, x86_32, armhf, aarch64, ppc64el, s390x}
- {Windows} x {x86_64},
- {MacOS X} x {x86_64}
- the new Shenandoah tests
- jcstress with/without aggressive Shenandoah verification
- specjvm2008 with/without aggressive Shenandoah verification
I'd like to thank my collegues at Red Hat: Christine Flood, she deserves
the credit for being the original inventor of Shenandoah, Aleksey
Shiplëv, Roland Westrelin & Zhengyu Gu for their countless
contributions, everybody else in Red Hat's OpenJDK team for testing,
advice and support, my collegues in Oracle's GC, runtime and compiler
teams for tirelessly helping with and reviewing all the GC interface and
related changes, and of course the many early adopters for reporting
bugs and success stories and feature requests: we wouldn't be here
without any of you!
Best regards,
Roman

[Roman Kennke]

Thanks, Magnus,

this makes sense. I'll incorporate that simplification.

Thanks,
Roman

Post by Magnus Ihse Bursie

Post by Erik Joelsson
Build changes look ok to me.

I agree, but I'd like to point out a (possible) style improvement. In
hotspot.m4,
if test "x$OPENJDK_TARGET_CPU" = "xx86_64" || test
"x$OPENJDK_TARGET_CPU" = "xaarch64" || test "x$OPENJDK_TARGET_CPU" ==
"xx86"; then
can be more succinctly expressed as
if test "x$OPENJDK_TARGET_CPU_ARCH" = "xx86" || test
"x$OPENJDK_TARGET_CPU" = "xaarch64" ; then
But it's a matter of preference, if you do not want to change. Otherwise
it looks good!
/Magnus

Post by Erik Joelsson
/Erik

Post by Roman Kennke
Hi,
This is the first round of changes for including Shenandoah GC into
mainline.
I divided the review into parts that roughly correspond to the mailing lists
that would normally review it, and I divided it into 'shared' code
changes and
'shenandoah' code changes (actually, mostly additions). The intend is to
eventually
push them as single 'combined' changeset, once reviewed.
   https://openjdk.java.net/jeps/189
  https://bugs.openjdk.java.net/browse/JDK-8214259
   http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/
For those who want to see the full change, have a look at the
shenandoah-complete
<http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shenandoah-complete/>
directory,
it contains the full combined webrev. Alternatively, there is the file
shenandoah-master.patch
<http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shenandoah-master.patch>,
which is what I intend to commit (and which should be equivalent to the
'shenandoah-complete' webrev).
  *) shenandoah-gc
<http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shenandoah-gc/>
     - Actual Shenandoah implementation, almost completely residing in
gc/shenandoah
  *) shared-gc
<http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shared-gc/>
     - This is mostly boilerplate that is common to any GC
     - referenceProcessor.cpp has a little change to make one assert not
fail (next to CMS and G1)
     - taskqueue.hpp has some small adjustments to enable subclassing
  *) shared-serviceability
<http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shared-serviceability/>
     - The usual code to support another GC
  *) shared-runtime
<http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shared-runtime/>
     - A number of friends declarations to allow Shenandoah iterators to
hook up with,
       e.g. ClassLoaderData, CodeCache, etc
     - Warning and disabling JFR LeakProfiler
     - fieldDescriptor.hpp added is_stable() accessor, for use in
Shenandoah C2 optimizations
     - Locks initialization in mutexLocker.cpp as usual
     - VM operations defines for Shenandoah's VM ops
     - globalDefinitions.hpp added UINT64_FORMAT_HEX_W for use in
Shenandoah's logging
     - The usual macros in macro.hpp
  *) shared-build
<http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shared-build/>
     - Add shenandoah feature, enabled by default, as agreed with
Vladimir K. beforehand
     - Some flags for shenandoah-enabled compilation to get
SUPPORT_BARRIER_ON_PRIMITIVES
       and SUPPORT_NOT_TO_SPACE_INVARIANT which is required for
Shenandoah's barriers
     - --param inline-unit-growth=1000 settings for 2 shenandoah source
files, which is
       useful to get the whole marking loop inlined (observed
significant
regression if we
       don't)
  *) shared-tests
<http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shared-tests/>
     - Test infrastructure to support Shenandoah
     - Shenandoah test groups
     - Exclude Shenandoah in various tests that can be run with
selected GC
     - Enable/add configure for Shenandoah for tests that make sense to
run with it
  *) shenandoah-tests
<http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shenandoah-tests/>
     - Shenandoah specific tests, most reside in gc/shenandoah
subdirectory
     - A couple of tests configurations have been added, e.g.
TestGCBasherWithShenandoah.java
I intentionally left out shared-compiler for now, because we have some
work left to do
there, but if you click around you'll find the patch anyway, in case you
want to take
a peek at it.
   - {Linux} x {x86_64, x86_32, armhf, aarch64, ppc64el, s390x}
   - {Windows} x {x86_64},
   - {MacOS X} x {x86_64}
   - the new Shenandoah tests
   - jcstress with/without aggressive Shenandoah verification
   - specjvm2008 with/without aggressive Shenandoah verification
I'd like to thank my collegues at Red Hat: Christine Flood, she deserves
the credit for being the original inventor of Shenandoah, Aleksey
Shiplëv, Roland Westrelin & Zhengyu Gu for their countless
contributions, everybody else in Red Hat's OpenJDK team for testing,
advice and support, my collegues in Oracle's GC, runtime and compiler
teams for tirelessly helping with and reviewing all the GC interface and
related changes, and of course the many early adopters for reporting
bugs and success stories and feature requests: we wouldn't be here
without any of you!
Best regards,
Roman

[Per Liden]

Small comment on the build changes. Instead of checking for zero variant
here:

+ # Only enable Shenandoah on supported arches
+ AC_MSG_CHECKING([if shenandoah can be built])
+ if HOTSPOT_CHECK_JVM_VARIANT(zero); then
+ DISABLED_JVM_FEATURES="$DISABLED_JVM_FEATURES shenandoahgc"
+ AC_MSG_RESULT([no, this JVM variant not supported])
+ else

I think you should just add shanandoahgc to the existing check, here:

# Disable unsupported GCs for Zero
if HOTSPOT_CHECK_JVM_VARIANT(zero); then
DISABLED_JVM_FEATURES="$DISABLED_JVM_FEATURES epsilongc g1gc zgc"
fi

cheers,
Per

Post by Erik Joelsson
Build changes look ok to me.
/Erik

Post by Roman Kennke
Hi,
This is the first round of changes for including Shenandoah GC into
mainline.
I divided the review into parts that roughly correspond to the mailing lists
that would normally review it, and I divided it into 'shared' code
changes and
'shenandoah' code changes (actually, mostly additions). The intend is to
eventually
push them as single 'combined' changeset, once reviewed.
   https://openjdk.java.net/jeps/189
  https://bugs.openjdk.java.net/browse/JDK-8214259
   http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/
For those who want to see the full change, have a look at the
shenandoah-complete
<http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shenandoah-complete/>
directory,
it contains the full combined webrev. Alternatively, there is the file
shenandoah-master.patch
<http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shenandoah-master.patch>,
which is what I intend to commit (and which should be equivalent to the
'shenandoah-complete' webrev).
  *) shenandoah-gc
<http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shenandoah-gc/>
     - Actual Shenandoah implementation, almost completely residing in
gc/shenandoah
  *) shared-gc
<http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shared-gc/>
     - This is mostly boilerplate that is common to any GC
     - referenceProcessor.cpp has a little change to make one assert not
fail (next to CMS and G1)
     - taskqueue.hpp has some small adjustments to enable subclassing
  *) shared-serviceability
<http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shared-serviceability/>
     - The usual code to support another GC
  *) shared-runtime
<http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shared-runtime/>
     - A number of friends declarations to allow Shenandoah iterators to
hook up with,
       e.g. ClassLoaderData, CodeCache, etc
     - Warning and disabling JFR LeakProfiler
     - fieldDescriptor.hpp added is_stable() accessor, for use in
Shenandoah C2 optimizations
     - Locks initialization in mutexLocker.cpp as usual
     - VM operations defines for Shenandoah's VM ops
     - globalDefinitions.hpp added UINT64_FORMAT_HEX_W for use in
Shenandoah's logging
     - The usual macros in macro.hpp
  *) shared-build
<http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shared-build/>
     - Add shenandoah feature, enabled by default, as agreed with
Vladimir K. beforehand
     - Some flags for shenandoah-enabled compilation to get
SUPPORT_BARRIER_ON_PRIMITIVES
       and SUPPORT_NOT_TO_SPACE_INVARIANT which is required for
Shenandoah's barriers
     - --param inline-unit-growth=1000 settings for 2 shenandoah source
files, which is
       useful to get the whole marking loop inlined (observed significant
regression if we
       don't)
  *) shared-tests
<http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shared-tests/>
     - Test infrastructure to support Shenandoah
     - Shenandoah test groups
     - Exclude Shenandoah in various tests that can be run with
selected GC
     - Enable/add configure for Shenandoah for tests that make sense to
run with it
  *) shenandoah-tests
<http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shenandoah-tests/>
     - Shenandoah specific tests, most reside in gc/shenandoah
subdirectory
     - A couple of tests configurations have been added, e.g.
TestGCBasherWithShenandoah.java
I intentionally left out shared-compiler for now, because we have some
work left to do
there, but if you click around you'll find the patch anyway, in case you
want to take
a peek at it.
   - {Linux} x {x86_64, x86_32, armhf, aarch64, ppc64el, s390x}
   - {Windows} x {x86_64},
   - {MacOS X} x {x86_64}
   - the new Shenandoah tests
   - jcstress with/without aggressive Shenandoah verification
   - specjvm2008 with/without aggressive Shenandoah verification
I'd like to thank my collegues at Red Hat: Christine Flood, she deserves
the credit for being the original inventor of Shenandoah, Aleksey
Shiplëv, Roland Westrelin & Zhengyu Gu for their countless
contributions, everybody else in Red Hat's OpenJDK team for testing,
advice and support, my collegues in Oracle's GC, runtime and compiler
teams for tirelessly helping with and reviewing all the GC interface and
related changes, and of course the many early adopters for reporting
bugs and success stories and feature requests: we wouldn't be here
without any of you!
Best regards,
Roman

[Roman Kennke]

Thanks, Per!

I will do this. Check out round 3 when it arrives. ;-)

Roman

Post by Per Liden
Small comment on the build changes. Instead of checking for zero variant
+  # Only enable Shenandoah on supported arches
+  AC_MSG_CHECKING([if shenandoah can be built])
+  if HOTSPOT_CHECK_JVM_VARIANT(zero); then
+    DISABLED_JVM_FEATURES="$DISABLED_JVM_FEATURES shenandoahgc"
+    AC_MSG_RESULT([no, this JVM variant not supported])
+  else
  # Disable unsupported GCs for Zero
  if HOTSPOT_CHECK_JVM_VARIANT(zero); then
    DISABLED_JVM_FEATURES="$DISABLED_JVM_FEATURES epsilongc g1gc zgc"
  fi
cheers,
Per

Post by Erik Joelsson
Build changes look ok to me.
/Erik

Post by Roman Kennke
Hi,
This is the first round of changes for including Shenandoah GC into
mainline.
I divided the review into parts that roughly correspond to the mailing lists
that would normally review it, and I divided it into 'shared' code
changes and
'shenandoah' code changes (actually, mostly additions). The intend is to
eventually
push them as single 'combined' changeset, once reviewed.
   https://openjdk.java.net/jeps/189
  https://bugs.openjdk.java.net/browse/JDK-8214259
   http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/
For those who want to see the full change, have a look at the
shenandoah-complete
<http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shenandoah-complete/>
directory,
it contains the full combined webrev. Alternatively, there is the file
shenandoah-master.patch
<http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shenandoah-master.patch>,
which is what I intend to commit (and which should be equivalent to the
'shenandoah-complete' webrev).
  *) shenandoah-gc
<http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shenandoah-gc/>
     - Actual Shenandoah implementation, almost completely residing in
gc/shenandoah
  *) shared-gc
<http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shared-gc/>
     - This is mostly boilerplate that is common to any GC
     - referenceProcessor.cpp has a little change to make one assert not
fail (next to CMS and G1)
     - taskqueue.hpp has some small adjustments to enable subclassing
  *) shared-serviceability
<http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shared-serviceability/>
     - The usual code to support another GC
  *) shared-runtime
<http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shared-runtime/>
     - A number of friends declarations to allow Shenandoah iterators to
hook up with,
       e.g. ClassLoaderData, CodeCache, etc
     - Warning and disabling JFR LeakProfiler
     - fieldDescriptor.hpp added is_stable() accessor, for use in
Shenandoah C2 optimizations
     - Locks initialization in mutexLocker.cpp as usual
     - VM operations defines for Shenandoah's VM ops
     - globalDefinitions.hpp added UINT64_FORMAT_HEX_W for use in
Shenandoah's logging
     - The usual macros in macro.hpp
  *) shared-build
<http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shared-build/>
     - Add shenandoah feature, enabled by default, as agreed with
Vladimir K. beforehand
     - Some flags for shenandoah-enabled compilation to get
SUPPORT_BARRIER_ON_PRIMITIVES
       and SUPPORT_NOT_TO_SPACE_INVARIANT which is required for
Shenandoah's barriers
     - --param inline-unit-growth=1000 settings for 2 shenandoah source
files, which is
       useful to get the whole marking loop inlined (observed
significant
regression if we
       don't)
  *) shared-tests
<http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shared-tests/>
     - Test infrastructure to support Shenandoah
     - Shenandoah test groups
     - Exclude Shenandoah in various tests that can be run with
selected GC
     - Enable/add configure for Shenandoah for tests that make sense to
run with it
  *) shenandoah-tests
<http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shenandoah-tests/>
     - Shenandoah specific tests, most reside in gc/shenandoah
subdirectory
     - A couple of tests configurations have been added, e.g.
TestGCBasherWithShenandoah.java
I intentionally left out shared-compiler for now, because we have some
work left to do
there, but if you click around you'll find the patch anyway, in case you
want to take
a peek at it.
   - {Linux} x {x86_64, x86_32, armhf, aarch64, ppc64el, s390x}
   - {Windows} x {x86_64},
   - {MacOS X} x {x86_64}
   - the new Shenandoah tests
   - jcstress with/without aggressive Shenandoah verification
   - specjvm2008 with/without aggressive Shenandoah verification
I'd like to thank my collegues at Red Hat: Christine Flood, she deserves
the credit for being the original inventor of Shenandoah, Aleksey
Shiplëv, Roland Westrelin & Zhengyu Gu for their countless
contributions, everybody else in Red Hat's OpenJDK team for testing,
advice and support, my collegues in Oracle's GC, runtime and compiler
teams for tirelessly helping with and reviewing all the GC interface and
related changes, and of course the many early adopters for reporting
bugs and success stories and feature requests: we wouldn't be here
without any of you!
Best regards,
Roman

[Kim Barrett]

A few quick comments (not a real review).

Post by Roman Kennke
*) shared-gc
- This is mostly boilerplate that is common to any GC
- referenceProcessor.cpp has a little change to make one assert not fail (next to CMS and G1)

Ick. But I don’t have a better suggestion that doesn’t involve adding a new API
to CollectedHeap for use by this assertion, which seems a bit excessive if there
are no other uses.

Post by Roman Kennke
- taskqueue.hpp has some small adjustments to enable subclassing

Why this change instead of JDK-8204947? As the description from that RFE says:
"The ShenandoahTaskTerminator from the Shenandoah project is a much better implementation of a task terminator.”

[Zhengyu Gu]

Hi Kim,

Post by Kim Barrett

Post by Roman Kennke
- taskqueue.hpp has some small adjustments to enable subclassing

"The ShenandoahTaskTerminator from the Shenandoah project is a much better implementation of a task terminator.”

We were busy on preparing Shenandoah for upstreaming, this did take a
back seat.

I am going to start to work on moving ShenandoahTaskTerminator to share
tomorrow, but it should not hold up the review.

Thanks,

-Zhengyu

[Roman Kennke]

Hi Kim,

Post by Roman Kennke
*) shared-gc
- This is mostly boilerplate that is common to any GC
- referenceProcessor.cpp has a little change to make one assert not fail (next to CMS and G1)

Ick. But I don’t have a better suggestion that doesn’t involve adding a new API
to CollectedHeap for use by this assertion, which seems a bit excessive if there
are no other uses.

Yeah.
I guess we could add a config _discovery_is_concurrent or similar in RP,
and check that. Or maybe one of _discovery_is_mt or _discovery_is_atomic
already covers that? I couldn't immediately tell/100% understand their
semantics. Seems worthy to look at after this?

Post by Roman Kennke
- taskqueue.hpp has some small adjustments to enable subclassing

"The ShenandoahTaskTerminator from the Shenandoah project is a much better implementation of a task terminator.”

Yeah, see Zhengyu's comment. Let's ignore those changes for this review
(for now), expect our impl ported to taskqueue.hpp/cpp soon.

Roman

[Kim Barrett]

Post by Zhengyu Gu
Hi Kim,

Post by Kim Barrett

Post by Roman Kennke
*) shared-gc
- This is mostly boilerplate that is common to any GC
- referenceProcessor.cpp has a little change to make one assert not fail (next to CMS and G1)

Ick. But I don’t have a better suggestion that doesn’t involve adding a new API
to CollectedHeap for use by this assertion, which seems a bit excessive if there
are no other uses.

Yeah.
I guess we could add a config _discovery_is_concurrent or similar in RP,
and check that. Or maybe one of _discovery_is_mt or _discovery_is_atomic
already covers that? I couldn't immediately tell/100% understand their
semantics. Seems worthy to look at after this?

It might be equivalent to _discovery_is_atomic; I don’t remember the exact
semantics right now. I think it’s unrelated to _discovery_is_mt.

Yes, looking at this later is fine. Please file an RFE.

Post by Zhengyu Gu

Post by Kim Barrett

Post by Roman Kennke
- taskqueue.hpp has some small adjustments to enable subclassing

"The ShenandoahTaskTerminator from the Shenandoah project is a much better implementation of a task terminator.”

Yeah, see Zhengyu's comment. Let's ignore those changes for this review
(for now), expect our impl ported to taskqueue.hpp/cpp soon.

I’m okay with that plan. Maybe add a comment in JDK-8204947 about this?

[Roman Kennke]

Hi Kim,

Post by Zhengyu Gu
Hi Kim,

Post by Roman Kennke
*) shared-gc
- This is mostly boilerplate that is common to any GC
- referenceProcessor.cpp has a little change to make one assert not fail (next to CMS and G1)

Ick. But I don’t have a better suggestion that doesn’t involve adding a new API
to CollectedHeap for use by this assertion, which seems a bit excessive if there
are no other uses.

Yeah.
I guess we could add a config _discovery_is_concurrent or similar in RP,
and check that. Or maybe one of _discovery_is_mt or _discovery_is_atomic
already covers that? I couldn't immediately tell/100% understand their
semantics. Seems worthy to look at after this?

It might be equivalent to _discovery_is_atomic; I don’t remember the exact
semantics right now. I think it’s unrelated to _discovery_is_mt.
Yes, looking at this later is fine. Please file an RFE.

Ok, done:
https://bugs.openjdk.java.net/browse/JDK-8214441

Post by Zhengyu Gu

Post by Roman Kennke
- taskqueue.hpp has some small adjustments to enable subclassing

"The ShenandoahTaskTerminator from the Shenandoah project is a much better implementation of a task terminator.”

Yeah, see Zhengyu's comment. Let's ignore those changes for this review
(for now), expect our impl ported to taskqueue.hpp/cpp soon.

I’m okay with that plan. Maybe add a comment in JDK-8204947 about this?

https://bugs.openjdk.java.net/browse/JDK-8204947?focusedCommentId=14226307&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-14226307

Thanks for reviewing,
Roman

[l***@oracle.com]

Hi

I looked at the tests changes only. It seems that  a some tests might
start failing if JDK built without Shenandoah GC. Also some test are not
going to be selected as expected.

Unfortunately logic of '@requires' and @run in jtreg tests doesn't
always work well for specific cases of different GC selection. The
'@requires' tags are combined with '&' and whole test is selected or
not. Test always execute ALL @run actions so it fails if option
-XX:+UseShenandoahGC is not supported (not valid). The only way to split
'run' actions is to add more @test with same sources. They could be in
the same file.

See detailed info about jtreg tags
here:http://openjdk.java.net/jtreg/tag-spec.html
<http://openjdk.java.net/jtreg/tag-spec.html>

Could you please run your tests with with JDK which built without
Shenandoah GC . It helps to verify that Shenandoah-specific tests/runs
are not selected when this GC is not supported.   Also it would be nice
to verify that there are no any valid tests which became filtered out
with your patch. The running of all test suite with available but not
selected Shenandoah GC and enabled Graal also might help to verify
@requires settings. (It doesn't help when Shenandoah GCis not supported
though.)

I haven't looked at the tests in directory gc/shenandoah in details. But
all of them should be guarded with @requires. Placing them in separate
directory is not enough. See G1 tests as example:

http://hg.openjdk.java.net/jdk/jdk/file/10c6e9066819/test/hotspot/jtreg/gc/g1/TestEagerReclaimHumongousRegions.java
<http://hg.openjdk.java.net/jdk/jdk/file/10c6e9066819/test/hotspot/jtreg/gc/g1/TestStringDeduplicationFullGC.java#l29>


See more detailed comments about shared tests:

http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shared-tests/test/hotspot/jtreg/TEST.groups.sdiff.html

219: tier2_gc_shenandoah = \

Usually tierN doesn't include tests from tierN-1. TierN is executed
after TierN-1 completed so no need to re-run the same tests.  The
typical groups might looks like:

tier1_gc_shenandoah = \
 gc/shenandoah/<tier1-tests> \
 other-tests

tier2_gc_shenandoah = \
   gc/shenandoah/<tier2-tests>\
  -:tier1_gc_shenandoah

tier3_gc_shenandoah = \
   gc/shenandoah/ \  //all-other-tests
  -:tier2_gc_shenandoah


http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shared-tests/test/hotspot/jtreg/gc/CriticalNativeArgs.java.html

http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shared-tests/test/hotspot/jtreg/gc/stress/CriticalNativeStress.java.html

So your test will be skipped if any of -XX:+UseEpsilonGC or
-XX:+UseShenandoahGC is set. Also test might run only all run actions or
none of them. It would be better to split this test into 2 tests. So
epsilon tests might be executed if Shenandoah is absent.

I think that (vm.bits == "64") is redundant in tag (you set x64 or arm64).

Please use 4-space indentation  in java code.


http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shared-tests/test/hotspot/jtreg/gc/TestFullGCCount.java.sdiff.html

Even original requires seems confusing to me (but it works for CMS/G1 pair)

28  * @requires !(vm.gc.ConcMarkSweep &
vm.opt.ExplicitGCInvokesConcurrent == true)

So currently test is executed if GC is CMS or default GC and
ExplicitGCInvokesConcurrent is not set to true.

With your additional requirements 'vm.gc == "Shenandoah"' test is not
selected if ANY GC is set. Test doesn't set any GC itself so only
default GC might be tested.  See [1].


http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shared-tests/test/hotspot/jtreg/gc/TestHumongousReferenceObject.java.sdiff.html

http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shared-tests/test/hotspot/jtreg/gc/TestSystemGC.java.sdiff.html

Tests will always just fail if -XX:+UseShenandoahGC  is not supported.
(invalid option) You might want to split test is it done for CMS GC.


http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shared-tests/test/hotspot/jtreg/gc/arguments/TestAlignmentToUseLargePages.java.sdiff.html

I think
56  * @requires vm.gc=="null" & !vm.graal.enabled
should be something like @requires vm.gc.Shenandoah & !vm.graal.enabled


http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shared-tests/test/hotspot/jtreg/gc/arguments/TestUseCompressedOopsErgo.java.sdiff.html

http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shared-tests/test/hotspot/jtreg/gc/class_unloading/TestClassUnloadingDisabled.java.sdiff.html

The same for  62  * @requires vm.gc=="null" & !vm.graal.enabled
and
72  * @requires vm.gc=="null" & !vm.graal.enabled


http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shared-tests/test/hotspot/jtreg/gc/ergonomics/TestDynamicNumberOfGCThreads.java.sdiff.html

http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shared-tests/test/hotspot/jtreg/gc/logging/TestGCId.java.sdiff.html

http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shared-tests/test/hotspot/jtreg/gc/metaspace/TestMetaspacePerfCounters.java.sdiff.html

http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shared-tests/test/hotspot/jtreg/runtime/CompressedOops/UseCompressedOops.java.sdiff.html

http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shared-tests/test/hotspot/jtreg/runtime/MemberName/MemberNameLeak.java.sdiff.html

http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shared-tests/test/hotspot/jtreg/gc/metaspace/TestMetaspacePerfCounters.java.sdiff.html
Also these tests are going to be run with all GC and fails if Shenandoah
is not supported.

Leonid

[1] http://openjdk.java.net/jtreg/tag-spec.html

Post by Roman Kennke
Hi,
This is the first round of changes for including Shenandoah GC into
mainline.
I divided the review into parts that roughly correspond to the mailing lists
that would normally review it, and I divided it into 'shared' code
changes and
'shenandoah' code changes (actually, mostly additions). The intend is
to eventually
push them as single 'combined' changeset, once reviewed.
https://openjdk.java.net/jeps/189
https://bugs.openjdk.java.net/browse/JDK-8214259
http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/
For those who want to see the full change, have a look at the
shenandoah-complete
<http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shenandoah-complete/>
directory,
it contains the full combined webrev. Alternatively, there is the file
shenandoah-master.patch
<http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shenandoah-master.patch>,
which is what I intend to commit (and which should be equivalent to
the 'shenandoah-complete' webrev).
 *) shenandoah-gc
<http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shenandoah-gc/>
    - Actual Shenandoah implementation, almost completely residing in
gc/shenandoah
 *) shared-gc
<http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shared-gc/>
    - This is mostly boilerplate that is common to any GC
    - referenceProcessor.cpp has a little change to make one assert
not fail (next to CMS and G1)
    - taskqueue.hpp has some small adjustments to enable subclassing
 *) shared-serviceability
<http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shared-serviceability/>
    - The usual code to support another GC
 *) shared-runtime
<http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shared-runtime/>
    - A number of friends declarations to allow Shenandoah iterators
to hook up with,
      e.g. ClassLoaderData, CodeCache, etc
    - Warning and disabling JFR LeakProfiler
    - fieldDescriptor.hpp added is_stable() accessor, for use in
Shenandoah C2 optimizations
    - Locks initialization in mutexLocker.cpp as usual
    - VM operations defines for Shenandoah's VM ops
    - globalDefinitions.hpp added UINT64_FORMAT_HEX_W for use in
Shenandoah's logging
    - The usual macros in macro.hpp
 *) shared-build
<http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shared-build/>
    - Add shenandoah feature, enabled by default, as agreed with
Vladimir K. beforehand
    - Some flags for shenandoah-enabled compilation to get
SUPPORT_BARRIER_ON_PRIMITIVES
      and SUPPORT_NOT_TO_SPACE_INVARIANT which is required for
Shenandoah's barriers
    - --param inline-unit-growth=1000 settings for 2 shenandoah source
files, which is
      useful to get the whole marking loop inlined (observed
significant regression if we
      don't)
 *) shared-tests
<http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shared-tests/>
    - Test infrastructure to support Shenandoah
    - Shenandoah test groups
    - Exclude Shenandoah in various tests that can be run with selected GC
    - Enable/add configure for Shenandoah for tests that make sense to
run with it
 *) shenandoah-tests
<http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shenandoah-tests/>
    - Shenandoah specific tests, most reside in gc/shenandoah subdirectory
    - A couple of tests configurations have been added, e.g.
TestGCBasherWithShenandoah.java
I intentionally left out shared-compiler for now, because we have some
work left to do
there, but if you click around you'll find the patch anyway, in case
you want to take
a peek at it.
  - {Linux} x {x86_64, x86_32, armhf, aarch64, ppc64el, s390x}
  - {Windows} x {x86_64},
  - {MacOS X} x {x86_64}
  - the new Shenandoah tests
  - jcstress with/without aggressive Shenandoah verification
  - specjvm2008 with/without aggressive Shenandoah verification
I'd like to thank my collegues at Red Hat: Christine Flood, she
deserves the credit for being the original inventor of Shenandoah,
Aleksey Shiplëv, Roland Westrelin & Zhengyu Gu for their countless
contributions, everybody else in Red Hat's OpenJDK team for testing,
advice and support, my collegues in Oracle's GC, runtime and compiler
teams for tirelessly helping with and reviewing all the GC interface
and related changes, and of course the many early adopters for
reporting bugs and success stories and feature requests: we wouldn't
be here without any of you!
Best regards,
Roman

[Roman Kennke]

Hi Leonid,

Post by l***@oracle.com
I looked at the tests changes only. It seems that  a some tests might
start failing if JDK built without Shenandoah GC. Also some test are not
going to be selected as expected.
always work well for specific cases of different GC selection. The
-XX:+UseShenandoahGC is not supported (not valid). The only way to split
the same file.
See detailed info about jtreg tags
here:http://openjdk.java.net/jtreg/tag-spec.html
<http://openjdk.java.net/jtreg/tag-spec.html>

Thanks for pointing this out.

I fixed all of what you pointed out (I think) and some more:
http://cr.openjdk.java.net/~rkennke/fix-shared-tests/webrev.02/

It will show up in round2 of this review.

Post by l***@oracle.com
Could you please run your tests with with JDK which built without
Shenandoah GC. It helps to verify that Shenandoah-specific tests/runs
are not selected when this GC is not supported.

I did now, and they are good (with above changes).

Post by l***@oracle.com
  Also it would be nice
to verify that there are no any valid tests which became filtered out
with your patch. The running of all test suite with available but not
selected Shenandoah GC and enabled Graal also might help to verify
@requires settings. (It doesn't help when Shenandoah GCis not supported
though.) 

I'll see into running more combinations. So far I did hotspot_gc and a
few others with and without Shenandoah.

Post by l***@oracle.com
I haven't looked at the tests in directory gc/shenandoah in details. But
http://hg.openjdk.java.net/jdk/jdk/file/10c6e9066819/test/hotspot/jtreg/gc/g1/TestEagerReclaimHumongousRegions.java
<http://hg.openjdk.java.net/jdk/jdk/file/10c6e9066819/test/hotspot/jtreg/gc/g1/TestStringDeduplicationFullGC.java#l29>

Right. We've done that now.

Post by l***@oracle.com
http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shared-tests/test/hotspot/jtreg/TEST.groups.sdiff.html
219: tier2_gc_shenandoah = \
Usually tierN doesn't include tests from tierN-1. TierN is executed
after TierN-1 completed so no need to re-run the same tests.  The
tier1_gc_shenandoah = \
 gc/shenandoah/<tier1-tests> \
 other-tests
tier2_gc_shenandoah = \
   gc/shenandoah/<tier2-tests>\
  -:tier1_gc_shenandoah
tier3_gc_shenandoah = \
   gc/shenandoah/ \  //all-other-tests
  -:tier2_gc_shenandoah

We fixed that.

Post by l***@oracle.com
http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shared-tests/test/hotspot/jtreg/gc/CriticalNativeArgs.java.html
http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shared-tests/test/hotspot/jtreg/gc/stress/CriticalNativeStress.java.html
So your test will be skipped if any of -XX:+UseEpsilonGC or
-XX:+UseShenandoahGC is set. Also test might run only all run actions or
none of them. It would be better to split this test into 2 tests. So
epsilon tests might be executed if Shenandoah is absent.
I think that (vm.bits == "64") is redundant in tag (you set x64 or arm64).
Please use 4-space indentation  in java code.

All fixed.

Post by l***@oracle.com
http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shared-tests/test/hotspot/jtreg/gc/TestFullGCCount.java.sdiff.html
Even original requires seems confusing to me (but it works for CMS/G1 pair)
vm.opt.ExplicitGCInvokesConcurrent == true) 
So currently test is executed if GC is CMS or default GC and
ExplicitGCInvokesConcurrent is not set to true.
With your additional requirements 'vm.gc == "Shenandoah"' test is not
selected if ANY GC is set. Test doesn't set any GC itself so only
default GC might be tested.  See [1].

I split them and fixed the @requires to run with Shenandoah but only
with -ExplicitGCInvokesConcurrent.

Post by l***@oracle.com
http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shared-tests/test/hotspot/jtreg/gc/TestHumongousReferenceObject.java.sdiff.html
http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shared-tests/test/hotspot/jtreg/gc/TestSystemGC.java.sdiff.html
Tests will always just fail if -XX:+UseShenandoahGC  is not supported.
(invalid option) You might want to split test is it done for CMS GC.

Done.

Post by l***@oracle.com
http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shared-tests/test/hotspot/jtreg/gc/arguments/TestAlignmentToUseLargePages.java.sdiff.html
I think 

Yes. Fixed them.

Post by l***@oracle.com
http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shared-tests/test/hotspot/jtreg/gc/arguments/TestUseCompressedOopsErgo.java.sdiff.html
http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shared-tests/test/hotspot/jtreg/gc/class_unloading/TestClassUnloadingDisabled.java.sdiff.html
and

Fixed them too.

Post by l***@oracle.com
http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shared-tests/test/hotspot/jtreg/gc/ergonomics/TestDynamicNumberOfGCThreads.java.sdiff.html
http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shared-tests/test/hotspot/jtreg/gc/logging/TestGCId.java.sdiff.html
http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shared-tests/test/hotspot/jtreg/gc/metaspace/TestMetaspacePerfCounters.java.sdiff.html
http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shared-tests/test/hotspot/jtreg/runtime/CompressedOops/UseCompressedOops.java.sdiff.html
http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shared-tests/test/hotspot/jtreg/runtime/MemberName/MemberNameLeak.java.sdiff.html
http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shared-tests/test/hotspot/jtreg/gc/metaspace/TestMetaspacePerfCounters.java.sdiff.html
Also these tests are going to be run with all GC and fails if Shenandoah
is not supported.

Right. I fixed those and a few others I have found. Those which drive
Shenandoah at runtime now have a runtime check
(GC.Shenandoah.isSupported() which uses WB). Others have split test
sections.

I'll upload round 2 of review changesets, which contains the fixes in a bit.

Thanks a *LOT* for detailed review.

Roman

Post by l***@oracle.com
Leonid
[1] http://openjdk.java.net/jtreg/tag-spec.html

Post by Roman Kennke
Hi,
This is the first round of changes for including Shenandoah GC into
mainline.
I divided the review into parts that roughly correspond to the mailing lists
that would normally review it, and I divided it into 'shared' code
changes and
'shenandoah' code changes (actually, mostly additions). The intend is
to eventually
push them as single 'combined' changeset, once reviewed.
  https://openjdk.java.net/jeps/189
 https://bugs.openjdk.java.net/browse/JDK-8214259
  http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/
For those who want to see the full change, have a look at the
shenandoah-complete
<http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shenandoah-complete/>
directory,
it contains the full combined webrev. Alternatively, there is the file
shenandoah-master.patch
<http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shenandoah-master.patch>,
which is what I intend to commit (and which should be equivalent to
the 'shenandoah-complete' webrev).
 *) shenandoah-gc
<http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shenandoah-gc/>
    - Actual Shenandoah implementation, almost completely residing in
gc/shenandoah
 *) shared-gc
<http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shared-gc/>
    - This is mostly boilerplate that is common to any GC
    - referenceProcessor.cpp has a little change to make one assert
not fail (next to CMS and G1)
    - taskqueue.hpp has some small adjustments to enable subclassing
 *) shared-serviceability
<http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shared-serviceability/>
    - The usual code to support another GC
 *) shared-runtime
<http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shared-runtime/>
    - A number of friends declarations to allow Shenandoah iterators
to hook up with,
      e.g. ClassLoaderData, CodeCache, etc
    - Warning and disabling JFR LeakProfiler
    - fieldDescriptor.hpp added is_stable() accessor, for use in
Shenandoah C2 optimizations
    - Locks initialization in mutexLocker.cpp as usual
    - VM operations defines for Shenandoah's VM ops
    - globalDefinitions.hpp added UINT64_FORMAT_HEX_W for use in
Shenandoah's logging
    - The usual macros in macro.hpp
 *) shared-build
<http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shared-build/>
    - Add shenandoah feature, enabled by default, as agreed with
Vladimir K. beforehand
    - Some flags for shenandoah-enabled compilation to get
SUPPORT_BARRIER_ON_PRIMITIVES
      and SUPPORT_NOT_TO_SPACE_INVARIANT which is required for
Shenandoah's barriers
    - --param inline-unit-growth=1000 settings for 2 shenandoah source
files, which is
      useful to get the whole marking loop inlined (observed
significant regression if we
      don't)
 *) shared-tests
<http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shared-tests/>
    - Test infrastructure to support Shenandoah
    - Shenandoah test groups
    - Exclude Shenandoah in various tests that can be run with selected GC
    - Enable/add configure for Shenandoah for tests that make sense to
run with it
 *) shenandoah-tests
<http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shenandoah-tests/>
    - Shenandoah specific tests, most reside in gc/shenandoah subdirectory
    - A couple of tests configurations have been added, e.g.
TestGCBasherWithShenandoah.java
I intentionally left out shared-compiler for now, because we have some
work left to do
there, but if you click around you'll find the patch anyway, in case
you want to take
a peek at it.
  - {Linux} x {x86_64, x86_32, armhf, aarch64, ppc64el, s390x}
  - {Windows} x {x86_64},
  - {MacOS X} x {x86_64}
  - the new Shenandoah tests
  - jcstress with/without aggressive Shenandoah verification
  - specjvm2008 with/without aggressive Shenandoah verification
I'd like to thank my collegues at Red Hat: Christine Flood, she
deserves the credit for being the original inventor of Shenandoah,
Aleksey Shiplëv, Roland Westrelin & Zhengyu Gu for their countless
contributions, everybody else in Red Hat's OpenJDK team for testing,
advice and support, my collegues in Oracle's GC, runtime and compiler
teams for tirelessly helping with and reviewing all the GC interface
and related changes, and of course the many early adopters for
reporting bugs and success stories and feature requests: we wouldn't
be here without any of you!
Best regards,
Roman

[l***@oracle.com]

Hi

Thank you for your fixes and additional testing. I'll check new version 
in the round-2.

Leonid

Post by Roman Kennke
Hi Leonid,

I looked at the tests changes only. It seems that  a some tests might
start failing if JDK built without Shenandoah GC. Also some test are not
going to be selected as expected.
always work well for specific cases of different GC selection. The
-XX:+UseShenandoahGC is not supported (not valid). The only way to split
the same file.
See detailed info about jtreg tags
here:http://openjdk.java.net/jtreg/tag-spec.html
<http://openjdk.java.net/jtreg/tag-spec.html>

Thanks for pointing this out.
http://cr.openjdk.java.net/~rkennke/fix-shared-tests/webrev.02/
It will show up in round2 of this review.

Could you please run your tests with with JDK which built without
Shenandoah GC. It helps to verify that Shenandoah-specific tests/runs
are not selected when this GC is not supported.

I did now, and they are good (with above changes).

  Also it would be nice
to verify that there are no any valid tests which became filtered out
with your patch. The running of all test suite with available but not
selected Shenandoah GC and enabled Graal also might help to verify
@requires settings. (It doesn't help when Shenandoah GCis not supported
though.)

I'll see into running more combinations. So far I did hotspot_gc and a
few others with and without Shenandoah.

I haven't looked at the tests in directory gc/shenandoah in details. But
http://hg.openjdk.java.net/jdk/jdk/file/10c6e9066819/test/hotspot/jtreg/gc/g1/TestEagerReclaimHumongousRegions.java
<http://hg.openjdk.java.net/jdk/jdk/file/10c6e9066819/test/hotspot/jtreg/gc/g1/TestStringDeduplicationFullGC.java#l29>

Right. We've done that now.

http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shared-tests/test/hotspot/jtreg/TEST.groups.sdiff.html
219: tier2_gc_shenandoah = \
Usually tierN doesn't include tests from tierN-1. TierN is executed
after TierN-1 completed so no need to re-run the same tests.  The
tier1_gc_shenandoah = \
 gc/shenandoah/<tier1-tests> \
 other-tests
tier2_gc_shenandoah = \
   gc/shenandoah/<tier2-tests>\
  -:tier1_gc_shenandoah
tier3_gc_shenandoah = \
   gc/shenandoah/ \  //all-other-tests
  -:tier2_gc_shenandoah

We fixed that.

http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shared-tests/test/hotspot/jtreg/gc/CriticalNativeArgs.java.html
http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shared-tests/test/hotspot/jtreg/gc/stress/CriticalNativeStress.java.html
So your test will be skipped if any of -XX:+UseEpsilonGC or
-XX:+UseShenandoahGC is set. Also test might run only all run actions or
none of them. It would be better to split this test into 2 tests. So
epsilon tests might be executed if Shenandoah is absent.
I think that (vm.bits == "64") is redundant in tag (you set x64 or arm64).
Please use 4-space indentation  in java code.

All fixed.

http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shared-tests/test/hotspot/jtreg/gc/TestFullGCCount.java.sdiff.html
Even original requires seems confusing to me (but it works for CMS/G1 pair)
vm.opt.ExplicitGCInvokesConcurrent == true)
So currently test is executed if GC is CMS or default GC and
ExplicitGCInvokesConcurrent is not set to true.
With your additional requirements 'vm.gc == "Shenandoah"' test is not
selected if ANY GC is set. Test doesn't set any GC itself so only
default GC might be tested.  See [1].

with -ExplicitGCInvokesConcurrent.

http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shared-tests/test/hotspot/jtreg/gc/TestHumongousReferenceObject.java.sdiff.html
http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shared-tests/test/hotspot/jtreg/gc/TestSystemGC.java.sdiff.html
Tests will always just fail if -XX:+UseShenandoahGC  is not supported.
(invalid option) You might want to split test is it done for CMS GC.

Done.

http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shared-tests/test/hotspot/jtreg/gc/arguments/TestAlignmentToUseLargePages.java.sdiff.html
I think

Yes. Fixed them.

http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shared-tests/test/hotspot/jtreg/gc/arguments/TestUseCompressedOopsErgo.java.sdiff.html
http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shared-tests/test/hotspot/jtreg/gc/class_unloading/TestClassUnloadingDisabled.java.sdiff.html
and

Fixed them too.

http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shared-tests/test/hotspot/jtreg/gc/ergonomics/TestDynamicNumberOfGCThreads.java.sdiff.html
http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shared-tests/test/hotspot/jtreg/gc/logging/TestGCId.java.sdiff.html
http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shared-tests/test/hotspot/jtreg/gc/metaspace/TestMetaspacePerfCounters.java.sdiff.html
http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shared-tests/test/hotspot/jtreg/runtime/CompressedOops/UseCompressedOops.java.sdiff.html
http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shared-tests/test/hotspot/jtreg/runtime/MemberName/MemberNameLeak.java.sdiff.html
http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shared-tests/test/hotspot/jtreg/gc/metaspace/TestMetaspacePerfCounters.java.sdiff.html
Also these tests are going to be run with all GC and fails if Shenandoah
is not supported.

Right. I fixed those and a few others I have found. Those which drive
Shenandoah at runtime now have a runtime check
(GC.Shenandoah.isSupported() which uses WB). Others have split test
sections.
I'll upload round 2 of review changesets, which contains the fixes in a bit.
Thanks a *LOT* for detailed review.
Roman

Leonid
[1] http://openjdk.java.net/jtreg/tag-spec.html

Post by Roman Kennke
Hi,
This is the first round of changes for including Shenandoah GC into
mainline.
I divided the review into parts that roughly correspond to the mailing lists
that would normally review it, and I divided it into 'shared' code
changes and
'shenandoah' code changes (actually, mostly additions). The intend is
to eventually
push them as single 'combined' changeset, once reviewed.
  https://openjdk.java.net/jeps/189
 https://bugs.openjdk.java.net/browse/JDK-8214259
  http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/
For those who want to see the full change, have a look at the
shenandoah-complete
<http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shenandoah-complete/>
directory,
it contains the full combined webrev. Alternatively, there is the file
shenandoah-master.patch
<http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shenandoah-master.patch>,
which is what I intend to commit (and which should be equivalent to
the 'shenandoah-complete' webrev).
 *) shenandoah-gc
<http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shenandoah-gc/>
    - Actual Shenandoah implementation, almost completely residing in
gc/shenandoah
 *) shared-gc
<http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shared-gc/>
    - This is mostly boilerplate that is common to any GC
    - referenceProcessor.cpp has a little change to make one assert
not fail (next to CMS and G1)
    - taskqueue.hpp has some small adjustments to enable subclassing
 *) shared-serviceability
<http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shared-serviceability/>
    - The usual code to support another GC
 *) shared-runtime
<http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shared-runtime/>
    - A number of friends declarations to allow Shenandoah iterators
to hook up with,
      e.g. ClassLoaderData, CodeCache, etc
    - Warning and disabling JFR LeakProfiler
    - fieldDescriptor.hpp added is_stable() accessor, for use in
Shenandoah C2 optimizations
    - Locks initialization in mutexLocker.cpp as usual
    - VM operations defines for Shenandoah's VM ops
    - globalDefinitions.hpp added UINT64_FORMAT_HEX_W for use in
Shenandoah's logging
    - The usual macros in macro.hpp
 *) shared-build
<http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shared-build/>
    - Add shenandoah feature, enabled by default, as agreed with
Vladimir K. beforehand
    - Some flags for shenandoah-enabled compilation to get
SUPPORT_BARRIER_ON_PRIMITIVES
      and SUPPORT_NOT_TO_SPACE_INVARIANT which is required for
Shenandoah's barriers
    - --param inline-unit-growth=1000 settings for 2 shenandoah source
files, which is
      useful to get the whole marking loop inlined (observed
significant regression if we
      don't)
 *) shared-tests
<http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shared-tests/>
    - Test infrastructure to support Shenandoah
    - Shenandoah test groups
    - Exclude Shenandoah in various tests that can be run with selected GC
    - Enable/add configure for Shenandoah for tests that make sense to
run with it
 *) shenandoah-tests
<http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shenandoah-tests/>
    - Shenandoah specific tests, most reside in gc/shenandoah subdirectory
    - A couple of tests configurations have been added, e.g.
TestGCBasherWithShenandoah.java
I intentionally left out shared-compiler for now, because we have some
work left to do
there, but if you click around you'll find the patch anyway, in case
you want to take
a peek at it.
  - {Linux} x {x86_64, x86_32, armhf, aarch64, ppc64el, s390x}
  - {Windows} x {x86_64},
  - {MacOS X} x {x86_64}
  - the new Shenandoah tests
  - jcstress with/without aggressive Shenandoah verification
  - specjvm2008 with/without aggressive Shenandoah verification
I'd like to thank my collegues at Red Hat: Christine Flood, she
deserves the credit for being the original inventor of Shenandoah,
Aleksey Shiplëv, Roland Westrelin & Zhengyu Gu for their countless
contributions, everybody else in Red Hat's OpenJDK team for testing,
advice and support, my collegues in Oracle's GC, runtime and compiler
teams for tirelessly helping with and reviewing all the GC interface
and related changes, and of course the many early adopters for
reporting bugs and success stories and feature requests: we wouldn't
be here without any of you!
Best regards,
Roman

[Kim Barrett]

Post by Roman Kennke
*) shared-gc
- This is mostly boilerplate that is common to any GC
- referenceProcessor.cpp has a little change to make one assert not fail (next to CMS and G1)
- taskqueue.hpp has some small adjustments to enable subclassing

I've reviewed the shared-gc webrev. I only found a few trivial nits.

------------------------------------------------------------------------------
src/hotspot/share/gc/shared/gcName.hpp
42 NA,
43 Shenandoah,

Putting Shenandoah after NA seems odd.

------------------------------------------------------------------------------
src/hotspot/share/gc/shared/gcConfig.cpp
63 CMSGC_ONLY(static CMSArguments cmsArguments;)
...
69 SHENANDOAHGC_ONLY(static ShenandoahArguments shenandoahArguments;)

Code alignment should probably be updated.

Similarly here:
73 static const SupportedGC SupportedGCs[] = {
...
79 SHENANDOAHGC_ONLY_ARG(SupportedGC(UseShenandoahGC, CollectedHeap::Shenandoah, shenandoahArguments, "shenandoah gc"))

and here:
97 void GCConfig::fail_if_unsupported_gc_is_selected() {
...
105 NOT_SHENANDOAHGC(FAIL_IF_SELECTED(UseShenandoahGC, true));

------------------------------------------------------------------------------
src/hotspot/share/gc/shared/collectedHeap.hpp
92 // ShenandoahHeap

Moving it after ParallelScavengeHeap would give a better order.

[Roman Kennke]

Hi Kim,

Post by Kim Barrett

Post by Roman Kennke
*) shared-gc
- This is mostly boilerplate that is common to any GC
- referenceProcessor.cpp has a little change to make one assert not fail (next to CMS and G1)
- taskqueue.hpp has some small adjustments to enable subclassing

I've reviewed the shared-gc webrev. I only found a few trivial nits.
------------------------------------------------------------------------------
src/hotspot/share/gc/shared/gcName.hpp
42 NA,
43 Shenandoah,
Putting Shenandoah after NA seems odd.
------------------------------------------------------------------------------
src/hotspot/share/gc/shared/gcConfig.cpp
63 CMSGC_ONLY(static CMSArguments cmsArguments;)
...
69 SHENANDOAHGC_ONLY(static ShenandoahArguments shenandoahArguments;)
Code alignment should probably be updated.
73 static const SupportedGC SupportedGCs[] = {
...
79 SHENANDOAHGC_ONLY_ARG(SupportedGC(UseShenandoahGC, CollectedHeap::Shenandoah, shenandoahArguments, "shenandoah gc"))
97 void GCConfig::fail_if_unsupported_gc_is_selected() {
...
105 NOT_SHENANDOAHGC(FAIL_IF_SELECTED(UseShenandoahGC, true));
------------------------------------------------------------------------------
src/hotspot/share/gc/shared/collectedHeap.hpp
92 // ShenandoahHeap
Moving it after ParallelScavengeHeap would give a better order.
------------------------------------------------------------------------------
src/hotspot/share/gc/shared/barrierSetConfig.hpp
36 SHENANDOAHGC_ONLY(f(Shenandoah))
Why is this "Shenandoah" while all the others are "<something>BarrierSet"?

Don't know. I'll change it to be consistent. I have proposed a patch
with all changes to shenandoah-dev:

http://mail.openjdk.java.net/pipermail/shenandoah-dev/2018-November/008420.html

Will post an updated upstreaming webrev soon.

Thanks for reviewing,
Roman

[Per Liden]

Hi Roman,

On 11/26/18 10:39 PM, Roman Kennke wrote:
[...]

Post by Roman Kennke
 *) shared-serviceability
<http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shared-serviceability/>
    - The usual code to support another GC

Just had a quick look at the SA part. I was thinking you'd have the same
problem as ZGC here, with regards to parsing the heap and potentially
reading garbage when you step on a Klass* which had been unloaded?

cheers,
Per

[Roman Kennke]

Hi Per,

Post by Vladimir Kozlov
Hi Roman,
[...]

Post by Roman Kennke
  *) shared-serviceability
<http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shared-serviceability/>
     - The usual code to support another GC

Just had a quick look at the SA part. I was thinking you'd have the same
problem as ZGC here, with regards to parsing the heap and potentially
reading garbage when you step on a Klass* which had been unloaded?

Possible. I am myself not very familiar with SA. I guess it depends on
how SA does it: if it iterates objects via CH::object_iterate() (e.g.
same entry point as, e.g., heap-dumping code), then we should be fine.
We're kicking off a traversal rather than straight scan there. If
however SA somehow makes a raw scan itself, then we'd have the problem
you describe.

Roman

[Per Liden]

Post by Roman Kennke
Hi Per,

Post by Vladimir Kozlov
Hi Roman,
[...]

Post by Roman Kennke
  *) shared-serviceability
<http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shared-serviceability/>
     - The usual code to support another GC

Just had a quick look at the SA part. I was thinking you'd have the same
problem as ZGC here, with regards to parsing the heap and potentially
reading garbage when you step on a Klass* which had been unloaded?

Possible. I am myself not very familiar with SA. I guess it depends on
how SA does it: if it iterates objects via CH::object_iterate() (e.g.
same entry point as, e.g., heap-dumping code), then we should be fine.
We're kicking off a traversal rather than straight scan there. If
however SA somehow makes a raw scan itself, then we'd have the problem
you describe.

The SA does a raw scan itself, which is the root of the problem.
ObejctHeap.iterateLiveRegions() will locate the first object in a region
by doing

OopHandle handle = bottom.addOffsetToAsOopHandle(0);

and to get the next object it does

handle.addOffsetToAsOopHandle(obj.getObjectSize());

and you'll crash. So I'm afraid this will not work for Shenandoah either.

cheers,
Per

Post by Roman Kennke
Roman

[Roman Kennke]

Post by Per Liden

Post by Roman Kennke
Hi Per,

Post by Vladimir Kozlov
Hi Roman,
[...]

Post by Roman Kennke
   *) shared-serviceability
<http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shared-serviceability/>
      - The usual code to support another GC

Just had a quick look at the SA part. I was thinking you'd have the same
problem as ZGC here, with regards to parsing the heap and potentially
reading garbage when you step on a Klass* which had been unloaded?

Possible. I am myself not very familiar with SA. I guess it depends on
how SA does it: if it iterates objects via CH::object_iterate() (e.g.
same entry point as, e.g., heap-dumping code), then we should be fine.
We're kicking off a traversal rather than straight scan there. If
however SA somehow makes a raw scan itself, then we'd have the problem
you describe.

The SA does a raw scan itself, which is the root of the problem.
ObejctHeap.iterateLiveRegions() will locate the first object in a region
by doing
  OopHandle handle = bottom.addOffsetToAsOopHandle(0);
and to get the next object it does
  handle.addOffsetToAsOopHandle(obj.getObjectSize());
and you'll crash. So I'm afraid this will not work for Shenandoah either.

Alright. I'll 'disable' it like you did with ZGC then. Thanks for
pointing it out.

I'm wondering: this would crash with G1 and
+ClassUnloadingWithConcurrentMark too, then?

Roman

[Per Liden]

Hi,

Post by Roman Kennke

Post by Per Liden

Post by Roman Kennke
Hi Per,

Post by Vladimir Kozlov
Hi Roman,
[...]

Post by Roman Kennke
   *) shared-serviceability
<http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shared-serviceability/>
      - The usual code to support another GC

Just had a quick look at the SA part. I was thinking you'd have the same
problem as ZGC here, with regards to parsing the heap and potentially
reading garbage when you step on a Klass* which had been unloaded?

Possible. I am myself not very familiar with SA. I guess it depends on
how SA does it: if it iterates objects via CH::object_iterate() (e.g.
same entry point as, e.g., heap-dumping code), then we should be fine.
We're kicking off a traversal rather than straight scan there. If
however SA somehow makes a raw scan itself, then we'd have the problem
you describe.

The SA does a raw scan itself, which is the root of the problem.
ObejctHeap.iterateLiveRegions() will locate the first object in a region
by doing
  OopHandle handle = bottom.addOffsetToAsOopHandle(0);
and to get the next object it does
  handle.addOffsetToAsOopHandle(obj.getObjectSize());
and you'll crash. So I'm afraid this will not work for Shenandoah either.

Alright. I'll 'disable' it like you did with ZGC then. Thanks for
pointing it out.
I'm wondering: this would crash with G1 and
+ClassUnloadingWithConcurrentMark too, then?

That's a very good point and I think you're actually right. Would be
good if someone from the G1 camp could confirm.

cheers,
Per

[Roman Kennke]

Alright, we fixed:
- The minor issues that Kim reported in shared-gc
- A lot of fixes in shared-tests according to Leonid's review
- Disabled SA heapdumping similar to ZGC as Per suggested

Some notes:
Leonid: test/hotspot/jtreg/gc/TestFullGCCount.java was actually
correct. The @requires there means to exclude runs with both CMS and
ExplicitGCInvokesConcurrent at the same time, because that would be
(expectedly) failing. It can run CMS, default GC and any other GC just
fine. Adding the same clause for Shenandoah means the same, and filters
the combination (+UseShenandoahGC)+(+ExplicitGCInvokesConcurrent). I
made the condition a bit clearer by avoiding triple-negation.

See:
http://mail.openjdk.java.net/pipermail/shenandoah-dev/2018-November/008457.html

Per: Disabling the SA part for heapdumping makes 2 tests fail:
- test/hotspot/jtreg/serviceability/sa/ClhsdbJhisto.java
- test/hotspot/jtreg/serviceability/sa/TestHeapDumpForLargeArray.java

we filter them for Shenandoah now. I'm wondering: how do you get past
those with ZGC?

See:
http://mail.openjdk.java.net/pipermail/shenandoah-dev/2018-November/008466.html

(Note to Leonid and tests reviewers: I'll add those related filters in
next round).

Vladimir: Roland integrated a bunch of changes to make loop* code look
better. I can tell that we're not done with C2 yet. Can you look over
the code and see what is ok, and especially what is not ok, so that we
can focus our efforts on the relevant parts?

Updated set of webrevs:
http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/01/

Thanks,
Roman

Post by Roman Kennke
Hi,
This is the first round of changes for including Shenandoah GC into
mainline.
I divided the review into parts that roughly correspond to the mailing lists
that would normally review it, and I divided it into 'shared' code
changes and
'shenandoah' code changes (actually, mostly additions). The intend is to
eventually
push them as single 'combined' changeset, once reviewed.
  https://openjdk.java.net/jeps/189
 https://bugs.openjdk.java.net/browse/JDK-8214259
  http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/
For those who want to see the full change, have a look at the
shenandoah-complete
<http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shenandoah-complete/>
directory,
it contains the full combined webrev. Alternatively, there is the file
shenandoah-master.patch
<http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shenandoah-master.patch>,
which is what I intend to commit (and which should be equivalent to the
'shenandoah-complete' webrev).
 *) shenandoah-gc
<http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shenandoah-gc/>
    - Actual Shenandoah implementation, almost completely residing in
gc/shenandoah
 *) shared-gc
<http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shared-gc/>
    - This is mostly boilerplate that is common to any GC
    - referenceProcessor.cpp has a little change to make one assert not
fail (next to CMS and G1)
    - taskqueue.hpp has some small adjustments to enable subclassing
 *) shared-serviceability
<http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shared-serviceability/>
    - The usual code to support another GC
 *) shared-runtime
<http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shared-runtime/>
    - A number of friends declarations to allow Shenandoah iterators to
hook up with,
      e.g. ClassLoaderData, CodeCache, etc
    - Warning and disabling JFR LeakProfiler
    - fieldDescriptor.hpp added is_stable() accessor, for use in
Shenandoah C2 optimizations
    - Locks initialization in mutexLocker.cpp as usual
    - VM operations defines for Shenandoah's VM ops
    - globalDefinitions.hpp added UINT64_FORMAT_HEX_W for use in
Shenandoah's logging
    - The usual macros in macro.hpp
 *) shared-build
<http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shared-build/>
    - Add shenandoah feature, enabled by default, as agreed with
Vladimir K. beforehand
    - Some flags for shenandoah-enabled compilation to get
SUPPORT_BARRIER_ON_PRIMITIVES
      and SUPPORT_NOT_TO_SPACE_INVARIANT which is required for
Shenandoah's barriers
    - --param inline-unit-growth=1000 settings for 2 shenandoah source
files, which is
      useful to get the whole marking loop inlined (observed significant
regression if we
      don't)
 *) shared-tests
<http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shared-tests/>
    - Test infrastructure to support Shenandoah
    - Shenandoah test groups
    - Exclude Shenandoah in various tests that can be run with selected GC
    - Enable/add configure for Shenandoah for tests that make sense to
run with it
 *) shenandoah-tests
<http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shenandoah-tests/>
    - Shenandoah specific tests, most reside in gc/shenandoah subdirectory
    - A couple of tests configurations have been added, e.g.
TestGCBasherWithShenandoah.java
I intentionally left out shared-compiler for now, because we have some
work left to do
there, but if you click around you'll find the patch anyway, in case you
want to take
a peek at it.
  - {Linux} x {x86_64, x86_32, armhf, aarch64, ppc64el, s390x}
  - {Windows} x {x86_64},
  - {MacOS X} x {x86_64}
  - the new Shenandoah tests
  - jcstress with/without aggressive Shenandoah verification
  - specjvm2008 with/without aggressive Shenandoah verification
I'd like to thank my collegues at Red Hat: Christine Flood, she deserves
the credit for being the original inventor of Shenandoah, Aleksey
Shiplëv, Roland Westrelin & Zhengyu Gu for their countless
contributions, everybody else in Red Hat's OpenJDK team for testing,
advice and support, my collegues in Oracle's GC, runtime and compiler
teams for tirelessly helping with and reviewing all the GC interface and
related changes, and of course the many early adopters for reporting
bugs and success stories and feature requests: we wouldn't be here
without any of you!
Best regards,
Roman

[Vladimir Kozlov]

About C2 changes.

.ad files. I am thinking may be we should put new code in separate <arch>_shenandoah.ad files and merge them only if
Shenandoah is included in built [1]. I don't like #ifdefs - you still generate this mach nodes even without Shenandoah.

debug_final_field_at() and debug_stable_field_at() are not used - could be excluded from changes.

loopnode.cpp Sometimes you add #if INCLUDE_SHENANDOAHGC and sometimes don't. Be consistent. I don't know why you need
#ifdef if such nodes should be generated without Shenandoah

compile.cpp - no need to include shenandoahBarrierSetC2.hpp

type.hpp - cast_to_nonconst() are not used.

phasetype.hpp, subnode.cpp - don't add files with only cleanup changes.

I wish loopnode.cpp and loopopts.cpp changes were more clean.

I like the idea where you expose only one ShenandoahBarrier node to C2 and hide all subnodes behind it. I really don't
want to see a lot of Shenandoah*Node new classes in C2 shared code.

Thanks,
Vladimir

[1] http://hg.openjdk.java.net/jdk/jdk/file/157c1130b46e/make/hotspot/gensrc/GensrcAdlc.gmk#l122

Post by Roman Kennke
- The minor issues that Kim reported in shared-gc
- A lot of fixes in shared-tests according to Leonid's review
- Disabled SA heapdumping similar to ZGC as Per suggested
Leonid: test/hotspot/jtreg/gc/TestFullGCCount.java was actually
ExplicitGCInvokesConcurrent at the same time, because that would be
(expectedly) failing. It can run CMS, default GC and any other GC just
fine. Adding the same clause for Shenandoah means the same, and filters
the combination (+UseShenandoahGC)+(+ExplicitGCInvokesConcurrent). I
made the condition a bit clearer by avoiding triple-negation.
http://mail.openjdk.java.net/pipermail/shenandoah-dev/2018-November/008457.html
- test/hotspot/jtreg/serviceability/sa/ClhsdbJhisto.java
- test/hotspot/jtreg/serviceability/sa/TestHeapDumpForLargeArray.java
we filter them for Shenandoah now. I'm wondering: how do you get past
those with ZGC?
http://mail.openjdk.java.net/pipermail/shenandoah-dev/2018-November/008466.html
(Note to Leonid and tests reviewers: I'll add those related filters in
next round).
Vladimir: Roland integrated a bunch of changes to make loop* code look
better. I can tell that we're not done with C2 yet. Can you look over
the code and see what is ok, and especially what is not ok, so that we
can focus our efforts on the relevant parts?
http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/01/
Thanks,
Roman

Post by Roman Kennke
Hi,
This is the first round of changes for including Shenandoah GC into
mainline.
I divided the review into parts that roughly correspond to the mailing lists
that would normally review it, and I divided it into 'shared' code
changes and
'shenandoah' code changes (actually, mostly additions). The intend is to
eventually
push them as single 'combined' changeset, once reviewed.
  https://openjdk.java.net/jeps/189
 https://bugs.openjdk.java.net/browse/JDK-8214259
  http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/
For those who want to see the full change, have a look at the
shenandoah-complete
<http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shenandoah-complete/>
directory,
it contains the full combined webrev. Alternatively, there is the file
shenandoah-master.patch
<http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shenandoah-master.patch>,
which is what I intend to commit (and which should be equivalent to the
'shenandoah-complete' webrev).
 *) shenandoah-gc
<http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shenandoah-gc/>
    - Actual Shenandoah implementation, almost completely residing in
gc/shenandoah
 *) shared-gc
<http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shared-gc/>
    - This is mostly boilerplate that is common to any GC
    - referenceProcessor.cpp has a little change to make one assert not
fail (next to CMS and G1)
    - taskqueue.hpp has some small adjustments to enable subclassing
 *) shared-serviceability
<http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shared-serviceability/>
    - The usual code to support another GC
 *) shared-runtime
<http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shared-runtime/>
    - A number of friends declarations to allow Shenandoah iterators to
hook up with,
      e.g. ClassLoaderData, CodeCache, etc
    - Warning and disabling JFR LeakProfiler
    - fieldDescriptor.hpp added is_stable() accessor, for use in
Shenandoah C2 optimizations
    - Locks initialization in mutexLocker.cpp as usual
    - VM operations defines for Shenandoah's VM ops
    - globalDefinitions.hpp added UINT64_FORMAT_HEX_W for use in
Shenandoah's logging
    - The usual macros in macro.hpp
 *) shared-build
<http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shared-build/>
    - Add shenandoah feature, enabled by default, as agreed with
Vladimir K. beforehand
    - Some flags for shenandoah-enabled compilation to get
SUPPORT_BARRIER_ON_PRIMITIVES
      and SUPPORT_NOT_TO_SPACE_INVARIANT which is required for
Shenandoah's barriers
    - --param inline-unit-growth=1000 settings for 2 shenandoah source
files, which is
      useful to get the whole marking loop inlined (observed significant
regression if we
      don't)
 *) shared-tests
<http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shared-tests/>
    - Test infrastructure to support Shenandoah
    - Shenandoah test groups
    - Exclude Shenandoah in various tests that can be run with selected GC
    - Enable/add configure for Shenandoah for tests that make sense to
run with it
 *) shenandoah-tests
<http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shenandoah-tests/>
    - Shenandoah specific tests, most reside in gc/shenandoah subdirectory
    - A couple of tests configurations have been added, e.g.
TestGCBasherWithShenandoah.java
I intentionally left out shared-compiler for now, because we have some
work left to do
there, but if you click around you'll find the patch anyway, in case you
want to take
a peek at it.
  - {Linux} x {x86_64, x86_32, armhf, aarch64, ppc64el, s390x}
  - {Windows} x {x86_64},
  - {MacOS X} x {x86_64}
  - the new Shenandoah tests
  - jcstress with/without aggressive Shenandoah verification
  - specjvm2008 with/without aggressive Shenandoah verification
I'd like to thank my collegues at Red Hat: Christine Flood, she deserves
the credit for being the original inventor of Shenandoah, Aleksey
Shiplëv, Roland Westrelin & Zhengyu Gu for their countless
contributions, everybody else in Red Hat's OpenJDK team for testing,
advice and support, my collegues in Oracle's GC, runtime and compiler
teams for tirelessly helping with and reviewing all the GC interface and
related changes, and of course the many early adopters for reporting
bugs and success stories and feature requests: we wouldn't be here
without any of you!
Best regards,
Roman

[Roman Kennke]

Hi Vladimir,

thanks for reviewing!

Post by Vladimir Kozlov
.ad files. I am thinking may be we should put new code in separate
<arch>_shenandoah.ad files and merge them only if Shenandoah is included
in built [1]. I don't like #ifdefs - you still generate this mach nodes
even without Shenandoah.

We're looking into improving the situation.

Post by Vladimir Kozlov
debug_final_field_at() and debug_stable_field_at() are not used - could
be excluded from changes.

Those are used from shenandoahBarrierSetC2.cpp and related code in in
the shenandoah-gc or shenandoah-complete webrevs.

Post by Vladimir Kozlov
loopnode.cpp Sometimes you add #if INCLUDE_SHENANDOAHGC and sometimes
don't. Be consistent. I don't know why you need #ifdef if such nodes
should be generated without Shenandoah

We'll look into improving that.

Post by Vladimir Kozlov
compile.cpp - no need to include shenandoahBarrierSetC2.hpp

Right. Will fix it.

Post by Vladimir Kozlov
type.hpp - cast_to_nonconst() are not used.

As above, this is used from shenandoahBarrierSetC2.cpp

Post by Vladimir Kozlov
phasetype.hpp, subnode.cpp - don't add files with only cleanup changes.

right. Those are leftovers and I'll remove them.

Post by Vladimir Kozlov
I wish loopnode.cpp and loopopts.cpp changes were more clean.

We'll see what we can do. Some of those could possibly be turned into
some sort of GC interface, but 1. it would be very Shenandoah-specific
and 2. require significant duplication of code.

Post by Vladimir Kozlov
I like the idea where you expose only one ShenandoahBarrier node to C2
and hide all subnodes behind it. I really don't want to see a lot of
Shenandoah*Node new classes in C2 shared code.

I understand that. We have ShenandoahBarrierNode superclass and a bunch
of ShenandoahReadBarrier, WriteBarrier and some other nodes 'behind'
this. The crux is probably all the various ShenandoahCompareAndSwap etc
variants, which need to be subclasses of their
non-Shenandoah-counterparts, and I don't see how to change that.

Thanks for reviewing and helping!
Roman

Post by Vladimir Kozlov
[1]
http://hg.openjdk.java.net/jdk/jdk/file/157c1130b46e/make/hotspot/gensrc/GensrcAdlc.gmk#l122

Post by Roman Kennke
- The minor issues that Kim reported in shared-gc
- A lot of fixes in shared-tests according to Leonid's review
- Disabled SA heapdumping similar to ZGC as Per suggested
Leonid:  test/hotspot/jtreg/gc/TestFullGCCount.java was actually
ExplicitGCInvokesConcurrent at the same time, because that would be
(expectedly) failing. It can run CMS, default GC and any other GC just
fine. Adding the same clause for Shenandoah means the same, and filters
the combination (+UseShenandoahGC)+(+ExplicitGCInvokesConcurrent). I
made the condition a bit clearer by avoiding triple-negation.
http://mail.openjdk.java.net/pipermail/shenandoah-dev/2018-November/008457.html
- test/hotspot/jtreg/serviceability/sa/ClhsdbJhisto.java
- test/hotspot/jtreg/serviceability/sa/TestHeapDumpForLargeArray.java
we filter them for Shenandoah now. I'm wondering: how do you get past
those with ZGC?
http://mail.openjdk.java.net/pipermail/shenandoah-dev/2018-November/008466.html
(Note to Leonid and tests reviewers: I'll add those related filters in
next round).
Vladimir: Roland integrated a bunch of changes to make loop* code look
better. I can tell that we're not done with C2 yet. Can you look over
the code and see what is ok, and especially what is not ok, so that we
can focus our efforts on the relevant parts?
http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/01/
Thanks,
Roman

Post by Roman Kennke
Hi,
This is the first round of changes for including Shenandoah GC into
mainline.
I divided the review into parts that roughly correspond to the mailing lists
that would normally review it, and I divided it into 'shared' code
changes and
'shenandoah' code changes (actually, mostly additions). The intend is to
eventually
push them as single 'combined' changeset, once reviewed.
   https://openjdk.java.net/jeps/189
  https://bugs.openjdk.java.net/browse/JDK-8214259
   http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/
For those who want to see the full change, have a look at the
shenandoah-complete
<http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shenandoah-complete/>
directory,
it contains the full combined webrev. Alternatively, there is the file
shenandoah-master.patch
<http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shenandoah-master.patch>,
which is what I intend to commit (and which should be equivalent to the
'shenandoah-complete' webrev).
  *) shenandoah-gc
<http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shenandoah-gc/>
     - Actual Shenandoah implementation, almost completely residing in
gc/shenandoah
  *) shared-gc
<http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shared-gc/>
     - This is mostly boilerplate that is common to any GC
     - referenceProcessor.cpp has a little change to make one assert not
fail (next to CMS and G1)
     - taskqueue.hpp has some small adjustments to enable subclassing
  *) shared-serviceability
<http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shared-serviceability/>
     - The usual code to support another GC
  *) shared-runtime
<http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shared-runtime/>
     - A number of friends declarations to allow Shenandoah iterators to
hook up with,
       e.g. ClassLoaderData, CodeCache, etc
     - Warning and disabling JFR LeakProfiler
     - fieldDescriptor.hpp added is_stable() accessor, for use in
Shenandoah C2 optimizations
     - Locks initialization in mutexLocker.cpp as usual
     - VM operations defines for Shenandoah's VM ops
     - globalDefinitions.hpp added UINT64_FORMAT_HEX_W for use in
Shenandoah's logging
     - The usual macros in macro.hpp
  *) shared-build
<http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shared-build/>
     - Add shenandoah feature, enabled by default, as agreed with
Vladimir K. beforehand
     - Some flags for shenandoah-enabled compilation to get
SUPPORT_BARRIER_ON_PRIMITIVES
       and SUPPORT_NOT_TO_SPACE_INVARIANT which is required for
Shenandoah's barriers
     - --param inline-unit-growth=1000 settings for 2 shenandoah source
files, which is
       useful to get the whole marking loop inlined (observed
significant
regression if we
       don't)
  *) shared-tests
<http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shared-tests/>
     - Test infrastructure to support Shenandoah
     - Shenandoah test groups
     - Exclude Shenandoah in various tests that can be run with
selected GC
     - Enable/add configure for Shenandoah for tests that make sense to
run with it
  *) shenandoah-tests
<http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shenandoah-tests/>
     - Shenandoah specific tests, most reside in gc/shenandoah
subdirectory
     - A couple of tests configurations have been added, e.g.
TestGCBasherWithShenandoah.java
I intentionally left out shared-compiler for now, because we have some
work left to do
there, but if you click around you'll find the patch anyway, in case you
want to take
a peek at it.
   - {Linux} x {x86_64, x86_32, armhf, aarch64, ppc64el, s390x}
   - {Windows} x {x86_64},
   - {MacOS X} x {x86_64}
   - the new Shenandoah tests
   - jcstress with/without aggressive Shenandoah verification
   - specjvm2008 with/without aggressive Shenandoah verification
I'd like to thank my collegues at Red Hat: Christine Flood, she deserves
the credit for being the original inventor of Shenandoah, Aleksey
Shiplëv, Roland Westrelin & Zhengyu Gu for their countless
contributions, everybody else in Red Hat's OpenJDK team for testing,
advice and support, my collegues in Oracle's GC, runtime and compiler
teams for tirelessly helping with and reviewing all the GC interface and
related changes, and of course the many early adopters for reporting
bugs and success stories and feature requests: we wouldn't be here
without any of you!
Best regards,
Roman

[Jini George]

Hi Roman,

The SA tests which iterate over the heap (ClhsdbJhisto.java,
TestHeapDumpFor*.java) fail if ZGC is passed as an option to these also
(Am planning on fixing these). It might be better to avoid invoking
those tests for Shenandoah now with an @requires tag.

Thank you,
Jini

Post by Roman Kennke
- The minor issues that Kim reported in shared-gc
- A lot of fixes in shared-tests according to Leonid's review
- Disabled SA heapdumping similar to ZGC as Per suggested
Leonid: test/hotspot/jtreg/gc/TestFullGCCount.java was actually
ExplicitGCInvokesConcurrent at the same time, because that would be
(expectedly) failing. It can run CMS, default GC and any other GC just
fine. Adding the same clause for Shenandoah means the same, and filters
the combination (+UseShenandoahGC)+(+ExplicitGCInvokesConcurrent). I
made the condition a bit clearer by avoiding triple-negation.
http://mail.openjdk.java.net/pipermail/shenandoah-dev/2018-November/008457.html
- test/hotspot/jtreg/serviceability/sa/ClhsdbJhisto.java
- test/hotspot/jtreg/serviceability/sa/TestHeapDumpForLargeArray.java
we filter them for Shenandoah now. I'm wondering: how do you get past
those with ZGC?
http://mail.openjdk.java.net/pipermail/shenandoah-dev/2018-November/008466.html
(Note to Leonid and tests reviewers: I'll add those related filters in
next round).
Vladimir: Roland integrated a bunch of changes to make loop* code look
better. I can tell that we're not done with C2 yet. Can you look over
the code and see what is ok, and especially what is not ok, so that we
can focus our efforts on the relevant parts?
http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/01/
Thanks,
Roman

Post by Roman Kennke
Hi,
This is the first round of changes for including Shenandoah GC into
mainline.
I divided the review into parts that roughly correspond to the mailing lists
that would normally review it, and I divided it into 'shared' code
changes and
'shenandoah' code changes (actually, mostly additions). The intend is to
eventually
push them as single 'combined' changeset, once reviewed.
  https://openjdk.java.net/jeps/189
 https://bugs.openjdk.java.net/browse/JDK-8214259
  http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/
For those who want to see the full change, have a look at the
shenandoah-complete
<http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shenandoah-complete/>
directory,
it contains the full combined webrev. Alternatively, there is the file
shenandoah-master.patch
<http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shenandoah-master.patch>,
which is what I intend to commit (and which should be equivalent to the
'shenandoah-complete' webrev).
 *) shenandoah-gc
<http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shenandoah-gc/>
    - Actual Shenandoah implementation, almost completely residing in
gc/shenandoah
 *) shared-gc
<http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shared-gc/>
    - This is mostly boilerplate that is common to any GC
    - referenceProcessor.cpp has a little change to make one assert not
fail (next to CMS and G1)
    - taskqueue.hpp has some small adjustments to enable subclassing
 *) shared-serviceability
<http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shared-serviceability/>
    - The usual code to support another GC
 *) shared-runtime
<http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shared-runtime/>
    - A number of friends declarations to allow Shenandoah iterators to
hook up with,
      e.g. ClassLoaderData, CodeCache, etc
    - Warning and disabling JFR LeakProfiler
    - fieldDescriptor.hpp added is_stable() accessor, for use in
Shenandoah C2 optimizations
    - Locks initialization in mutexLocker.cpp as usual
    - VM operations defines for Shenandoah's VM ops
    - globalDefinitions.hpp added UINT64_FORMAT_HEX_W for use in
Shenandoah's logging
    - The usual macros in macro.hpp
 *) shared-build
<http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shared-build/>
    - Add shenandoah feature, enabled by default, as agreed with
Vladimir K. beforehand
    - Some flags for shenandoah-enabled compilation to get
SUPPORT_BARRIER_ON_PRIMITIVES
      and SUPPORT_NOT_TO_SPACE_INVARIANT which is required for
Shenandoah's barriers
    - --param inline-unit-growth=1000 settings for 2 shenandoah source
files, which is
      useful to get the whole marking loop inlined (observed significant
regression if we
      don't)
 *) shared-tests
<http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shared-tests/>
    - Test infrastructure to support Shenandoah
    - Shenandoah test groups
    - Exclude Shenandoah in various tests that can be run with selected GC
    - Enable/add configure for Shenandoah for tests that make sense to
run with it
 *) shenandoah-tests
<http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shenandoah-tests/>
    - Shenandoah specific tests, most reside in gc/shenandoah subdirectory
    - A couple of tests configurations have been added, e.g.
TestGCBasherWithShenandoah.java
I intentionally left out shared-compiler for now, because we have some
work left to do
there, but if you click around you'll find the patch anyway, in case you
want to take
a peek at it.
  - {Linux} x {x86_64, x86_32, armhf, aarch64, ppc64el, s390x}
  - {Windows} x {x86_64},
  - {MacOS X} x {x86_64}
  - the new Shenandoah tests
  - jcstress with/without aggressive Shenandoah verification
  - specjvm2008 with/without aggressive Shenandoah verification
I'd like to thank my collegues at Red Hat: Christine Flood, she deserves
the credit for being the original inventor of Shenandoah, Aleksey
Shiplëv, Roland Westrelin & Zhengyu Gu for their countless
contributions, everybody else in Red Hat's OpenJDK team for testing,
advice and support, my collegues in Oracle's GC, runtime and compiler
teams for tirelessly helping with and reviewing all the GC interface and
related changes, and of course the many early adopters for reporting
bugs and success stories and feature requests: we wouldn't be here
without any of you!
Best regards,
Roman

[Roman Kennke]

Hi Jini,

Post by Jini George
The SA tests which iterate over the heap (ClhsdbJhisto.java,
TestHeapDumpFor*.java) fail if ZGC is passed as an option to these also
(Am planning on fixing these). It might be better to avoid invoking

Thanks for clarifying. Yeah, we already have a fix in shenandoah/jdk and
it will appear in next round of reviews.

Thanks,
Roman

Post by Jini George
Thank you,
Jini

Post by Roman Kennke
- The minor issues that Kim reported in shared-gc
- A lot of fixes in shared-tests according to Leonid's review
- Disabled SA heapdumping similar to ZGC as Per suggested
Leonid:  test/hotspot/jtreg/gc/TestFullGCCount.java was actually
ExplicitGCInvokesConcurrent at the same time, because that would be
(expectedly) failing. It can run CMS, default GC and any other GC just
fine. Adding the same clause for Shenandoah means the same, and filters
the combination (+UseShenandoahGC)+(+ExplicitGCInvokesConcurrent). I
made the condition a bit clearer by avoiding triple-negation.
http://mail.openjdk.java.net/pipermail/shenandoah-dev/2018-November/008457.html
- test/hotspot/jtreg/serviceability/sa/ClhsdbJhisto.java
- test/hotspot/jtreg/serviceability/sa/TestHeapDumpForLargeArray.java
we filter them for Shenandoah now. I'm wondering: how do you get past
those with ZGC?
http://mail.openjdk.java.net/pipermail/shenandoah-dev/2018-November/008466.html
(Note to Leonid and tests reviewers: I'll add those related filters in
next round).
Vladimir: Roland integrated a bunch of changes to make loop* code look
better. I can tell that we're not done with C2 yet. Can you look over
the code and see what is ok, and especially what is not ok, so that we
can focus our efforts on the relevant parts?
http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/01/
Thanks,
Roman

Post by Roman Kennke
Hi,
This is the first round of changes for including Shenandoah GC into
mainline.
I divided the review into parts that roughly correspond to the mailing lists
that would normally review it, and I divided it into 'shared' code
changes and
'shenandoah' code changes (actually, mostly additions). The intend is to
eventually
push them as single 'combined' changeset, once reviewed.
   https://openjdk.java.net/jeps/189
  https://bugs.openjdk.java.net/browse/JDK-8214259
   http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/
For those who want to see the full change, have a look at the
shenandoah-complete
<http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shenandoah-complete/>
directory,
it contains the full combined webrev. Alternatively, there is the file
shenandoah-master.patch
<http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shenandoah-master.patch>,
which is what I intend to commit (and which should be equivalent to the
'shenandoah-complete' webrev).
  *) shenandoah-gc
<http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shenandoah-gc/>
     - Actual Shenandoah implementation, almost completely residing in
gc/shenandoah
  *) shared-gc
<http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shared-gc/>
     - This is mostly boilerplate that is common to any GC
     - referenceProcessor.cpp has a little change to make one assert not
fail (next to CMS and G1)
     - taskqueue.hpp has some small adjustments to enable subclassing
  *) shared-serviceability
<http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shared-serviceability/>
     - The usual code to support another GC
  *) shared-runtime
<http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shared-runtime/>
     - A number of friends declarations to allow Shenandoah iterators to
hook up with,
       e.g. ClassLoaderData, CodeCache, etc
     - Warning and disabling JFR LeakProfiler
     - fieldDescriptor.hpp added is_stable() accessor, for use in
Shenandoah C2 optimizations
     - Locks initialization in mutexLocker.cpp as usual
     - VM operations defines for Shenandoah's VM ops
     - globalDefinitions.hpp added UINT64_FORMAT_HEX_W for use in
Shenandoah's logging
     - The usual macros in macro.hpp
  *) shared-build
<http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shared-build/>
     - Add shenandoah feature, enabled by default, as agreed with
Vladimir K. beforehand
     - Some flags for shenandoah-enabled compilation to get
SUPPORT_BARRIER_ON_PRIMITIVES
       and SUPPORT_NOT_TO_SPACE_INVARIANT which is required for
Shenandoah's barriers
     - --param inline-unit-growth=1000 settings for 2 shenandoah source
files, which is
       useful to get the whole marking loop inlined (observed
significant
regression if we
       don't)
  *) shared-tests
<http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shared-tests/>
     - Test infrastructure to support Shenandoah
     - Shenandoah test groups
     - Exclude Shenandoah in various tests that can be run with
selected GC
     - Enable/add configure for Shenandoah for tests that make sense to
run with it
  *) shenandoah-tests
<http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shenandoah-tests/>
     - Shenandoah specific tests, most reside in gc/shenandoah
subdirectory
     - A couple of tests configurations have been added, e.g.
TestGCBasherWithShenandoah.java
I intentionally left out shared-compiler for now, because we have some
work left to do
there, but if you click around you'll find the patch anyway, in case you
want to take
a peek at it.
   - {Linux} x {x86_64, x86_32, armhf, aarch64, ppc64el, s390x}
   - {Windows} x {x86_64},
   - {MacOS X} x {x86_64}
   - the new Shenandoah tests
   - jcstress with/without aggressive Shenandoah verification
   - specjvm2008 with/without aggressive Shenandoah verification
I'd like to thank my collegues at Red Hat: Christine Flood, she deserves
the credit for being the original inventor of Shenandoah, Aleksey
Shiplëv, Roland Westrelin & Zhengyu Gu for their countless
contributions, everybody else in Red Hat's OpenJDK team for testing,
advice and support, my collegues in Oracle's GC, runtime and compiler
teams for tirelessly helping with and reviewing all the GC interface and
related changes, and of course the many early adopters for reporting
bugs and success stories and feature requests: we wouldn't be here
without any of you!
Best regards,
Roman

[Per Liden]

Oh, I filed a bug [1] and sent out a fix for review, before noticing
Jini's mail.

[1] https://bugs.openjdk.java.net/browse/JDK-8214484

/Per

Post by Vladimir Kozlov
Hi Roman,
The SA tests which iterate over the heap (ClhsdbJhisto.java,
TestHeapDumpFor*.java) fail if ZGC is passed as an option to these also
(Am planning on fixing these). It might be better to avoid invoking
Thank you,
Jini

Post by Roman Kennke
- The minor issues that Kim reported in shared-gc
- A lot of fixes in shared-tests according to Leonid's review
- Disabled SA heapdumping similar to ZGC as Per suggested
Leonid:  test/hotspot/jtreg/gc/TestFullGCCount.java was actually
ExplicitGCInvokesConcurrent at the same time, because that would be
(expectedly) failing. It can run CMS, default GC and any other GC just
fine. Adding the same clause for Shenandoah means the same, and filters
the combination (+UseShenandoahGC)+(+ExplicitGCInvokesConcurrent). I
made the condition a bit clearer by avoiding triple-negation.
http://mail.openjdk.java.net/pipermail/shenandoah-dev/2018-November/008457.html
- test/hotspot/jtreg/serviceability/sa/ClhsdbJhisto.java
- test/hotspot/jtreg/serviceability/sa/TestHeapDumpForLargeArray.java
we filter them for Shenandoah now. I'm wondering: how do you get past
those with ZGC?
http://mail.openjdk.java.net/pipermail/shenandoah-dev/2018-November/008466.html
(Note to Leonid and tests reviewers: I'll add those related filters in
next round).
Vladimir: Roland integrated a bunch of changes to make loop* code look
better. I can tell that we're not done with C2 yet. Can you look over
the code and see what is ok, and especially what is not ok, so that we
can focus our efforts on the relevant parts?
http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/01/
Thanks,
Roman

Post by Roman Kennke
Hi,
This is the first round of changes for including Shenandoah GC into
mainline.
I divided the review into parts that roughly correspond to the mailing lists
that would normally review it, and I divided it into 'shared' code
changes and
'shenandoah' code changes (actually, mostly additions). The intend is to
eventually
push them as single 'combined' changeset, once reviewed.
   https://openjdk.java.net/jeps/189
  https://bugs.openjdk.java.net/browse/JDK-8214259
   http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/
For those who want to see the full change, have a look at the
shenandoah-complete
<http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shenandoah-complete/>
directory,
it contains the full combined webrev. Alternatively, there is the file
shenandoah-master.patch
<http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shenandoah-master.patch>,
which is what I intend to commit (and which should be equivalent to the
'shenandoah-complete' webrev).
  *) shenandoah-gc
<http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shenandoah-gc/>
     - Actual Shenandoah implementation, almost completely residing in
gc/shenandoah
  *) shared-gc
<http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shared-gc/>
     - This is mostly boilerplate that is common to any GC
     - referenceProcessor.cpp has a little change to make one assert not
fail (next to CMS and G1)
     - taskqueue.hpp has some small adjustments to enable subclassing
  *) shared-serviceability
<http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shared-serviceability/>
     - The usual code to support another GC
  *) shared-runtime
<http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shared-runtime/>
     - A number of friends declarations to allow Shenandoah iterators to
hook up with,
       e.g. ClassLoaderData, CodeCache, etc
     - Warning and disabling JFR LeakProfiler
     - fieldDescriptor.hpp added is_stable() accessor, for use in
Shenandoah C2 optimizations
     - Locks initialization in mutexLocker.cpp as usual
     - VM operations defines for Shenandoah's VM ops
     - globalDefinitions.hpp added UINT64_FORMAT_HEX_W for use in
Shenandoah's logging
     - The usual macros in macro.hpp
  *) shared-build
<http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shared-build/>
     - Add shenandoah feature, enabled by default, as agreed with
Vladimir K. beforehand
     - Some flags for shenandoah-enabled compilation to get
SUPPORT_BARRIER_ON_PRIMITIVES
       and SUPPORT_NOT_TO_SPACE_INVARIANT which is required for
Shenandoah's barriers
     - --param inline-unit-growth=1000 settings for 2 shenandoah source
files, which is
       useful to get the whole marking loop inlined (observed
significant
regression if we
       don't)
  *) shared-tests
<http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shared-tests/>
     - Test infrastructure to support Shenandoah
     - Shenandoah test groups
     - Exclude Shenandoah in various tests that can be run with
selected GC
     - Enable/add configure for Shenandoah for tests that make sense to
run with it
  *) shenandoah-tests
<http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shenandoah-tests/>
     - Shenandoah specific tests, most reside in gc/shenandoah
subdirectory
     - A couple of tests configurations have been added, e.g.
TestGCBasherWithShenandoah.java
I intentionally left out shared-compiler for now, because we have some
work left to do
there, but if you click around you'll find the patch anyway, in case you
want to take
a peek at it.
   - {Linux} x {x86_64, x86_32, armhf, aarch64, ppc64el, s390x}
   - {Windows} x {x86_64},
   - {MacOS X} x {x86_64}
   - the new Shenandoah tests
   - jcstress with/without aggressive Shenandoah verification
   - specjvm2008 with/without aggressive Shenandoah verification
I'd like to thank my collegues at Red Hat: Christine Flood, she deserves
the credit for being the original inventor of Shenandoah, Aleksey
Shiplëv, Roland Westrelin & Zhengyu Gu for their countless
contributions, everybody else in Red Hat's OpenJDK team for testing,
advice and support, my collegues in Oracle's GC, runtime and compiler
teams for tirelessly helping with and reviewing all the GC interface and
related changes, and of course the many early adopters for reporting
bugs and success stories and feature requests: we wouldn't be here
without any of you!
Best regards,
Roman

[Per Liden]

Hi Roman,

On 11/28/18 11:29 PM, Roman Kennke wrote:
[...]

Post by Roman Kennke
- test/hotspot/jtreg/serviceability/sa/ClhsdbJhisto.java
- test/hotspot/jtreg/serviceability/sa/TestHeapDumpForLargeArray.java
we filter them for Shenandoah now. I'm wondering: how do you get past
those with ZGC?

Good catch! These don't work with ZGC either. As far as I can tell,
these tests are not part of any test tier, so they are never executed in
our CI pipeline and have gone under the radar.

I'll file a bug and add the appropriate @require tag to exclude ZGC. I'm
not sure if there's a reason they are excluded from our CI, or if they
should be added. Leonid, do you know?

Thanks!
/Per

[Roman Kennke]

Hello all,

Thanks so far for all the reviews and support!

I forgot to update the 'round' yesterday. We are in round 3 now :-)
Also, I fixed the numbering of my webrevs to match the review-round. ;-)

Things we've changed today:
- We moved shenandoah-specific code out of .ad files into our own .ad
files under gc/shenandoah (in shenandoah-gc), how cool is that? This
requires an addition in build machinery though, see
make/hotspot/gensrc/GensrcAdlc.gmk (in shared-build).
- Improved zero-disabling and build-code-simplification as suggested by
Magnus and Per
- Cleaned up some leftovers in C2
- Improved C2 loop opts code by introducing another APIs in
BarrierSetC2. See the new APIs in shared-gc under BarrierSetC2.hpp.
- I don't see where it makes sense to put INCLUDE_SHENANDOAHGC guards now.
- We would all very much prefer to keep ShenandoahXYZNode names, as
noted earlier. This stuff is Shenandoah-specific, so let's just call it
that.
- Rehashed Shenandoah tests (formatting, naming, directory layout, etc)
- Rebased on jdk-12+22

- Question: let us know if you need separate RFE for the new BSC2 APIs?

http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/03/

Thanks,
Roman

Post by Roman Kennke
- The minor issues that Kim reported in shared-gc
- A lot of fixes in shared-tests according to Leonid's review
- Disabled SA heapdumping similar to ZGC as Per suggested
Leonid: test/hotspot/jtreg/gc/TestFullGCCount.java was actually
ExplicitGCInvokesConcurrent at the same time, because that would be
(expectedly) failing. It can run CMS, default GC and any other GC just
fine. Adding the same clause for Shenandoah means the same, and filters
the combination (+UseShenandoahGC)+(+ExplicitGCInvokesConcurrent). I
made the condition a bit clearer by avoiding triple-negation.
http://mail.openjdk.java.net/pipermail/shenandoah-dev/2018-November/008457.html
- test/hotspot/jtreg/serviceability/sa/ClhsdbJhisto.java
- test/hotspot/jtreg/serviceability/sa/TestHeapDumpForLargeArray.java
we filter them for Shenandoah now. I'm wondering: how do you get past
those with ZGC?
http://mail.openjdk.java.net/pipermail/shenandoah-dev/2018-November/008466.html
(Note to Leonid and tests reviewers: I'll add those related filters in
next round).
Vladimir: Roland integrated a bunch of changes to make loop* code look
better. I can tell that we're not done with C2 yet. Can you look over
the code and see what is ok, and especially what is not ok, so that we
can focus our efforts on the relevant parts?
http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/01/
Thanks,
Roman

Post by Roman Kennke
Hi,
This is the first round of changes for including Shenandoah GC into
mainline.
I divided the review into parts that roughly correspond to the mailing lists
that would normally review it, and I divided it into 'shared' code
changes and
'shenandoah' code changes (actually, mostly additions). The intend is to
eventually
push them as single 'combined' changeset, once reviewed.
  https://openjdk.java.net/jeps/189
 https://bugs.openjdk.java.net/browse/JDK-8214259
  http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/
For those who want to see the full change, have a look at the
shenandoah-complete
<http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shenandoah-complete/>
directory,
it contains the full combined webrev. Alternatively, there is the file
shenandoah-master.patch
<http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shenandoah-master.patch>,
which is what I intend to commit (and which should be equivalent to the
'shenandoah-complete' webrev).
 *) shenandoah-gc
<http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shenandoah-gc/>
    - Actual Shenandoah implementation, almost completely residing in
gc/shenandoah
 *) shared-gc
<http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shared-gc/>
    - This is mostly boilerplate that is common to any GC
    - referenceProcessor.cpp has a little change to make one assert not
fail (next to CMS and G1)
    - taskqueue.hpp has some small adjustments to enable subclassing
 *) shared-serviceability
<http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shared-serviceability/>
    - The usual code to support another GC
 *) shared-runtime
<http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shared-runtime/>
    - A number of friends declarations to allow Shenandoah iterators to
hook up with,
      e.g. ClassLoaderData, CodeCache, etc
    - Warning and disabling JFR LeakProfiler
    - fieldDescriptor.hpp added is_stable() accessor, for use in
Shenandoah C2 optimizations
    - Locks initialization in mutexLocker.cpp as usual
    - VM operations defines for Shenandoah's VM ops
    - globalDefinitions.hpp added UINT64_FORMAT_HEX_W for use in
Shenandoah's logging
    - The usual macros in macro.hpp
 *) shared-build
<http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shared-build/>
    - Add shenandoah feature, enabled by default, as agreed with
Vladimir K. beforehand
    - Some flags for shenandoah-enabled compilation to get
SUPPORT_BARRIER_ON_PRIMITIVES
      and SUPPORT_NOT_TO_SPACE_INVARIANT which is required for
Shenandoah's barriers
    - --param inline-unit-growth=1000 settings for 2 shenandoah source
files, which is
      useful to get the whole marking loop inlined (observed significant
regression if we
      don't)
 *) shared-tests
<http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shared-tests/>
    - Test infrastructure to support Shenandoah
    - Shenandoah test groups
    - Exclude Shenandoah in various tests that can be run with selected GC
    - Enable/add configure for Shenandoah for tests that make sense to
run with it
 *) shenandoah-tests
<http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shenandoah-tests/>
    - Shenandoah specific tests, most reside in gc/shenandoah subdirectory
    - A couple of tests configurations have been added, e.g.
TestGCBasherWithShenandoah.java
I intentionally left out shared-compiler for now, because we have some
work left to do
there, but if you click around you'll find the patch anyway, in case you
want to take
a peek at it.
  - {Linux} x {x86_64, x86_32, armhf, aarch64, ppc64el, s390x}
  - {Windows} x {x86_64},
  - {MacOS X} x {x86_64}
  - the new Shenandoah tests
  - jcstress with/without aggressive Shenandoah verification
  - specjvm2008 with/without aggressive Shenandoah verification
I'd like to thank my collegues at Red Hat: Christine Flood, she deserves
the credit for being the original inventor of Shenandoah, Aleksey
Shiplëv, Roland Westrelin & Zhengyu Gu for their countless
contributions, everybody else in Red Hat's OpenJDK team for testing,
advice and support, my collegues in Oracle's GC, runtime and compiler
teams for tirelessly helping with and reviewing all the GC interface and
related changes, and of course the many early adopters for reporting
bugs and success stories and feature requests: we wouldn't be here
without any of you!
Best regards,
Roman

[Per Liden]

Hi Roman,

On 11/29/18 9:41 PM, Roman Kennke wrote:
[...]

Post by Roman Kennke
http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/03/

Some comments (I didn't look at the compiler part):


src/hotspot/share/gc/shared/barrierSetConfig.hpp
src/hotspot/share/gc/shared/barrierSetConfig.inline.hpp
-------------------------------------------------------
Please insert the Shenandoah lines so that the lists remain
alphabetically sorted on GC names.


src/hotspot/share/gc/shared/gcCause.cpp
---------------------------------------
Add the Shenandoah stuff after _dcmd_gc_run to match the order in the
header file.


src/hotspot/share/gc/shared/gcConfig.cpp
----------------------------------------
Please insert the Shenandoah lines so that the lists remain
alphabetically sorted on GC names. Only the last change is in the right
place now.


src/hotspot/share/gc/shared/gcTrace.hpp
---------------------------------------
Please move this new class to a gc/shenandoah/shanandoahTrace.hpp file
(or something similar).


src/hotspot/share/gc/shared/gc_globals.hpp
src/hotspot/share/gc/shared/vmStructs_gc.hpp
------------------------------------------
Please insert the INCLUDE_SHENANDOAHGC/SHENANDOAHGC_ONLY stuff so that
the lists remain alphabetically sorted on GC names.


src/hotspot/share/utilities/globalDefinitions.hpp
-------------------------------------------------
I think you want to call the new macro UINT64_FORMAT_X_W, to maintain
similarity to it's sibling UINT64_FORMAT_X. Also please adjust the
indentation/alignment so the list of macros remains nicely structured.


src/hotspot/share/utilities/macros.hpp
--------------------------------------
Please insert the Shenandoah lines so that the lists remain
alphabetically sorted on GC names.


src/jdk.hotspot.agent/share/classes/sun/jvm/hotspot/gc/shenandoah/ShenandoahHeap.java
src/jdk.hotspot.agent/share/classes/sun/jvm/hotspot/gc/shenandoah/ShenandoahHeapRegion.java
-------------------------------------------------------------------------------
Since the heap walking is now disabled, it looks like there's quite a
bit of code here that can be removed.


test/hotspot/jtreg/*
--------------------
As Aleksey and I discussed in a separate thread. Using
'vm.gc.Shenandoah' does not mean the same thing as 'vm.gc ==
"Shenandoah"', and you typically want to use 'vm.gc == "Shenandoah"' in
most cases (I did't check all), like in tests that aren't Shenandoah
specific.


src/hotspot/share/gc/shenandoah/shenandoah_globals.hpp
------------------------------------------------------
I don't think you want to expose the following flags. Setting any of
them to false will crash the VM, right? If you really want to keep them
I'd suggest you make them develop-flags.

346 diagnostic(bool, ShenandoahSATBBarrier, true,
\
347 "Turn on/off SATB barriers in Shenandoah")
\
348
\
349 diagnostic(bool, ShenandoahKeepAliveBarrier, true,
\
350 "Turn on/off keep alive barriers in Shenandoah")
\
351
\
352 diagnostic(bool, ShenandoahWriteBarrier, true,
\
353 "Turn on/off write barriers in Shenandoah")
\
354
\
355 diagnostic(bool, ShenandoahReadBarrier, true,
\
356 "Turn on/off read barriers in Shenandoah")
\
357
\
358 diagnostic(bool, ShenandoahStoreValEnqueueBarrier, false,
\
359 "Turn on/off enqueuing of oops for storeval barriers")
\
360
\
361 diagnostic(bool, ShenandoahStoreValReadBarrier, true,
\
362 "Turn on/off store val read barriers in Shenandoah")
\
363
\
364 diagnostic(bool, ShenandoahCASBarrier, true,
\
365 "Turn on/off CAS barriers in Shenandoah")
\
366
\
367 diagnostic(bool, ShenandoahAcmpBarrier, true,
\
368 "Turn on/off acmp barriers in Shenandoah")
\
369
\
370 diagnostic(bool, ShenandoahCloneBarrier, true,
\
371 "Turn on/off clone barriers in Shenandoah")
\


cheers,
Per

[Roman Kennke]

Hi Per,

Thanks for taking time to look at this!

We will take care of all the issues you mentioned.

Regarding the flags, I think they are useful as they are now. Shenandoah
can be run in 'passive' mode, which doesn't involve any concurrent GC
(iow, it runs kindof like Parallel GC). In this mode we can selectively
turn on/off different kinds of barriers. This is useful:
- for debugging. if we see a crash and we suspect it's caused by a
certain type of barrier, we can turn on/off those barriers to narrow down
- for performance experiments: passive w/o any barriers give a good
baseline against which we can measure impact of types of barriers.

Debugging may or may not be useful in develop mode (some bugs only show
up in product build), performance work quite definitely is not useful in
develop builds, and therefore I think it makes sense to keep them as
diagnostic, because that is what they are: diagnostic flags.

Makes sense?

Thanks,
Roman

Post by Kim Barrett
[...]

Post by Roman Kennke
http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/03/

src/hotspot/share/gc/shared/barrierSetConfig.hpp
src/hotspot/share/gc/shared/barrierSetConfig.inline.hpp
-------------------------------------------------------
Please insert the Shenandoah lines so that the lists remain
alphabetically sorted on GC names.
src/hotspot/share/gc/shared/gcCause.cpp
---------------------------------------
Add the Shenandoah stuff after _dcmd_gc_run to match the order in the
header file.
src/hotspot/share/gc/shared/gcConfig.cpp
----------------------------------------
Please insert the Shenandoah lines so that the lists remain
alphabetically sorted on GC names. Only the last change is in the right
place now.
src/hotspot/share/gc/shared/gcTrace.hpp
---------------------------------------
Please move this new class to a gc/shenandoah/shanandoahTrace.hpp file
(or something similar).
src/hotspot/share/gc/shared/gc_globals.hpp
src/hotspot/share/gc/shared/vmStructs_gc.hpp
------------------------------------------
Please insert the INCLUDE_SHENANDOAHGC/SHENANDOAHGC_ONLY stuff so that
the lists remain alphabetically sorted on GC names.
src/hotspot/share/utilities/globalDefinitions.hpp
-------------------------------------------------
I think you want to call the new macro UINT64_FORMAT_X_W, to maintain
similarity to it's sibling UINT64_FORMAT_X. Also please adjust the
indentation/alignment so the list of macros remains nicely structured.
src/hotspot/share/utilities/macros.hpp
--------------------------------------
Please insert the Shenandoah lines so that the lists remain
alphabetically sorted on GC names.
src/jdk.hotspot.agent/share/classes/sun/jvm/hotspot/gc/shenandoah/ShenandoahHeap.java
src/jdk.hotspot.agent/share/classes/sun/jvm/hotspot/gc/shenandoah/ShenandoahHeapRegion.java
-------------------------------------------------------------------------------
Since the heap walking is now disabled, it looks like there's quite a
bit of code here that can be removed.
test/hotspot/jtreg/*
--------------------
As Aleksey and I discussed in a separate thread. Using
'vm.gc.Shenandoah' does not mean the same thing as 'vm.gc ==
"Shenandoah"', and you typically want to use 'vm.gc == "Shenandoah"' in
most cases (I did't check all), like in tests that aren't Shenandoah
specific.
src/hotspot/share/gc/shenandoah/shenandoah_globals.hpp
------------------------------------------------------
I don't think you want to expose the following flags. Setting any of
them to false will crash the VM, right? If you really want to keep them
I'd suggest you make them develop-flags.
 346   diagnostic(bool, ShenandoahSATBBarrier, true,          \
 347           "Turn on/off SATB barriers in Shenandoah")          \
 348          \
 349   diagnostic(bool, ShenandoahKeepAliveBarrier, true,          \
 350           "Turn on/off keep alive barriers in Shenandoah")          \
 351          \
 352   diagnostic(bool, ShenandoahWriteBarrier, true,          \
 353           "Turn on/off write barriers in Shenandoah")          \
 354          \
 355   diagnostic(bool, ShenandoahReadBarrier, true,          \
 356           "Turn on/off read barriers in Shenandoah")          \
 357          \
 358   diagnostic(bool, ShenandoahStoreValEnqueueBarrier, false,          \
 359           "Turn on/off enqueuing of oops for storeval barriers")
         \
 360          \
 361   diagnostic(bool, ShenandoahStoreValReadBarrier, true,          \
 362           "Turn on/off store val read barriers in Shenandoah")
         \
 363          \
 364   diagnostic(bool, ShenandoahCASBarrier, true,          \
 365           "Turn on/off CAS barriers in Shenandoah")          \
 366          \
 367   diagnostic(bool, ShenandoahAcmpBarrier, true,          \
 368           "Turn on/off acmp barriers in Shenandoah")          \
 369          \
 370   diagnostic(bool, ShenandoahCloneBarrier, true,          \
 371           "Turn on/off clone barriers in Shenandoah")          \
cheers,
Per

[Per Liden]

Hi Roman,

Post by Roman Kennke
Hi Per,
Thanks for taking time to look at this!
We will take care of all the issues you mentioned.
Regarding the flags, I think they are useful as they are now. Shenandoah
can be run in 'passive' mode, which doesn't involve any concurrent GC
(iow, it runs kindof like Parallel GC). In this mode we can selectively
- for debugging. if we see a crash and we suspect it's caused by a
certain type of barrier, we can turn on/off those barriers to narrow down
- for performance experiments: passive w/o any barriers give a good
baseline against which we can measure impact of types of barriers.
Debugging may or may not be useful in develop mode (some bugs only show
up in product build), performance work quite definitely is not useful in
develop builds, and therefore I think it makes sense to keep them as
diagnostic, because that is what they are: diagnostic flags.
Makes sense?

I can see how these flags can be useful. But I think you might be in
violating-spec-territory here, if you're allowing users to turn on flags
that crash the VM. If they are safe to use in 'passive' mode, then I
think there should be code in shenandoahArguments.cpp that
rejects/corrects flag combinations that are unstable.

I think the correct way to think about this is: We should pass the TCK,
regardless of which features are enabled/disabled (unless the VM barfs
at start-up because the chosen combination of flags are incompatible or
isn't supported in the current environment, etc).

CC:ing Mikael here, who might be able to shed some light on what's ok
and not ok here.

cheers,
Per

Post by Roman Kennke
Thanks,
Roman

Post by Kim Barrett
[...]

Post by Roman Kennke
http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/03/

src/hotspot/share/gc/shared/barrierSetConfig.hpp
src/hotspot/share/gc/shared/barrierSetConfig.inline.hpp
-------------------------------------------------------
Please insert the Shenandoah lines so that the lists remain
alphabetically sorted on GC names.
src/hotspot/share/gc/shared/gcCause.cpp
---------------------------------------
Add the Shenandoah stuff after _dcmd_gc_run to match the order in the
header file.
src/hotspot/share/gc/shared/gcConfig.cpp
----------------------------------------
Please insert the Shenandoah lines so that the lists remain
alphabetically sorted on GC names. Only the last change is in the right
place now.
src/hotspot/share/gc/shared/gcTrace.hpp
---------------------------------------
Please move this new class to a gc/shenandoah/shanandoahTrace.hpp file
(or something similar).
src/hotspot/share/gc/shared/gc_globals.hpp
src/hotspot/share/gc/shared/vmStructs_gc.hpp
------------------------------------------
Please insert the INCLUDE_SHENANDOAHGC/SHENANDOAHGC_ONLY stuff so that
the lists remain alphabetically sorted on GC names.
src/hotspot/share/utilities/globalDefinitions.hpp
-------------------------------------------------
I think you want to call the new macro UINT64_FORMAT_X_W, to maintain
similarity to it's sibling UINT64_FORMAT_X. Also please adjust the
indentation/alignment so the list of macros remains nicely structured.
src/hotspot/share/utilities/macros.hpp
--------------------------------------
Please insert the Shenandoah lines so that the lists remain
alphabetically sorted on GC names.
src/jdk.hotspot.agent/share/classes/sun/jvm/hotspot/gc/shenandoah/ShenandoahHeap.java
src/jdk.hotspot.agent/share/classes/sun/jvm/hotspot/gc/shenandoah/ShenandoahHeapRegion.java
-------------------------------------------------------------------------------
Since the heap walking is now disabled, it looks like there's quite a
bit of code here that can be removed.
test/hotspot/jtreg/*
--------------------
As Aleksey and I discussed in a separate thread. Using
'vm.gc.Shenandoah' does not mean the same thing as 'vm.gc ==
"Shenandoah"', and you typically want to use 'vm.gc == "Shenandoah"' in
most cases (I did't check all), like in tests that aren't Shenandoah
specific.
src/hotspot/share/gc/shenandoah/shenandoah_globals.hpp
------------------------------------------------------
I don't think you want to expose the following flags. Setting any of
them to false will crash the VM, right? If you really want to keep them
I'd suggest you make them develop-flags.
 346   diagnostic(bool, ShenandoahSATBBarrier, true,          \
 347           "Turn on/off SATB barriers in Shenandoah")          \
 348          \
 349   diagnostic(bool, ShenandoahKeepAliveBarrier, true,          \
 350           "Turn on/off keep alive barriers in Shenandoah")          \
 351          \
 352   diagnostic(bool, ShenandoahWriteBarrier, true,          \
 353           "Turn on/off write barriers in Shenandoah")          \
 354          \
 355   diagnostic(bool, ShenandoahReadBarrier, true,          \
 356           "Turn on/off read barriers in Shenandoah")          \
 357          \
 358   diagnostic(bool, ShenandoahStoreValEnqueueBarrier, false,          \
 359           "Turn on/off enqueuing of oops for storeval barriers")
         \
 360          \
 361   diagnostic(bool, ShenandoahStoreValReadBarrier, true,          \
 362           "Turn on/off store val read barriers in Shenandoah")
         \
 363          \
 364   diagnostic(bool, ShenandoahCASBarrier, true,          \
 365           "Turn on/off CAS barriers in Shenandoah")          \
 366          \
 367   diagnostic(bool, ShenandoahAcmpBarrier, true,          \
 368           "Turn on/off acmp barriers in Shenandoah")          \
 369          \
 370   diagnostic(bool, ShenandoahCloneBarrier, true,          \
 371           "Turn on/off clone barriers in Shenandoah")          \
cheers,
Per

[Roman Kennke]

Post by Per Liden

Post by Roman Kennke
Hi Per,
Thanks for taking time to look at this!
We will take care of all the issues you mentioned.
Regarding the flags, I think they are useful as they are now. Shenandoah
can be run in 'passive' mode, which doesn't involve any concurrent GC
(iow, it runs kindof like Parallel GC). In this mode we can selectively
- for debugging. if we see a crash and we suspect it's caused by a
certain type of barrier, we can turn on/off those barriers to narrow down
- for performance experiments: passive w/o any barriers give a good
baseline against which we can measure impact of types of barriers.
Debugging may or may not be useful in develop mode (some bugs only show
up in product build), performance work quite definitely is not useful in
develop builds, and therefore I think it makes sense to keep them as
diagnostic, because that is what they are: diagnostic flags.
Makes sense?

I can see how these flags can be useful. But I think you might be in
violating-spec-territory here, if you're allowing users to turn on flags
that crash the VM. If they are safe to use in 'passive' mode, then I
think there should be code in shenandoahArguments.cpp that
rejects/corrects flag combinations that are unstable.

Let us see if we can do that.

If you think this violates the spec, then please point to the part that
says diagnostic (!!) options must pass the TCK and/or not crash the JVM?
I mean, it's called diagnostic for a reason. It seems counter-useful to
bury diagnostic flags that we would be using for diagnostics.

Post by Per Liden
I think the correct way to think about this is: We should pass the TCK,
regardless of which features are enabled/disabled (unless the VM barfs
at start-up because the chosen combination of flags are incompatible or
isn't supported in the current environment, etc).
CC:ing Mikael here, who might be able to shed some light on what's ok
and not ok here.

Yeah, again, where is it said that diagnostic flags must even pass the
TCK? That is totally new to me.

Thanks,
Roman

[Per Liden]

Hi,

Post by Roman Kennke

Post by Per Liden

Post by Roman Kennke
Hi Per,
Thanks for taking time to look at this!
We will take care of all the issues you mentioned.
Regarding the flags, I think they are useful as they are now. Shenandoah
can be run in 'passive' mode, which doesn't involve any concurrent GC
(iow, it runs kindof like Parallel GC). In this mode we can selectively
- for debugging. if we see a crash and we suspect it's caused by a
certain type of barrier, we can turn on/off those barriers to narrow down
- for performance experiments: passive w/o any barriers give a good
baseline against which we can measure impact of types of barriers.
Debugging may or may not be useful in develop mode (some bugs only show
up in product build), performance work quite definitely is not useful in
develop builds, and therefore I think it makes sense to keep them as
diagnostic, because that is what they are: diagnostic flags.
Makes sense?

I can see how these flags can be useful. But I think you might be in
violating-spec-territory here, if you're allowing users to turn on flags
that crash the VM. If they are safe to use in 'passive' mode, then I
think there should be code in shenandoahArguments.cpp that
rejects/corrects flag combinations that are unstable.

Let us see if we can do that.
If you think this violates the spec, then please point to the part that
says diagnostic (!!) options must pass the TCK and/or not crash the JVM?
I mean, it's called diagnostic for a reason. It seems counter-useful to
bury diagnostic flags that we would be using for diagnostics.

Post by Per Liden
I think the correct way to think about this is: We should pass the TCK,
regardless of which features are enabled/disabled (unless the VM barfs
at start-up because the chosen combination of flags are incompatible or
isn't supported in the current environment, etc).
CC:ing Mikael here, who might be able to shed some light on what's ok
and not ok here.

Yeah, again, where is it said that diagnostic flags must even pass the
TCK? That is totally new to me.

I'm not really the right person to speak authoritatively about this.
However, from what I understand it doesn't matter what kind of flag it
is. If a user can enable it in a production/shipped build I think it
must pass the TCK.

But let's have someone who knows more about the rules around that weight
in on this.

cheers,
Per

[David Holmes]

Post by Per Liden
Hi,

Post by Roman Kennke

Post by Per Liden

Post by Roman Kennke
Hi Per,
Thanks for taking time to look at this!
We will take care of all the issues you mentioned.
Regarding the flags, I think they are useful as they are now. Shenandoah
can be run in 'passive' mode, which doesn't involve any concurrent GC
(iow, it runs kindof like Parallel GC). In this mode we can selectively
- for debugging. if we see a crash and we suspect it's caused by a
certain type of barrier, we can turn on/off those barriers to narrow down
- for performance experiments: passive w/o any barriers give a good
baseline against which we can measure impact of types of barriers.
Debugging may or may not be useful in develop mode (some bugs only show
up in product build), performance work quite definitely is not useful in
develop builds, and therefore I think it makes sense to keep them as
diagnostic, because that is what they are: diagnostic flags.
Makes sense?

I can see how these flags can be useful. But I think you might be in
violating-spec-territory here, if you're allowing users to turn on flags
that crash the VM. If they are safe to use in 'passive' mode, then I
think there should be code in shenandoahArguments.cpp that
rejects/corrects flag combinations that are unstable.

Let us see if we can do that.
If you think this violates the spec, then please point to the part that
says diagnostic (!!) options must pass the TCK and/or not crash the JVM?
I mean, it's called diagnostic for a reason. It seems counter-useful to
bury diagnostic flags that we would be using for diagnostics.

Post by Per Liden
I think the correct way to think about this is: We should pass the TCK,
regardless of which features are enabled/disabled (unless the VM barfs
at start-up because the chosen combination of flags are incompatible or
isn't supported in the current environment, etc).
CC:ing Mikael here, who might be able to shed some light on what's ok
and not ok here.

Yeah, again, where is it said that diagnostic flags must even pass the
TCK? That is totally new to me.

I'm not really the right person to speak authoritatively about this.
However, from what I understand it doesn't matter what kind of flag it
is. If a user can enable it in a production/shipped build I think it
must pass the TCK.

That cannot be true in a strict sense ref:

./share/runtime/globals.hpp: diagnostic(ccstr, AbortVMOnException, NULL,

which would obviously fail the TCK.

However I think there is a distinction between a flag that causes the VM
to abruptly terminate versus a flag that allows the VM to run in a
manner that violates the spec.

David
-----

Post by Per Liden
But let's have someone who knows more about the rules around that weight
in on this.
cheers,
Per

[Vladimir Kozlov]

Can you do simple verification of flags and exit VM gracefully with error message?

I think it should be fine. Crashing VM (unintentionally) is definitely not acceptable.

Thanks,
Vladimir

Post by Roman Kennke

Post by Per Liden

Post by Roman Kennke
Hi Per,
Thanks for taking time to look at this!
We will take care of all the issues you mentioned.
Regarding the flags, I think they are useful as they are now. Shenandoah
can be run in 'passive' mode, which doesn't involve any concurrent GC
(iow, it runs kindof like Parallel GC). In this mode we can selectively
- for debugging. if we see a crash and we suspect it's caused by a
certain type of barrier, we can turn on/off those barriers to narrow down
- for performance experiments: passive w/o any barriers give a good
baseline against which we can measure impact of types of barriers.
Debugging may or may not be useful in develop mode (some bugs only show
up in product build), performance work quite definitely is not useful in
develop builds, and therefore I think it makes sense to keep them as
diagnostic, because that is what they are: diagnostic flags.
Makes sense?

I can see how these flags can be useful. But I think you might be in
violating-spec-territory here, if you're allowing users to turn on flags
that crash the VM. If they are safe to use in 'passive' mode, then I
think there should be code in shenandoahArguments.cpp that
rejects/corrects flag combinations that are unstable.

Let us see if we can do that.
If you think this violates the spec, then please point to the part that
says diagnostic (!!) options must pass the TCK and/or not crash the JVM?
I mean, it's called diagnostic for a reason. It seems counter-useful to
bury diagnostic flags that we would be using for diagnostics.

Post by Per Liden
I think the correct way to think about this is: We should pass the TCK,
regardless of which features are enabled/disabled (unless the VM barfs
at start-up because the chosen combination of flags are incompatible or
isn't supported in the current environment, etc).
CC:ing Mikael here, who might be able to shed some light on what's ok
and not ok here.

Yeah, again, where is it said that diagnostic flags must even pass the
TCK? That is totally new to me.
Thanks,
Roman

[Roman Kennke]

Yes, Aleksey just implemented it. It will appear in round 4 in a bit.

Thanks,
Roman

Post by Vladimir Kozlov
Can you do simple verification of flags and exit VM gracefully with error message?
I think it should be fine. Crashing VM (unintentionally) is definitely not acceptable.
Thanks,
Vladimir

Post by Roman Kennke

Post by Per Liden

Post by Roman Kennke
Hi Per,
Thanks for taking time to look at this!
We will take care of all the issues you mentioned.
Regarding the flags, I think they are useful as they are now.

Shenandoah

Post by Roman Kennke

Post by Per Liden

Post by Roman Kennke
can be run in 'passive' mode, which doesn't involve any concurrent

GC

Post by Roman Kennke

Post by Per Liden

Post by Roman Kennke
(iow, it runs kindof like Parallel GC). In this mode we can

selectively

Post by Roman Kennke

Post by Per Liden

Post by Roman Kennke
- for debugging. if we see a crash and we suspect it's caused by a
certain type of barrier, we can turn on/off those barriers to

narrow down

Post by Roman Kennke

Post by Per Liden

Post by Roman Kennke
- for performance experiments: passive w/o any barriers give a good
baseline against which we can measure impact of types of barriers.
Debugging may or may not be useful in develop mode (some bugs only

show

Post by Roman Kennke

Post by Per Liden

Post by Roman Kennke
up in product build), performance work quite definitely is not

useful in

Post by Roman Kennke

Post by Per Liden

Post by Roman Kennke
develop builds, and therefore I think it makes sense to keep them

as

Post by Roman Kennke

Post by Per Liden

Post by Roman Kennke
diagnostic, because that is what they are: diagnostic flags.
Makes sense?

I can see how these flags can be useful. But I think you might be in
violating-spec-territory here, if you're allowing users to turn on

flags

Post by Roman Kennke

Post by Per Liden
that crash the VM. If they are safe to use in 'passive' mode, then I
think there should be code in shenandoahArguments.cpp that
rejects/corrects flag combinations that are unstable.

Let us see if we can do that.
If you think this violates the spec, then please point to the part

that

Post by Roman Kennke
says diagnostic (!!) options must pass the TCK and/or not crash the

JVM?

Post by Roman Kennke
I mean, it's called diagnostic for a reason. It seems counter-useful

to

Post by Roman Kennke
bury diagnostic flags that we would be using for diagnostics.

Post by Per Liden
I think the correct way to think about this is: We should pass the

TCK,

Post by Roman Kennke

Post by Per Liden
regardless of which features are enabled/disabled (unless the VM

barfs

Post by Roman Kennke

Post by Per Liden
at start-up because the chosen combination of flags are incompatible

or

Post by Roman Kennke

Post by Per Liden
isn't supported in the current environment, etc).
CC:ing Mikael here, who might be able to shed some light on what's

ok

Post by Roman Kennke

Post by Per Liden
and not ok here.

Yeah, again, where is it said that diagnostic flags must even pass

the

Post by Roman Kennke
TCK? That is totally new to me.
Thanks,
Roman

[Roman Kennke]

Hi all,

here comes round 4 of Shenandoah upstreaming review:

This includes fixes for the issues that Per brought up:
- Verify and gracefully reject dangerous flags combinations that
disables required barriers
- Revisited @requires filters in tests
- Trim unused code from Shenandoah's SA impl
- Move ShenandoahGCTracer to gc/shenandoah
- Fix ordering of GC names in various files
- Rename UINT64_FORMAT_HEX_W to UINT64_FORMAT_X_W

http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/04/

Thanks everybody for taking time to review this!
Roman

Post by Roman Kennke
Hello all,
Thanks so far for all the reviews and support!
I forgot to update the 'round' yesterday. We are in round 3 now :-)
Also, I fixed the numbering of my webrevs to match the review-round. ;-)
- We moved shenandoah-specific code out of .ad files into our own .ad
files under gc/shenandoah (in shenandoah-gc), how cool is that? This
requires an addition in build machinery though, see
make/hotspot/gensrc/GensrcAdlc.gmk (in shared-build).
- Improved zero-disabling and build-code-simplification as suggested by
Magnus and Per
- Cleaned up some leftovers in C2
- Improved C2 loop opts code by introducing another APIs in
BarrierSetC2. See the new APIs in shared-gc under BarrierSetC2.hpp.
- I don't see where it makes sense to put INCLUDE_SHENANDOAHGC guards now.
- We would all very much prefer to keep ShenandoahXYZNode names, as
noted earlier. This stuff is Shenandoah-specific, so let's just call it
that.
- Rehashed Shenandoah tests (formatting, naming, directory layout, etc)
- Rebased on jdk-12+22
- Question: let us know if you need separate RFE for the new BSC2 APIs?
http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/03/
Thanks,
Roman

Post by Roman Kennke
- The minor issues that Kim reported in shared-gc
- A lot of fixes in shared-tests according to Leonid's review
- Disabled SA heapdumping similar to ZGC as Per suggested
Leonid: test/hotspot/jtreg/gc/TestFullGCCount.java was actually
ExplicitGCInvokesConcurrent at the same time, because that would be
(expectedly) failing. It can run CMS, default GC and any other GC just
fine. Adding the same clause for Shenandoah means the same, and filters
the combination (+UseShenandoahGC)+(+ExplicitGCInvokesConcurrent). I
made the condition a bit clearer by avoiding triple-negation.
http://mail.openjdk.java.net/pipermail/shenandoah-dev/2018-November/008457.html
- test/hotspot/jtreg/serviceability/sa/ClhsdbJhisto.java
- test/hotspot/jtreg/serviceability/sa/TestHeapDumpForLargeArray.java
we filter them for Shenandoah now. I'm wondering: how do you get past
those with ZGC?
http://mail.openjdk.java.net/pipermail/shenandoah-dev/2018-November/008466.html
(Note to Leonid and tests reviewers: I'll add those related filters in
next round).
Vladimir: Roland integrated a bunch of changes to make loop* code look
better. I can tell that we're not done with C2 yet. Can you look over
the code and see what is ok, and especially what is not ok, so that we
can focus our efforts on the relevant parts?
http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/01/
Thanks,
Roman

Post by Roman Kennke
Hi,
This is the first round of changes for including Shenandoah GC into
mainline.
I divided the review into parts that roughly correspond to the mailing lists
that would normally review it, and I divided it into 'shared' code
changes and
'shenandoah' code changes (actually, mostly additions). The intend is to
eventually
push them as single 'combined' changeset, once reviewed.
  https://openjdk.java.net/jeps/189
 https://bugs.openjdk.java.net/browse/JDK-8214259
  http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/
For those who want to see the full change, have a look at the
shenandoah-complete
<http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shenandoah-complete/>
directory,
it contains the full combined webrev. Alternatively, there is the file
shenandoah-master.patch
<http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shenandoah-master.patch>,
which is what I intend to commit (and which should be equivalent to the
'shenandoah-complete' webrev).
 *) shenandoah-gc
<http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shenandoah-gc/>
    - Actual Shenandoah implementation, almost completely residing in
gc/shenandoah
 *) shared-gc
<http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shared-gc/>
    - This is mostly boilerplate that is common to any GC
    - referenceProcessor.cpp has a little change to make one assert not
fail (next to CMS and G1)
    - taskqueue.hpp has some small adjustments to enable subclassing
 *) shared-serviceability
<http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shared-serviceability/>
    - The usual code to support another GC
 *) shared-runtime
<http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shared-runtime/>
    - A number of friends declarations to allow Shenandoah iterators to
hook up with,
      e.g. ClassLoaderData, CodeCache, etc
    - Warning and disabling JFR LeakProfiler
    - fieldDescriptor.hpp added is_stable() accessor, for use in
Shenandoah C2 optimizations
    - Locks initialization in mutexLocker.cpp as usual
    - VM operations defines for Shenandoah's VM ops
    - globalDefinitions.hpp added UINT64_FORMAT_HEX_W for use in
Shenandoah's logging
    - The usual macros in macro.hpp
 *) shared-build
<http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shared-build/>
    - Add shenandoah feature, enabled by default, as agreed with
Vladimir K. beforehand
    - Some flags for shenandoah-enabled compilation to get
SUPPORT_BARRIER_ON_PRIMITIVES
      and SUPPORT_NOT_TO_SPACE_INVARIANT which is required for
Shenandoah's barriers
    - --param inline-unit-growth=1000 settings for 2 shenandoah source
files, which is
      useful to get the whole marking loop inlined (observed significant
regression if we
      don't)
 *) shared-tests
<http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shared-tests/>
    - Test infrastructure to support Shenandoah
    - Shenandoah test groups
    - Exclude Shenandoah in various tests that can be run with selected GC
    - Enable/add configure for Shenandoah for tests that make sense to
run with it
 *) shenandoah-tests
<http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shenandoah-tests/>
    - Shenandoah specific tests, most reside in gc/shenandoah subdirectory
    - A couple of tests configurations have been added, e.g.
TestGCBasherWithShenandoah.java
I intentionally left out shared-compiler for now, because we have some
work left to do
there, but if you click around you'll find the patch anyway, in case you
want to take
a peek at it.
  - {Linux} x {x86_64, x86_32, armhf, aarch64, ppc64el, s390x}
  - {Windows} x {x86_64},
  - {MacOS X} x {x86_64}
  - the new Shenandoah tests
  - jcstress with/without aggressive Shenandoah verification
  - specjvm2008 with/without aggressive Shenandoah verification
I'd like to thank my collegues at Red Hat: Christine Flood, she deserves
the credit for being the original inventor of Shenandoah, Aleksey
Shiplëv, Roland Westrelin & Zhengyu Gu for their countless
contributions, everybody else in Red Hat's OpenJDK team for testing,
advice and support, my collegues in Oracle's GC, runtime and compiler
teams for tirelessly helping with and reviewing all the GC interface and
related changes, and of course the many early adopters for reporting
bugs and success stories and feature requests: we wouldn't be here
without any of you!
Best regards,
Roman

[c***@oracle.com]

Hi, I looked at the runtime changes, which are very few, since you've
upstreamed most of the runtime dependent changes already.
They look good to me.
Thanks,
Coleen

(trying again with all the lists).

Post by Roman Kennke
Hi all,
- Verify and gracefully reject dangerous flags combinations that
disables required barriers
- Trim unused code from Shenandoah's SA impl
- Move ShenandoahGCTracer to gc/shenandoah
- Fix ordering of GC names in various files
- Rename UINT64_FORMAT_HEX_W to UINT64_FORMAT_X_W
http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/04/
Thanks everybody for taking time to review this!
Roman

Post by Roman Kennke
Hello all,
Thanks so far for all the reviews and support!
I forgot to update the 'round' yesterday. We are in round 3 now :-)
Also, I fixed the numbering of my webrevs to match the review-round. ;-)
- We moved shenandoah-specific code out of .ad files into our own .ad
files under gc/shenandoah (in shenandoah-gc), how cool is that? This
requires an addition in build machinery though, see
make/hotspot/gensrc/GensrcAdlc.gmk (in shared-build).
- Improved zero-disabling and build-code-simplification as suggested by
Magnus and Per
- Cleaned up some leftovers in C2
- Improved C2 loop opts code by introducing another APIs in
BarrierSetC2. See the new APIs in shared-gc under BarrierSetC2.hpp.
- I don't see where it makes sense to put INCLUDE_SHENANDOAHGC guards now.
- We would all very much prefer to keep ShenandoahXYZNode names, as
noted earlier. This stuff is Shenandoah-specific, so let's just call it
that.
- Rehashed Shenandoah tests (formatting, naming, directory layout, etc)
- Rebased on jdk-12+22
- Question: let us know if you need separate RFE for the new BSC2 APIs?
http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/03/
Thanks,
Roman

Post by Roman Kennke
- The minor issues that Kim reported in shared-gc
- A lot of fixes in shared-tests according to Leonid's review
- Disabled SA heapdumping similar to ZGC as Per suggested
Leonid: test/hotspot/jtreg/gc/TestFullGCCount.java was actually
ExplicitGCInvokesConcurrent at the same time, because that would be
(expectedly) failing. It can run CMS, default GC and any other GC just
fine. Adding the same clause for Shenandoah means the same, and filters
the combination (+UseShenandoahGC)+(+ExplicitGCInvokesConcurrent). I
made the condition a bit clearer by avoiding triple-negation.
http://mail.openjdk.java.net/pipermail/shenandoah-dev/2018-November/008457.html
- test/hotspot/jtreg/serviceability/sa/ClhsdbJhisto.java
- test/hotspot/jtreg/serviceability/sa/TestHeapDumpForLargeArray.java
we filter them for Shenandoah now. I'm wondering: how do you get past
those with ZGC?
http://mail.openjdk.java.net/pipermail/shenandoah-dev/2018-November/008466.html
(Note to Leonid and tests reviewers: I'll add those related filters in
next round).
Vladimir: Roland integrated a bunch of changes to make loop* code look
better. I can tell that we're not done with C2 yet. Can you look over
the code and see what is ok, and especially what is not ok, so that we
can focus our efforts on the relevant parts?
http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/01/
Thanks,
Roman

Post by Roman Kennke
Hi,
This is the first round of changes for including Shenandoah GC into
mainline.
I divided the review into parts that roughly correspond to the mailing lists
that would normally review it, and I divided it into 'shared' code
changes and
'shenandoah' code changes (actually, mostly additions). The intend is to
eventually
push them as single 'combined' changeset, once reviewed.
  https://openjdk.java.net/jeps/189
 https://bugs.openjdk.java.net/browse/JDK-8214259
  http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/
For those who want to see the full change, have a look at the
shenandoah-complete
<http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shenandoah-complete/>
directory,
it contains the full combined webrev. Alternatively, there is the file
shenandoah-master.patch
<http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shenandoah-master.patch>,
which is what I intend to commit (and which should be equivalent to the
'shenandoah-complete' webrev).
 *) shenandoah-gc
<http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shenandoah-gc/>
    - Actual Shenandoah implementation, almost completely residing in
gc/shenandoah
 *) shared-gc
<http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shared-gc/>
    - This is mostly boilerplate that is common to any GC
    - referenceProcessor.cpp has a little change to make one assert not
fail (next to CMS and G1)
    - taskqueue.hpp has some small adjustments to enable subclassing
 *) shared-serviceability
<http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shared-serviceability/>
    - The usual code to support another GC
 *) shared-runtime
<http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shared-runtime/>
    - A number of friends declarations to allow Shenandoah iterators to
hook up with,
      e.g. ClassLoaderData, CodeCache, etc
    - Warning and disabling JFR LeakProfiler
    - fieldDescriptor.hpp added is_stable() accessor, for use in
Shenandoah C2 optimizations
    - Locks initialization in mutexLocker.cpp as usual
    - VM operations defines for Shenandoah's VM ops
    - globalDefinitions.hpp added UINT64_FORMAT_HEX_W for use in
Shenandoah's logging
    - The usual macros in macro.hpp
 *) shared-build
<http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shared-build/>
    - Add shenandoah feature, enabled by default, as agreed with
Vladimir K. beforehand
    - Some flags for shenandoah-enabled compilation to get
SUPPORT_BARRIER_ON_PRIMITIVES
      and SUPPORT_NOT_TO_SPACE_INVARIANT which is required for
Shenandoah's barriers
    - --param inline-unit-growth=1000 settings for 2 shenandoah source
files, which is
      useful to get the whole marking loop inlined (observed significant
regression if we
      don't)
 *) shared-tests
<http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shared-tests/>
    - Test infrastructure to support Shenandoah
    - Shenandoah test groups
    - Exclude Shenandoah in various tests that can be run with selected GC
    - Enable/add configure for Shenandoah for tests that make sense to
run with it
 *) shenandoah-tests
<http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shenandoah-tests/>
    - Shenandoah specific tests, most reside in gc/shenandoah subdirectory
    - A couple of tests configurations have been added, e.g.
TestGCBasherWithShenandoah.java
I intentionally left out shared-compiler for now, because we have some
work left to do
there, but if you click around you'll find the patch anyway, in case you
want to take
a peek at it.
  - {Linux} x {x86_64, x86_32, armhf, aarch64, ppc64el, s390x}
  - {Windows} x {x86_64},
  - {MacOS X} x {x86_64}
  - the new Shenandoah tests
  - jcstress with/without aggressive Shenandoah verification
  - specjvm2008 with/without aggressive Shenandoah verification
I'd like to thank my collegues at Red Hat: Christine Flood, she deserves
the credit for being the original inventor of Shenandoah, Aleksey
Shiplëv, Roland Westrelin & Zhengyu Gu for their countless
contributions, everybody else in Red Hat's OpenJDK team for testing,
advice and support, my collegues in Oracle's GC, runtime and compiler
teams for tirelessly helping with and reviewing all the GC interface and
related changes, and of course the many early adopters for reporting
bugs and success stories and feature requests: we wouldn't be here
without any of you!
Best regards,
Roman

[John Rose]

Hi, I looked at the runtime changes, which are very few, since you've upstreamed most of the runtime dependent changes already.
They look good to me.

"since you've upstreamed most of the runtime dependent changes already"
Yes. That's what winning looks like!
— John

[Vladimir Kozlov]

Much better.

Why you need to include shenandoahBarrierSetC2.hpp in /opto/classes.cpp ? Why not include only shenandoahSupport.hpp
when new nodes are defined?

I think it is fine to not use #ifdef in loopopts.cpp when you check is_ShenandoahBarrier(). And you don't do that in
other files.

Code in opto/macro.cpp is ugly but it is only the place so we can live with it I think.

Thanks,
Vladimir

Post by Roman Kennke
Hi all,
- Verify and gracefully reject dangerous flags combinations that
disables required barriers
- Trim unused code from Shenandoah's SA impl
- Move ShenandoahGCTracer to gc/shenandoah
- Fix ordering of GC names in various files
- Rename UINT64_FORMAT_HEX_W to UINT64_FORMAT_X_W
http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/04/
Thanks everybody for taking time to review this!
Roman

Post by Roman Kennke
Hello all,
Thanks so far for all the reviews and support!
I forgot to update the 'round' yesterday. We are in round 3 now :-)
Also, I fixed the numbering of my webrevs to match the review-round. ;-)
- We moved shenandoah-specific code out of .ad files into our own .ad
files under gc/shenandoah (in shenandoah-gc), how cool is that? This
requires an addition in build machinery though, see
make/hotspot/gensrc/GensrcAdlc.gmk (in shared-build).
- Improved zero-disabling and build-code-simplification as suggested by
Magnus and Per
- Cleaned up some leftovers in C2
- Improved C2 loop opts code by introducing another APIs in
BarrierSetC2. See the new APIs in shared-gc under BarrierSetC2.hpp.
- I don't see where it makes sense to put INCLUDE_SHENANDOAHGC guards now.
- We would all very much prefer to keep ShenandoahXYZNode names, as
noted earlier. This stuff is Shenandoah-specific, so let's just call it
that.
- Rehashed Shenandoah tests (formatting, naming, directory layout, etc)
- Rebased on jdk-12+22
- Question: let us know if you need separate RFE for the new BSC2 APIs?
http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/03/
Thanks,
Roman

Post by Roman Kennke
- The minor issues that Kim reported in shared-gc
- A lot of fixes in shared-tests according to Leonid's review
- Disabled SA heapdumping similar to ZGC as Per suggested
Leonid: test/hotspot/jtreg/gc/TestFullGCCount.java was actually
ExplicitGCInvokesConcurrent at the same time, because that would be
(expectedly) failing. It can run CMS, default GC and any other GC just
fine. Adding the same clause for Shenandoah means the same, and filters
the combination (+UseShenandoahGC)+(+ExplicitGCInvokesConcurrent). I
made the condition a bit clearer by avoiding triple-negation.
http://mail.openjdk.java.net/pipermail/shenandoah-dev/2018-November/008457.html
- test/hotspot/jtreg/serviceability/sa/ClhsdbJhisto.java
- test/hotspot/jtreg/serviceability/sa/TestHeapDumpForLargeArray.java
we filter them for Shenandoah now. I'm wondering: how do you get past
those with ZGC?
http://mail.openjdk.java.net/pipermail/shenandoah-dev/2018-November/008466.html
(Note to Leonid and tests reviewers: I'll add those related filters in
next round).
Vladimir: Roland integrated a bunch of changes to make loop* code look
better. I can tell that we're not done with C2 yet. Can you look over
the code and see what is ok, and especially what is not ok, so that we
can focus our efforts on the relevant parts?
http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/01/
Thanks,
Roman

Post by Roman Kennke
Hi,
This is the first round of changes for including Shenandoah GC into
mainline.
I divided the review into parts that roughly correspond to the mailing lists
that would normally review it, and I divided it into 'shared' code
changes and
'shenandoah' code changes (actually, mostly additions). The intend is to
eventually
push them as single 'combined' changeset, once reviewed.
  https://openjdk.java.net/jeps/189
 https://bugs.openjdk.java.net/browse/JDK-8214259
  http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/
For those who want to see the full change, have a look at the
shenandoah-complete
<http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shenandoah-complete/>
directory,
it contains the full combined webrev. Alternatively, there is the file
shenandoah-master.patch
<http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shenandoah-master.patch>,
which is what I intend to commit (and which should be equivalent to the
'shenandoah-complete' webrev).
 *) shenandoah-gc
<http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shenandoah-gc/>
    - Actual Shenandoah implementation, almost completely residing in
gc/shenandoah
 *) shared-gc
<http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shared-gc/>
    - This is mostly boilerplate that is common to any GC
    - referenceProcessor.cpp has a little change to make one assert not
fail (next to CMS and G1)
    - taskqueue.hpp has some small adjustments to enable subclassing
 *) shared-serviceability
<http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shared-serviceability/>
    - The usual code to support another GC
 *) shared-runtime
<http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shared-runtime/>
    - A number of friends declarations to allow Shenandoah iterators to
hook up with,
      e.g. ClassLoaderData, CodeCache, etc
    - Warning and disabling JFR LeakProfiler
    - fieldDescriptor.hpp added is_stable() accessor, for use in
Shenandoah C2 optimizations
    - Locks initialization in mutexLocker.cpp as usual
    - VM operations defines for Shenandoah's VM ops
    - globalDefinitions.hpp added UINT64_FORMAT_HEX_W for use in
Shenandoah's logging
    - The usual macros in macro.hpp
 *) shared-build
<http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shared-build/>
    - Add shenandoah feature, enabled by default, as agreed with
Vladimir K. beforehand
    - Some flags for shenandoah-enabled compilation to get
SUPPORT_BARRIER_ON_PRIMITIVES
      and SUPPORT_NOT_TO_SPACE_INVARIANT which is required for
Shenandoah's barriers
    - --param inline-unit-growth=1000 settings for 2 shenandoah source
files, which is
      useful to get the whole marking loop inlined (observed significant
regression if we
      don't)
 *) shared-tests
<http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shared-tests/>
    - Test infrastructure to support Shenandoah
    - Shenandoah test groups
    - Exclude Shenandoah in various tests that can be run with selected GC
    - Enable/add configure for Shenandoah for tests that make sense to
run with it
 *) shenandoah-tests
<http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shenandoah-tests/>
    - Shenandoah specific tests, most reside in gc/shenandoah subdirectory
    - A couple of tests configurations have been added, e.g.
TestGCBasherWithShenandoah.java
I intentionally left out shared-compiler for now, because we have some
work left to do
there, but if you click around you'll find the patch anyway, in case you
want to take
a peek at it.
  - {Linux} x {x86_64, x86_32, armhf, aarch64, ppc64el, s390x}
  - {Windows} x {x86_64},
  - {MacOS X} x {x86_64}
  - the new Shenandoah tests
  - jcstress with/without aggressive Shenandoah verification
  - specjvm2008 with/without aggressive Shenandoah verification
I'd like to thank my collegues at Red Hat: Christine Flood, she deserves
the credit for being the original inventor of Shenandoah, Aleksey
Shiplëv, Roland Westrelin & Zhengyu Gu for their countless
contributions, everybody else in Red Hat's OpenJDK team for testing,
advice and support, my collegues in Oracle's GC, runtime and compiler
teams for tirelessly helping with and reviewing all the GC interface and
related changes, and of course the many early adopters for reporting
bugs and success stories and feature requests: we wouldn't be here
without any of you!
Best regards,
Roman

[Roman Kennke]

Hi Vladimir,

Post by Vladimir Kozlov
Much better.

Thanks!

Post by Vladimir Kozlov
Why you need to include shenandoahBarrierSetC2.hpp in /opto/classes.cpp
? Why not include only shenandoahSupport.hpp when new nodes are defined?

For no particular reason, I guess. I'll fix it in next round.

Post by Vladimir Kozlov
I think it is fine to not use #ifdef in loopopts.cpp when you check
is_ShenandoahBarrier(). And you don't do that in other files.
Code in opto/macro.cpp is ugly but it is only the place so we can live with it I think.

Ok, thanks! We'll see if we can improve macro.cpp.

Thanks,
Roman

Post by Vladimir Kozlov
Thanks,
Vladimir

Post by Roman Kennke
Hi all,
- Verify and gracefully reject dangerous flags combinations that
disables required barriers
- Trim unused code from Shenandoah's SA impl
- Move ShenandoahGCTracer to gc/shenandoah
- Fix ordering of GC names in various files
- Rename UINT64_FORMAT_HEX_W to UINT64_FORMAT_X_W
http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/04/
Thanks everybody for taking time to review this!
Roman

Post by Roman Kennke
Hello all,
Thanks so far for all the reviews and support!
I forgot to update the 'round' yesterday. We are in round 3 now :-)
Also, I fixed the numbering of my webrevs to match the review-round. ;-)
- We moved shenandoah-specific code out of .ad files into our own .ad
files under gc/shenandoah (in shenandoah-gc), how cool is that? This
requires an addition in build machinery though, see
make/hotspot/gensrc/GensrcAdlc.gmk (in shared-build).
- Improved zero-disabling and build-code-simplification as suggested by
Magnus and Per
- Cleaned up some leftovers in C2
- Improved C2 loop opts code by introducing another APIs in
BarrierSetC2. See the new APIs in shared-gc under BarrierSetC2.hpp.
- I don't see where it makes sense to put INCLUDE_SHENANDOAHGC guards now.
- We would all very much prefer to keep ShenandoahXYZNode names, as
noted earlier. This stuff is Shenandoah-specific, so let's just call it
that.
- Rehashed Shenandoah tests (formatting, naming, directory layout, etc)
- Rebased on jdk-12+22
- Question: let us know if you need separate RFE for the new BSC2 APIs?
http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/03/
Thanks,
Roman

Post by Roman Kennke
- The minor issues that Kim reported in shared-gc
- A lot of fixes in shared-tests according to Leonid's review
- Disabled SA heapdumping similar to ZGC as Per suggested
Leonid:  test/hotspot/jtreg/gc/TestFullGCCount.java was actually
ExplicitGCInvokesConcurrent at the same time, because that would be
(expectedly) failing. It can run CMS, default GC and any other GC just
fine. Adding the same clause for Shenandoah means the same, and filters
the combination (+UseShenandoahGC)+(+ExplicitGCInvokesConcurrent). I
made the condition a bit clearer by avoiding triple-negation.
http://mail.openjdk.java.net/pipermail/shenandoah-dev/2018-November/008457.html
- test/hotspot/jtreg/serviceability/sa/ClhsdbJhisto.java
- test/hotspot/jtreg/serviceability/sa/TestHeapDumpForLargeArray.java
we filter them for Shenandoah now. I'm wondering: how do you get past
those with ZGC?
http://mail.openjdk.java.net/pipermail/shenandoah-dev/2018-November/008466.html
(Note to Leonid and tests reviewers: I'll add those related filters in
next round).
Vladimir: Roland integrated a bunch of changes to make loop* code look
better. I can tell that we're not done with C2 yet. Can you look over
the code and see what is ok, and especially what is not ok, so that we
can focus our efforts on the relevant parts?
http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/01/
Thanks,
Roman

Post by Roman Kennke
Hi,
This is the first round of changes for including Shenandoah GC into
mainline.
I divided the review into parts that roughly correspond to the mailing lists
that would normally review it, and I divided it into 'shared' code
changes and
'shenandoah' code changes (actually, mostly additions). The intend is to
eventually
push them as single 'combined' changeset, once reviewed.
   https://openjdk.java.net/jeps/189
  https://bugs.openjdk.java.net/browse/JDK-8214259
   http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/
For those who want to see the full change, have a look at the
shenandoah-complete
<http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shenandoah-complete/>
directory,
it contains the full combined webrev. Alternatively, there is the file
shenandoah-master.patch
<http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shenandoah-master.patch>,
which is what I intend to commit (and which should be equivalent to the
'shenandoah-complete' webrev).
  *) shenandoah-gc
<http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shenandoah-gc/>
     - Actual Shenandoah implementation, almost completely residing in
gc/shenandoah
  *) shared-gc
<http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shared-gc/>
     - This is mostly boilerplate that is common to any GC
     - referenceProcessor.cpp has a little change to make one
assert not
fail (next to CMS and G1)
     - taskqueue.hpp has some small adjustments to enable subclassing
  *) shared-serviceability
<http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shared-serviceability/>
     - The usual code to support another GC
  *) shared-runtime
<http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shared-runtime/>
     - A number of friends declarations to allow Shenandoah
iterators to
hook up with,
       e.g. ClassLoaderData, CodeCache, etc
     - Warning and disabling JFR LeakProfiler
     - fieldDescriptor.hpp added is_stable() accessor, for use in
Shenandoah C2 optimizations
     - Locks initialization in mutexLocker.cpp as usual
     - VM operations defines for Shenandoah's VM ops
     - globalDefinitions.hpp added UINT64_FORMAT_HEX_W for use in
Shenandoah's logging
     - The usual macros in macro.hpp
  *) shared-build
<http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shared-build/>
     - Add shenandoah feature, enabled by default, as agreed with
Vladimir K. beforehand
     - Some flags for shenandoah-enabled compilation to get
SUPPORT_BARRIER_ON_PRIMITIVES
       and SUPPORT_NOT_TO_SPACE_INVARIANT which is required for
Shenandoah's barriers
     - --param inline-unit-growth=1000 settings for 2 shenandoah
source
files, which is
       useful to get the whole marking loop inlined (observed
significant
regression if we
       don't)
  *) shared-tests
<http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shared-tests/>
     - Test infrastructure to support Shenandoah
     - Shenandoah test groups
     - Exclude Shenandoah in various tests that can be run with
selected GC
     - Enable/add configure for Shenandoah for tests that make
sense to
run with it
  *) shenandoah-tests
<http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shenandoah-tests/>
     - Shenandoah specific tests, most reside in gc/shenandoah
subdirectory
     - A couple of tests configurations have been added, e.g.
TestGCBasherWithShenandoah.java
I intentionally left out shared-compiler for now, because we have some
work left to do
there, but if you click around you'll find the patch anyway, in case you
want to take
a peek at it.
   - {Linux} x {x86_64, x86_32, armhf, aarch64, ppc64el, s390x}
   - {Windows} x {x86_64},
   - {MacOS X} x {x86_64}
   - the new Shenandoah tests
   - jcstress with/without aggressive Shenandoah verification
   - specjvm2008 with/without aggressive Shenandoah verification
I'd like to thank my collegues at Red Hat: Christine Flood, she deserves
the credit for being the original inventor of Shenandoah, Aleksey
Shiplëv, Roland Westrelin & Zhengyu Gu for their countless
contributions, everybody else in Red Hat's OpenJDK team for testing,
advice and support, my collegues in Oracle's GC, runtime and compiler
teams for tirelessly helping with and reviewing all the GC
interface and
related changes, and of course the many early adopters for reporting
bugs and success stories and feature requests: we wouldn't be here
without any of you!
Best regards,
Roman

[Roman Kennke]

Hello Vladimir,

Post by Vladimir Kozlov
Why you need to include shenandoahBarrierSetC2.hpp in /opto/classes.cpp
? Why not include only shenandoahSupport.hpp when new nodes are defined?

I discussed this with my team. shenandoahBarrierSetC2.hpp is supposed to
be the entry point for external C2 code. No C2 code is supposed to
include shenandoahSupport.hpp, this is just an implementation detail of
shenandoahBarrierSetC2.hpp. This seems symmetrical to how ZGC includes
zBarrierSetC2.hpp from external C2 code. Is that ok to leave as it is?

Cheers,
Roman

Post by Vladimir Kozlov
I think it is fine to not use #ifdef in loopopts.cpp when you check
is_ShenandoahBarrier(). And you don't do that in other files.
Code in opto/macro.cpp is ugly but it is only the place so we can live with it I think.
Thanks,
Vladimir

Post by Roman Kennke
Hi all,
- Verify and gracefully reject dangerous flags combinations that
disables required barriers
- Trim unused code from Shenandoah's SA impl
- Move ShenandoahGCTracer to gc/shenandoah
- Fix ordering of GC names in various files
- Rename UINT64_FORMAT_HEX_W to UINT64_FORMAT_X_W
http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/04/
Thanks everybody for taking time to review this!
Roman

Post by Roman Kennke
Hello all,
Thanks so far for all the reviews and support!
I forgot to update the 'round' yesterday. We are in round 3 now :-)
Also, I fixed the numbering of my webrevs to match the review-round. ;-)
- We moved shenandoah-specific code out of .ad files into our own .ad
files under gc/shenandoah (in shenandoah-gc), how cool is that? This
requires an addition in build machinery though, see
make/hotspot/gensrc/GensrcAdlc.gmk (in shared-build).
- Improved zero-disabling and build-code-simplification as suggested by
Magnus and Per
- Cleaned up some leftovers in C2
- Improved C2 loop opts code by introducing another APIs in
BarrierSetC2. See the new APIs in shared-gc under BarrierSetC2.hpp.
- I don't see where it makes sense to put INCLUDE_SHENANDOAHGC guards now.
- We would all very much prefer to keep ShenandoahXYZNode names, as
noted earlier. This stuff is Shenandoah-specific, so let's just call it
that.
- Rehashed Shenandoah tests (formatting, naming, directory layout, etc)
- Rebased on jdk-12+22
- Question: let us know if you need separate RFE for the new BSC2 APIs?
http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/03/
Thanks,
Roman

Post by Roman Kennke
- The minor issues that Kim reported in shared-gc
- A lot of fixes in shared-tests according to Leonid's review
- Disabled SA heapdumping similar to ZGC as Per suggested
Leonid:  test/hotspot/jtreg/gc/TestFullGCCount.java was actually
ExplicitGCInvokesConcurrent at the same time, because that would be
(expectedly) failing. It can run CMS, default GC and any other GC just
fine. Adding the same clause for Shenandoah means the same, and filters
the combination (+UseShenandoahGC)+(+ExplicitGCInvokesConcurrent). I
made the condition a bit clearer by avoiding triple-negation.
http://mail.openjdk.java.net/pipermail/shenandoah-dev/2018-November/008457.html
- test/hotspot/jtreg/serviceability/sa/ClhsdbJhisto.java
- test/hotspot/jtreg/serviceability/sa/TestHeapDumpForLargeArray.java
we filter them for Shenandoah now. I'm wondering: how do you get past
those with ZGC?
http://mail.openjdk.java.net/pipermail/shenandoah-dev/2018-November/008466.html
(Note to Leonid and tests reviewers: I'll add those related filters in
next round).
Vladimir: Roland integrated a bunch of changes to make loop* code look
better. I can tell that we're not done with C2 yet. Can you look over
the code and see what is ok, and especially what is not ok, so that we
can focus our efforts on the relevant parts?
http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/01/
Thanks,
Roman

Post by Roman Kennke
Hi,
This is the first round of changes for including Shenandoah GC into
mainline.
I divided the review into parts that roughly correspond to the mailing lists
that would normally review it, and I divided it into 'shared' code
changes and
'shenandoah' code changes (actually, mostly additions). The intend is to
eventually
push them as single 'combined' changeset, once reviewed.
   https://openjdk.java.net/jeps/189
  https://bugs.openjdk.java.net/browse/JDK-8214259
   http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/
For those who want to see the full change, have a look at the
shenandoah-complete
<http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shenandoah-complete/>
directory,
it contains the full combined webrev. Alternatively, there is the file
shenandoah-master.patch
<http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shenandoah-master.patch>,
which is what I intend to commit (and which should be equivalent to the
'shenandoah-complete' webrev).
  *) shenandoah-gc
<http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shenandoah-gc/>
     - Actual Shenandoah implementation, almost completely residing in
gc/shenandoah
  *) shared-gc
<http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shared-gc/>
     - This is mostly boilerplate that is common to any GC
     - referenceProcessor.cpp has a little change to make one
assert not
fail (next to CMS and G1)
     - taskqueue.hpp has some small adjustments to enable subclassing
  *) shared-serviceability
<http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shared-serviceability/>
     - The usual code to support another GC
  *) shared-runtime
<http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shared-runtime/>
     - A number of friends declarations to allow Shenandoah
iterators to
hook up with,
       e.g. ClassLoaderData, CodeCache, etc
     - Warning and disabling JFR LeakProfiler
     - fieldDescriptor.hpp added is_stable() accessor, for use in
Shenandoah C2 optimizations
     - Locks initialization in mutexLocker.cpp as usual
     - VM operations defines for Shenandoah's VM ops
     - globalDefinitions.hpp added UINT64_FORMAT_HEX_W for use in
Shenandoah's logging
     - The usual macros in macro.hpp
  *) shared-build
<http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shared-build/>
     - Add shenandoah feature, enabled by default, as agreed with
Vladimir K. beforehand
     - Some flags for shenandoah-enabled compilation to get
SUPPORT_BARRIER_ON_PRIMITIVES
       and SUPPORT_NOT_TO_SPACE_INVARIANT which is required for
Shenandoah's barriers
     - --param inline-unit-growth=1000 settings for 2 shenandoah
source
files, which is
       useful to get the whole marking loop inlined (observed
significant
regression if we
       don't)
  *) shared-tests
<http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shared-tests/>
     - Test infrastructure to support Shenandoah
     - Shenandoah test groups
     - Exclude Shenandoah in various tests that can be run with
selected GC
     - Enable/add configure for Shenandoah for tests that make
sense to
run with it
  *) shenandoah-tests
<http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shenandoah-tests/>
     - Shenandoah specific tests, most reside in gc/shenandoah
subdirectory
     - A couple of tests configurations have been added, e.g.
TestGCBasherWithShenandoah.java
I intentionally left out shared-compiler for now, because we have some
work left to do
there, but if you click around you'll find the patch anyway, in case you
want to take
a peek at it.
   - {Linux} x {x86_64, x86_32, armhf, aarch64, ppc64el, s390x}
   - {Windows} x {x86_64},
   - {MacOS X} x {x86_64}
   - the new Shenandoah tests
   - jcstress with/without aggressive Shenandoah verification
   - specjvm2008 with/without aggressive Shenandoah verification
I'd like to thank my collegues at Red Hat: Christine Flood, she deserves
the credit for being the original inventor of Shenandoah, Aleksey
Shiplëv, Roland Westrelin & Zhengyu Gu for their countless
contributions, everybody else in Red Hat's OpenJDK team for testing,
advice and support, my collegues in Oracle's GC, runtime and compiler
teams for tirelessly helping with and reviewing all the GC
interface and
related changes, and of course the many early adopters for reporting
bugs and success stories and feature requests: we wouldn't be here
without any of you!
Best regards,
Roman

[Vladimir Kozlov]

Post by Roman Kennke
Hello Vladimir,

Post by Vladimir Kozlov
Why you need to include shenandoahBarrierSetC2.hpp in /opto/classes.cpp
? Why not include only shenandoahSupport.hpp when new nodes are defined?

I discussed this with my team. shenandoahBarrierSetC2.hpp is supposed to
be the entry point for external C2 code. No C2 code is supposed to
include shenandoahSupport.hpp, this is just an implementation detail of
shenandoahBarrierSetC2.hpp. This seems symmetrical to how ZGC includes
zBarrierSetC2.hpp from external C2 code. Is that ok to leave as it is?

Yes, it is fine.

Vladimir

Post by Roman Kennke
Cheers,
Roman

Post by Vladimir Kozlov
I think it is fine to not use #ifdef in loopopts.cpp when you check
is_ShenandoahBarrier(). And you don't do that in other files.
Code in opto/macro.cpp is ugly but it is only the place so we can live with it I think.
Thanks,
Vladimir

Post by Roman Kennke
Hi all,
- Verify and gracefully reject dangerous flags combinations that
disables required barriers
- Trim unused code from Shenandoah's SA impl
- Move ShenandoahGCTracer to gc/shenandoah
- Fix ordering of GC names in various files
- Rename UINT64_FORMAT_HEX_W to UINT64_FORMAT_X_W
http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/04/
Thanks everybody for taking time to review this!
Roman

Post by Roman Kennke
Hello all,
Thanks so far for all the reviews and support!
I forgot to update the 'round' yesterday. We are in round 3 now :-)
Also, I fixed the numbering of my webrevs to match the review-round. ;-)
- We moved shenandoah-specific code out of .ad files into our own .ad
files under gc/shenandoah (in shenandoah-gc), how cool is that? This
requires an addition in build machinery though, see
make/hotspot/gensrc/GensrcAdlc.gmk (in shared-build).
- Improved zero-disabling and build-code-simplification as suggested by
Magnus and Per
- Cleaned up some leftovers in C2
- Improved C2 loop opts code by introducing another APIs in
BarrierSetC2. See the new APIs in shared-gc under BarrierSetC2.hpp.
- I don't see where it makes sense to put INCLUDE_SHENANDOAHGC guards now.
- We would all very much prefer to keep ShenandoahXYZNode names, as
noted earlier. This stuff is Shenandoah-specific, so let's just call it
that.
- Rehashed Shenandoah tests (formatting, naming, directory layout, etc)
- Rebased on jdk-12+22
- Question: let us know if you need separate RFE for the new BSC2 APIs?
http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/03/
Thanks,
Roman

Post by Roman Kennke
- The minor issues that Kim reported in shared-gc
- A lot of fixes in shared-tests according to Leonid's review
- Disabled SA heapdumping similar to ZGC as Per suggested
Leonid:  test/hotspot/jtreg/gc/TestFullGCCount.java was actually
ExplicitGCInvokesConcurrent at the same time, because that would be
(expectedly) failing. It can run CMS, default GC and any other GC just
fine. Adding the same clause for Shenandoah means the same, and filters
the combination (+UseShenandoahGC)+(+ExplicitGCInvokesConcurrent). I
made the condition a bit clearer by avoiding triple-negation.
http://mail.openjdk.java.net/pipermail/shenandoah-dev/2018-November/008457.html
- test/hotspot/jtreg/serviceability/sa/ClhsdbJhisto.java
- test/hotspot/jtreg/serviceability/sa/TestHeapDumpForLargeArray.java
we filter them for Shenandoah now. I'm wondering: how do you get past
those with ZGC?
http://mail.openjdk.java.net/pipermail/shenandoah-dev/2018-November/008466.html
(Note to Leonid and tests reviewers: I'll add those related filters in
next round).
Vladimir: Roland integrated a bunch of changes to make loop* code look
better. I can tell that we're not done with C2 yet. Can you look over
the code and see what is ok, and especially what is not ok, so that we
can focus our efforts on the relevant parts?
http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/01/
Thanks,
Roman

Post by Roman Kennke
Hi,
This is the first round of changes for including Shenandoah GC into
mainline.
I divided the review into parts that roughly correspond to the mailing lists
that would normally review it, and I divided it into 'shared' code
changes and
'shenandoah' code changes (actually, mostly additions). The intend is to
eventually
push them as single 'combined' changeset, once reviewed.
   https://openjdk.java.net/jeps/189
  https://bugs.openjdk.java.net/browse/JDK-8214259
   http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/
For those who want to see the full change, have a look at the
shenandoah-complete
<http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shenandoah-complete/>
directory,
it contains the full combined webrev. Alternatively, there is the file
shenandoah-master.patch
<http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shenandoah-master.patch>,
which is what I intend to commit (and which should be equivalent to the
'shenandoah-complete' webrev).
  *) shenandoah-gc
<http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shenandoah-gc/>
     - Actual Shenandoah implementation, almost completely residing in
gc/shenandoah
  *) shared-gc
<http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shared-gc/>
     - This is mostly boilerplate that is common to any GC
     - referenceProcessor.cpp has a little change to make one
assert not
fail (next to CMS and G1)
     - taskqueue.hpp has some small adjustments to enable subclassing
  *) shared-serviceability
<http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shared-serviceability/>
     - The usual code to support another GC
  *) shared-runtime
<http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shared-runtime/>
     - A number of friends declarations to allow Shenandoah
iterators to
hook up with,
       e.g. ClassLoaderData, CodeCache, etc
     - Warning and disabling JFR LeakProfiler
     - fieldDescriptor.hpp added is_stable() accessor, for use in
Shenandoah C2 optimizations
     - Locks initialization in mutexLocker.cpp as usual
     - VM operations defines for Shenandoah's VM ops
     - globalDefinitions.hpp added UINT64_FORMAT_HEX_W for use in
Shenandoah's logging
     - The usual macros in macro.hpp
  *) shared-build
<http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shared-build/>
     - Add shenandoah feature, enabled by default, as agreed with
Vladimir K. beforehand
     - Some flags for shenandoah-enabled compilation to get
SUPPORT_BARRIER_ON_PRIMITIVES
       and SUPPORT_NOT_TO_SPACE_INVARIANT which is required for
Shenandoah's barriers
     - --param inline-unit-growth=1000 settings for 2 shenandoah
source
files, which is
       useful to get the whole marking loop inlined (observed
significant
regression if we
       don't)
  *) shared-tests
<http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shared-tests/>
     - Test infrastructure to support Shenandoah
     - Shenandoah test groups
     - Exclude Shenandoah in various tests that can be run with
selected GC
     - Enable/add configure for Shenandoah for tests that make
sense to
run with it
  *) shenandoah-tests
<http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shenandoah-tests/>
     - Shenandoah specific tests, most reside in gc/shenandoah
subdirectory
     - A couple of tests configurations have been added, e.g.
TestGCBasherWithShenandoah.java
I intentionally left out shared-compiler for now, because we have some
work left to do
there, but if you click around you'll find the patch anyway, in case you
want to take
a peek at it.
   - {Linux} x {x86_64, x86_32, armhf, aarch64, ppc64el, s390x}
   - {Windows} x {x86_64},
   - {MacOS X} x {x86_64}
   - the new Shenandoah tests
   - jcstress with/without aggressive Shenandoah verification
   - specjvm2008 with/without aggressive Shenandoah verification
I'd like to thank my collegues at Red Hat: Christine Flood, she deserves
the credit for being the original inventor of Shenandoah, Aleksey
Shiplëv, Roland Westrelin & Zhengyu Gu for their countless
contributions, everybody else in Red Hat's OpenJDK team for testing,
advice and support, my collegues in Oracle's GC, runtime and compiler
teams for tirelessly helping with and reviewing all the GC interface and
related changes, and of course the many early adopters for reporting
bugs and success stories and feature requests: we wouldn't be here
without any of you!
Best regards,
Roman

[Roman Kennke]

Post by Vladimir Kozlov

Post by Roman Kennke
Hello Vladimir,

Post by Vladimir Kozlov
Why you need to include shenandoahBarrierSetC2.hpp in /opto/classes.cpp
? Why not include only shenandoahSupport.hpp when new nodes are defined?

I discussed this with my team. shenandoahBarrierSetC2.hpp is supposed to
be the entry point for external C2 code. No C2 code is supposed to
include shenandoahSupport.hpp, this is just an implementation detail of
shenandoahBarrierSetC2.hpp. This seems symmetrical to how ZGC includes
zBarrierSetC2.hpp from external C2 code. Is that ok to leave as it is?

Yes, it is fine.

Ok great! Thanks!

Can I consider the shared-compiler part reviewed by you then?

Roman

Post by Vladimir Kozlov
Vladimir

Post by Roman Kennke
Cheers,
Roman

Post by Vladimir Kozlov
I think it is fine to not use #ifdef in loopopts.cpp when you check
is_ShenandoahBarrier(). And you don't do that in other files.
Code in opto/macro.cpp is ugly but it is only the place so we can live with it I think.
Thanks,
Vladimir

Post by Roman Kennke
Hi all,
- Verify and gracefully reject dangerous flags combinations that
disables required barriers
- Trim unused code from Shenandoah's SA impl
- Move ShenandoahGCTracer to gc/shenandoah
- Fix ordering of GC names in various files
- Rename UINT64_FORMAT_HEX_W to UINT64_FORMAT_X_W
http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/04/
Thanks everybody for taking time to review this!
Roman

Post by Roman Kennke
Hello all,
Thanks so far for all the reviews and support!
I forgot to update the 'round' yesterday. We are in round 3 now :-)
Also, I fixed the numbering of my webrevs to match the
review-round. ;-)
- We moved shenandoah-specific code out of .ad files into our own .ad
files under gc/shenandoah (in shenandoah-gc), how cool is that? This
requires an addition in build machinery though, see
make/hotspot/gensrc/GensrcAdlc.gmk (in shared-build).
- Improved zero-disabling and build-code-simplification as
suggested by
Magnus and Per
- Cleaned up some leftovers in C2
- Improved C2 loop opts code by introducing another APIs in
BarrierSetC2. See the new APIs in shared-gc under BarrierSetC2.hpp.
- I don't see where it makes sense to put INCLUDE_SHENANDOAHGC guards now.
- We would all very much prefer to keep ShenandoahXYZNode names, as
noted earlier. This stuff is Shenandoah-specific, so let's just call it
that.
- Rehashed Shenandoah tests (formatting, naming, directory layout, etc)
- Rebased on jdk-12+22
- Question: let us know if you need separate RFE for the new BSC2 APIs?
http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/03/
Thanks,
Roman

Post by Roman Kennke
- The minor issues that Kim reported in shared-gc
- A lot of fixes in shared-tests according to Leonid's review
- Disabled SA heapdumping similar to ZGC as Per suggested
Leonid:  test/hotspot/jtreg/gc/TestFullGCCount.java was actually
ExplicitGCInvokesConcurrent at the same time, because that would be
(expectedly) failing. It can run CMS, default GC and any other GC just
fine. Adding the same clause for Shenandoah means the same, and filters
the combination (+UseShenandoahGC)+(+ExplicitGCInvokesConcurrent). I
made the condition a bit clearer by avoiding triple-negation.
http://mail.openjdk.java.net/pipermail/shenandoah-dev/2018-November/008457.html
- test/hotspot/jtreg/serviceability/sa/ClhsdbJhisto.java
- test/hotspot/jtreg/serviceability/sa/TestHeapDumpForLargeArray.java
we filter them for Shenandoah now. I'm wondering: how do you get past
those with ZGC?
http://mail.openjdk.java.net/pipermail/shenandoah-dev/2018-November/008466.html
(Note to Leonid and tests reviewers: I'll add those related filters in
next round).
Vladimir: Roland integrated a bunch of changes to make loop* code look
better. I can tell that we're not done with C2 yet. Can you look over
the code and see what is ok, and especially what is not ok, so that we
can focus our efforts on the relevant parts?
http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/01/
Thanks,
Roman

Post by Roman Kennke
Hi,
This is the first round of changes for including Shenandoah GC into
mainline.
I divided the review into parts that roughly correspond to the mailing lists
that would normally review it, and I divided it into 'shared' code
changes and
'shenandoah' code changes (actually, mostly additions). The intend is to
eventually
push them as single 'combined' changeset, once reviewed.
    https://openjdk.java.net/jeps/189
   https://bugs.openjdk.java.net/browse/JDK-8214259
    http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/
For those who want to see the full change, have a look at the
shenandoah-complete
<http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shenandoah-complete/>
directory,
it contains the full combined webrev. Alternatively, there is the file
shenandoah-master.patch
<http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shenandoah-master.patch>,
which is what I intend to commit (and which should be equivalent to the
'shenandoah-complete' webrev).
   *) shenandoah-gc
<http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shenandoah-gc/>
      - Actual Shenandoah implementation, almost completely
residing in
gc/shenandoah
   *) shared-gc
<http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shared-gc/>
      - This is mostly boilerplate that is common to any GC
      - referenceProcessor.cpp has a little change to make one
assert not
fail (next to CMS and G1)
      - taskqueue.hpp has some small adjustments to enable
subclassing
   *) shared-serviceability
<http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shared-serviceability/>
      - The usual code to support another GC
   *) shared-runtime
<http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shared-runtime/>
      - A number of friends declarations to allow Shenandoah
iterators to
hook up with,
        e.g. ClassLoaderData, CodeCache, etc
      - Warning and disabling JFR LeakProfiler
      - fieldDescriptor.hpp added is_stable() accessor, for use in
Shenandoah C2 optimizations
      - Locks initialization in mutexLocker.cpp as usual
      - VM operations defines for Shenandoah's VM ops
      - globalDefinitions.hpp added UINT64_FORMAT_HEX_W for use in
Shenandoah's logging
      - The usual macros in macro.hpp
   *) shared-build
<http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shared-build/>
      - Add shenandoah feature, enabled by default, as agreed with
Vladimir K. beforehand
      - Some flags for shenandoah-enabled compilation to get
SUPPORT_BARRIER_ON_PRIMITIVES
        and SUPPORT_NOT_TO_SPACE_INVARIANT which is required for
Shenandoah's barriers
      - --param inline-unit-growth=1000 settings for 2 shenandoah
source
files, which is
        useful to get the whole marking loop inlined (observed
significant
regression if we
        don't)
   *) shared-tests
<http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shared-tests/>
      - Test infrastructure to support Shenandoah
      - Shenandoah test groups
      - Exclude Shenandoah in various tests that can be run with
selected GC
      - Enable/add configure for Shenandoah for tests that make
sense to
run with it
   *) shenandoah-tests
<http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shenandoah-tests/>
      - Shenandoah specific tests, most reside in gc/shenandoah
subdirectory
      - A couple of tests configurations have been added, e.g.
TestGCBasherWithShenandoah.java
I intentionally left out shared-compiler for now, because we have some
work left to do
there, but if you click around you'll find the patch anyway, in case you
want to take
a peek at it.
    - {Linux} x {x86_64, x86_32, armhf, aarch64, ppc64el, s390x}
    - {Windows} x {x86_64},
    - {MacOS X} x {x86_64}
    - the new Shenandoah tests
    - jcstress with/without aggressive Shenandoah verification
    - specjvm2008 with/without aggressive Shenandoah verification
I'd like to thank my collegues at Red Hat: Christine Flood, she deserves
the credit for being the original inventor of Shenandoah, Aleksey
Shiplëv, Roland Westrelin & Zhengyu Gu for their countless
contributions, everybody else in Red Hat's OpenJDK team for testing,
advice and support, my collegues in Oracle's GC, runtime and compiler
teams for tirelessly helping with and reviewing all the GC interface and
related changes, and of course the many early adopters for reporting
bugs and success stories and feature requests: we wouldn't be here
without any of you!
Best regards,
Roman

[Vladimir Kozlov]

Yes, compiler changes reviewed.

Vladimir

Post by Roman Kennke

Post by Vladimir Kozlov

Post by Roman Kennke
Hello Vladimir,

Post by Vladimir Kozlov
Why you need to include shenandoahBarrierSetC2.hpp in /opto/classes.cpp
? Why not include only shenandoahSupport.hpp when new nodes are defined?

I discussed this with my team. shenandoahBarrierSetC2.hpp is supposed to
be the entry point for external C2 code. No C2 code is supposed to
include shenandoahSupport.hpp, this is just an implementation detail of
shenandoahBarrierSetC2.hpp. This seems symmetrical to how ZGC includes
zBarrierSetC2.hpp from external C2 code. Is that ok to leave as it is?

Yes, it is fine.

Ok great! Thanks!
Can I consider the shared-compiler part reviewed by you then?
Roman

Post by Vladimir Kozlov
Vladimir

Post by Roman Kennke
Cheers,
Roman

Post by Vladimir Kozlov
I think it is fine to not use #ifdef in loopopts.cpp when you check
is_ShenandoahBarrier(). And you don't do that in other files.
Code in opto/macro.cpp is ugly but it is only the place so we can live
with it I think.
Thanks,
Vladimir

Post by Roman Kennke
Hi all,
- Verify and gracefully reject dangerous flags combinations that
disables required barriers
- Trim unused code from Shenandoah's SA impl
- Move ShenandoahGCTracer to gc/shenandoah
- Fix ordering of GC names in various files
- Rename UINT64_FORMAT_HEX_W to UINT64_FORMAT_X_W
http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/04/
Thanks everybody for taking time to review this!
Roman

Post by Roman Kennke
Hello all,
Thanks so far for all the reviews and support!
I forgot to update the 'round' yesterday. We are in round 3 now :-)
Also, I fixed the numbering of my webrevs to match the
review-round. ;-)
- We moved shenandoah-specific code out of .ad files into our own .ad
files under gc/shenandoah (in shenandoah-gc), how cool is that? This
requires an addition in build machinery though, see
make/hotspot/gensrc/GensrcAdlc.gmk (in shared-build).
- Improved zero-disabling and build-code-simplification as suggested by
Magnus and Per
- Cleaned up some leftovers in C2
- Improved C2 loop opts code by introducing another APIs in
BarrierSetC2. See the new APIs in shared-gc under BarrierSetC2.hpp.
- I don't see where it makes sense to put INCLUDE_SHENANDOAHGC guards now.
- We would all very much prefer to keep ShenandoahXYZNode names, as
noted earlier. This stuff is Shenandoah-specific, so let's just call it
that.
- Rehashed Shenandoah tests (formatting, naming, directory layout, etc)
- Rebased on jdk-12+22
- Question: let us know if you need separate RFE for the new BSC2 APIs?
http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/03/
Thanks,
Roman

Post by Roman Kennke
- The minor issues that Kim reported in shared-gc
- A lot of fixes in shared-tests according to Leonid's review
- Disabled SA heapdumping similar to ZGC as Per suggested
Leonid:  test/hotspot/jtreg/gc/TestFullGCCount.java was actually
ExplicitGCInvokesConcurrent at the same time, because that would be
(expectedly) failing. It can run CMS, default GC and any other GC just
fine. Adding the same clause for Shenandoah means the same, and filters
the combination (+UseShenandoahGC)+(+ExplicitGCInvokesConcurrent). I
made the condition a bit clearer by avoiding triple-negation.
http://mail.openjdk.java.net/pipermail/shenandoah-dev/2018-November/008457.html
- test/hotspot/jtreg/serviceability/sa/ClhsdbJhisto.java
- test/hotspot/jtreg/serviceability/sa/TestHeapDumpForLargeArray.java
we filter them for Shenandoah now. I'm wondering: how do you get past
those with ZGC?
http://mail.openjdk.java.net/pipermail/shenandoah-dev/2018-November/008466.html
(Note to Leonid and tests reviewers: I'll add those related filters in
next round).
Vladimir: Roland integrated a bunch of changes to make loop* code look
better. I can tell that we're not done with C2 yet. Can you look over
the code and see what is ok, and especially what is not ok, so that we
can focus our efforts on the relevant parts?
http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/01/
Thanks,
Roman

Post by Roman Kennke
Hi,
This is the first round of changes for including Shenandoah GC into
mainline.
I divided the review into parts that roughly correspond to the mailing lists
that would normally review it, and I divided it into 'shared' code
changes and
'shenandoah' code changes (actually, mostly additions). The intend is to
eventually
push them as single 'combined' changeset, once reviewed.
    https://openjdk.java.net/jeps/189
   https://bugs.openjdk.java.net/browse/JDK-8214259
    http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/
For those who want to see the full change, have a look at the
shenandoah-complete
<http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shenandoah-complete/>
directory,
it contains the full combined webrev. Alternatively, there is the file
shenandoah-master.patch
<http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shenandoah-master.patch>,
which is what I intend to commit (and which should be equivalent to the
'shenandoah-complete' webrev).
   *) shenandoah-gc
<http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shenandoah-gc/>
      - Actual Shenandoah implementation, almost completely
residing in
gc/shenandoah
   *) shared-gc
<http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shared-gc/>
      - This is mostly boilerplate that is common to any GC
      - referenceProcessor.cpp has a little change to make one
assert not
fail (next to CMS and G1)
      - taskqueue.hpp has some small adjustments to enable
subclassing
   *) shared-serviceability
<http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shared-serviceability/>
      - The usual code to support another GC
   *) shared-runtime
<http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shared-runtime/>
      - A number of friends declarations to allow Shenandoah
iterators to
hook up with,
        e.g. ClassLoaderData, CodeCache, etc
      - Warning and disabling JFR LeakProfiler
      - fieldDescriptor.hpp added is_stable() accessor, for use in
Shenandoah C2 optimizations
      - Locks initialization in mutexLocker.cpp as usual
      - VM operations defines for Shenandoah's VM ops
      - globalDefinitions.hpp added UINT64_FORMAT_HEX_W for use in
Shenandoah's logging
      - The usual macros in macro.hpp
   *) shared-build
<http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shared-build/>
      - Add shenandoah feature, enabled by default, as agreed with
Vladimir K. beforehand
      - Some flags for shenandoah-enabled compilation to get
SUPPORT_BARRIER_ON_PRIMITIVES
        and SUPPORT_NOT_TO_SPACE_INVARIANT which is required for
Shenandoah's barriers
      - --param inline-unit-growth=1000 settings for 2 shenandoah
source
files, which is
        useful to get the whole marking loop inlined (observed
significant
regression if we
        don't)
   *) shared-tests
<http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shared-tests/>
      - Test infrastructure to support Shenandoah
      - Shenandoah test groups
      - Exclude Shenandoah in various tests that can be run with
selected GC
      - Enable/add configure for Shenandoah for tests that make
sense to
run with it
   *) shenandoah-tests
<http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shenandoah-tests/>
      - Shenandoah specific tests, most reside in gc/shenandoah
subdirectory
      - A couple of tests configurations have been added, e.g.
TestGCBasherWithShenandoah.java
I intentionally left out shared-compiler for now, because we have some
work left to do
there, but if you click around you'll find the patch anyway, in case you
want to take
a peek at it.
    - {Linux} x {x86_64, x86_32, armhf, aarch64, ppc64el, s390x}
    - {Windows} x {x86_64},
    - {MacOS X} x {x86_64}
    - the new Shenandoah tests
    - jcstress with/without aggressive Shenandoah verification
    - specjvm2008 with/without aggressive Shenandoah verification
I'd like to thank my collegues at Red Hat: Christine Flood, she deserves
the credit for being the original inventor of Shenandoah, Aleksey
Shiplëv, Roland Westrelin & Zhengyu Gu for their countless
contributions, everybody else in Red Hat's OpenJDK team for testing,
advice and support, my collegues in Oracle's GC, runtime and compiler
teams for tirelessly helping with and reviewing all the GC interface and
related changes, and of course the many early adopters for reporting
bugs and success stories and feature requests: we wouldn't be here
without any of you!
Best regards,
Roman

[Jini George]

Hi Roman,

A few comments on the SA changes:

==> Could you please add the following lines in
src/jdk.hotspot.agent/share/classes/sun/jvm/hotspot/HSDB.java from line
1120 onwards to avoid the "[Unknown generation]" message with hsdb while
displaying the Stack Memory for a mutator thread ?

else if (collHeap instanceof ShenandoahHeap) {
ShenandoahHeap heap = (ShenandoahHeap) collHeap;
anno = "ShenandoahHeap ";
bad = false;
}

==>
src/jdk.hotspot.agent/share/classes/sun/jvm/hotspot/tools/HeapSummary.java

The printGCAlgorithm() method would need to be updated to read in the
UseShenandoahGC flag to avoid the default "Mark Sweep Compact GC" being
displayed with jhsdb jmap -heap.

==>
src/jdk.hotspot.agent/share/classes/sun/jvm/hotspot/gc/shared/GCName.java

Could you please add "Shenandoah" to the GCName enum list ?

==>
src/jdk.hotspot.agent/share/classes/sun/jvm/hotspot/gc/shared/GCCause.java

Could you please update the GCCause enum values to include these:

_shenandoah_stop_vm,
_shenandoah_allocation_failure_evac,
_shenandoah_concurrent_gc,
_shenandoah_traversal_gc,
_shenandoah_upgrade_to_full_gc,

==> share/classes/sun/jvm/hotspot/runtime/VMOps.java

It would be good to add 'ShenandoahOperation' to the VMOps enum (though
it is probably not in sync now).

Thank you,
Jini.

Post by Roman Kennke
Hi all,
- Verify and gracefully reject dangerous flags combinations that
disables required barriers
- Trim unused code from Shenandoah's SA impl
- Move ShenandoahGCTracer to gc/shenandoah
- Fix ordering of GC names in various files
- Rename UINT64_FORMAT_HEX_W to UINT64_FORMAT_X_W
http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/04/
Thanks everybody for taking time to review this!
Roman

Post by Roman Kennke
Hello all,
Thanks so far for all the reviews and support!
I forgot to update the 'round' yesterday. We are in round 3 now :-)
Also, I fixed the numbering of my webrevs to match the review-round. ;-)
- We moved shenandoah-specific code out of .ad files into our own .ad
files under gc/shenandoah (in shenandoah-gc), how cool is that? This
requires an addition in build machinery though, see
make/hotspot/gensrc/GensrcAdlc.gmk (in shared-build).
- Improved zero-disabling and build-code-simplification as suggested by
Magnus and Per
- Cleaned up some leftovers in C2
- Improved C2 loop opts code by introducing another APIs in
BarrierSetC2. See the new APIs in shared-gc under BarrierSetC2.hpp.
- I don't see where it makes sense to put INCLUDE_SHENANDOAHGC guards now.
- We would all very much prefer to keep ShenandoahXYZNode names, as
noted earlier. This stuff is Shenandoah-specific, so let's just call it
that.
- Rehashed Shenandoah tests (formatting, naming, directory layout, etc)
- Rebased on jdk-12+22
- Question: let us know if you need separate RFE for the new BSC2 APIs?
http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/03/
Thanks,
Roman

Post by Roman Kennke
- The minor issues that Kim reported in shared-gc
- A lot of fixes in shared-tests according to Leonid's review
- Disabled SA heapdumping similar to ZGC as Per suggested
Leonid: test/hotspot/jtreg/gc/TestFullGCCount.java was actually
ExplicitGCInvokesConcurrent at the same time, because that would be
(expectedly) failing. It can run CMS, default GC and any other GC just
fine. Adding the same clause for Shenandoah means the same, and filters
the combination (+UseShenandoahGC)+(+ExplicitGCInvokesConcurrent). I
made the condition a bit clearer by avoiding triple-negation.
http://mail.openjdk.java.net/pipermail/shenandoah-dev/2018-November/008457.html
- test/hotspot/jtreg/serviceability/sa/ClhsdbJhisto.java
- test/hotspot/jtreg/serviceability/sa/TestHeapDumpForLargeArray.java
we filter them for Shenandoah now. I'm wondering: how do you get past
those with ZGC?
http://mail.openjdk.java.net/pipermail/shenandoah-dev/2018-November/008466.html
(Note to Leonid and tests reviewers: I'll add those related filters in
next round).
Vladimir: Roland integrated a bunch of changes to make loop* code look
better. I can tell that we're not done with C2 yet. Can you look over
the code and see what is ok, and especially what is not ok, so that we
can focus our efforts on the relevant parts?
http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/01/
Thanks,
Roman

Post by Roman Kennke
Hi,
This is the first round of changes for including Shenandoah GC into
mainline.
I divided the review into parts that roughly correspond to the mailing lists
that would normally review it, and I divided it into 'shared' code
changes and
'shenandoah' code changes (actually, mostly additions). The intend is to
eventually
push them as single 'combined' changeset, once reviewed.
  https://openjdk.java.net/jeps/189
 https://bugs.openjdk.java.net/browse/JDK-8214259
  http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/
For those who want to see the full change, have a look at the
shenandoah-complete
<http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shenandoah-complete/>
directory,
it contains the full combined webrev. Alternatively, there is the file
shenandoah-master.patch
<http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shenandoah-master.patch>,
which is what I intend to commit (and which should be equivalent to the
'shenandoah-complete' webrev).
 *) shenandoah-gc
<http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shenandoah-gc/>
    - Actual Shenandoah implementation, almost completely residing in
gc/shenandoah
 *) shared-gc
<http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shared-gc/>
    - This is mostly boilerplate that is common to any GC
    - referenceProcessor.cpp has a little change to make one assert not
fail (next to CMS and G1)
    - taskqueue.hpp has some small adjustments to enable subclassing
 *) shared-serviceability
<http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shared-serviceability/>
    - The usual code to support another GC
 *) shared-runtime
<http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shared-runtime/>
    - A number of friends declarations to allow Shenandoah iterators to
hook up with,
      e.g. ClassLoaderData, CodeCache, etc
    - Warning and disabling JFR LeakProfiler
    - fieldDescriptor.hpp added is_stable() accessor, for use in
Shenandoah C2 optimizations
    - Locks initialization in mutexLocker.cpp as usual
    - VM operations defines for Shenandoah's VM ops
    - globalDefinitions.hpp added UINT64_FORMAT_HEX_W for use in
Shenandoah's logging
    - The usual macros in macro.hpp
 *) shared-build
<http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shared-build/>
    - Add shenandoah feature, enabled by default, as agreed with
Vladimir K. beforehand
    - Some flags for shenandoah-enabled compilation to get
SUPPORT_BARRIER_ON_PRIMITIVES
      and SUPPORT_NOT_TO_SPACE_INVARIANT which is required for
Shenandoah's barriers
    - --param inline-unit-growth=1000 settings for 2 shenandoah source
files, which is
      useful to get the whole marking loop inlined (observed significant
regression if we
      don't)
 *) shared-tests
<http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shared-tests/>
    - Test infrastructure to support Shenandoah
    - Shenandoah test groups
    - Exclude Shenandoah in various tests that can be run with selected GC
    - Enable/add configure for Shenandoah for tests that make sense to
run with it
 *) shenandoah-tests
<http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shenandoah-tests/>
    - Shenandoah specific tests, most reside in gc/shenandoah subdirectory
    - A couple of tests configurations have been added, e.g.
TestGCBasherWithShenandoah.java
I intentionally left out shared-compiler for now, because we have some
work left to do
there, but if you click around you'll find the patch anyway, in case you
want to take
a peek at it.
  - {Linux} x {x86_64, x86_32, armhf, aarch64, ppc64el, s390x}
  - {Windows} x {x86_64},
  - {MacOS X} x {x86_64}
  - the new Shenandoah tests
  - jcstress with/without aggressive Shenandoah verification
  - specjvm2008 with/without aggressive Shenandoah verification
I'd like to thank my collegues at Red Hat: Christine Flood, she deserves
the credit for being the original inventor of Shenandoah, Aleksey
Shiplëv, Roland Westrelin & Zhengyu Gu for their countless
contributions, everybody else in Red Hat's OpenJDK team for testing,
advice and support, my collegues in Oracle's GC, runtime and compiler
teams for tirelessly helping with and reviewing all the GC interface and
related changes, and of course the many early adopters for reporting
bugs and success stories and feature requests: we wouldn't be here
without any of you!
Best regards,
Roman

[Roman Kennke]

Hi Jini,

Thanks for your suggestions. I've added this to Shenandoah's dev:

http://cr.openjdk.java.net/~rkennke/shenandoah-sa/webrev.01/

and it will show up in next round of webrevs.

Thanks,
Roman

Post by Jini George
==> Could you please add the following lines in
src/jdk.hotspot.agent/share/classes/sun/jvm/hotspot/HSDB.java from line
1120 onwards to avoid the "[Unknown generation]" message with hsdb while
displaying the Stack Memory for a mutator thread ?
else if (collHeap instanceof ShenandoahHeap) {
   ShenandoahHeap heap = (ShenandoahHeap) collHeap;
   anno = "ShenandoahHeap ";
   bad = false;
}
==>
src/jdk.hotspot.agent/share/classes/sun/jvm/hotspot/tools/HeapSummary.java
The printGCAlgorithm() method would need to be updated to read in the
UseShenandoahGC flag to avoid the default "Mark Sweep Compact GC" being
displayed with jhsdb jmap -heap.
==>
src/jdk.hotspot.agent/share/classes/sun/jvm/hotspot/gc/shared/GCName.java
Could you please add "Shenandoah" to the GCName enum list ?
==>
src/jdk.hotspot.agent/share/classes/sun/jvm/hotspot/gc/shared/GCCause.java
    _shenandoah_stop_vm,
    _shenandoah_allocation_failure_evac,
    _shenandoah_concurrent_gc,
    _shenandoah_traversal_gc,
    _shenandoah_upgrade_to_full_gc,
==> share/classes/sun/jvm/hotspot/runtime/VMOps.java
It would be good to add 'ShenandoahOperation' to the VMOps enum (though
it is probably not in sync now).
Thank you,
Jini.

Post by Roman Kennke
Hi all,
- Verify and gracefully reject dangerous flags combinations that
disables required barriers
- Trim unused code from Shenandoah's SA impl
- Move ShenandoahGCTracer to gc/shenandoah
- Fix ordering of GC names in various files
- Rename UINT64_FORMAT_HEX_W to UINT64_FORMAT_X_W
http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/04/
Thanks everybody for taking time to review this!
Roman

Post by Roman Kennke
Hello all,
Thanks so far for all the reviews and support!
I forgot to update the 'round' yesterday. We are in round 3 now :-)
Also, I fixed the numbering of my webrevs to match the review-round. ;-)
- We moved shenandoah-specific code out of .ad files into our own .ad
files under gc/shenandoah (in shenandoah-gc), how cool is that? This
requires an addition in build machinery though, see
make/hotspot/gensrc/GensrcAdlc.gmk (in shared-build).
- Improved zero-disabling and build-code-simplification as suggested by
Magnus and Per
- Cleaned up some leftovers in C2
- Improved C2 loop opts code by introducing another APIs in
BarrierSetC2. See the new APIs in shared-gc under BarrierSetC2.hpp.
- I don't see where it makes sense to put INCLUDE_SHENANDOAHGC guards now.
- We would all very much prefer to keep ShenandoahXYZNode names, as
noted earlier. This stuff is Shenandoah-specific, so let's just call it
that.
- Rehashed Shenandoah tests (formatting, naming, directory layout, etc)
- Rebased on jdk-12+22
- Question: let us know if you need separate RFE for the new BSC2 APIs?
http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/03/
Thanks,
Roman

Post by Roman Kennke
- The minor issues that Kim reported in shared-gc
- A lot of fixes in shared-tests according to Leonid's review
- Disabled SA heapdumping similar to ZGC as Per suggested
Leonid:  test/hotspot/jtreg/gc/TestFullGCCount.java was actually
ExplicitGCInvokesConcurrent at the same time, because that would be
(expectedly) failing. It can run CMS, default GC and any other GC just
fine. Adding the same clause for Shenandoah means the same, and filters
the combination (+UseShenandoahGC)+(+ExplicitGCInvokesConcurrent). I
made the condition a bit clearer by avoiding triple-negation.
http://mail.openjdk.java.net/pipermail/shenandoah-dev/2018-November/008457.html
- test/hotspot/jtreg/serviceability/sa/ClhsdbJhisto.java
- test/hotspot/jtreg/serviceability/sa/TestHeapDumpForLargeArray.java
we filter them for Shenandoah now. I'm wondering: how do you get past
those with ZGC?
http://mail.openjdk.java.net/pipermail/shenandoah-dev/2018-November/008466.html
(Note to Leonid and tests reviewers: I'll add those related filters in
next round).
Vladimir: Roland integrated a bunch of changes to make loop* code look
better. I can tell that we're not done with C2 yet. Can you look over
the code and see what is ok, and especially what is not ok, so that we
can focus our efforts on the relevant parts?
http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/01/
Thanks,
Roman

Post by Roman Kennke
Hi,
This is the first round of changes for including Shenandoah GC into
mainline.
I divided the review into parts that roughly correspond to the mailing lists
that would normally review it, and I divided it into 'shared' code
changes and
'shenandoah' code changes (actually, mostly additions). The intend is to
eventually
push them as single 'combined' changeset, once reviewed.
   https://openjdk.java.net/jeps/189
  https://bugs.openjdk.java.net/browse/JDK-8214259
   http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/
For those who want to see the full change, have a look at the
shenandoah-complete
<http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shenandoah-complete/>
directory,
it contains the full combined webrev. Alternatively, there is the file
shenandoah-master.patch
<http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shenandoah-master.patch>,
which is what I intend to commit (and which should be equivalent to the
'shenandoah-complete' webrev).
  *) shenandoah-gc
<http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shenandoah-gc/>
     - Actual Shenandoah implementation, almost completely residing in
gc/shenandoah
  *) shared-gc
<http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shared-gc/>
     - This is mostly boilerplate that is common to any GC
     - referenceProcessor.cpp has a little change to make one
assert not
fail (next to CMS and G1)
     - taskqueue.hpp has some small adjustments to enable subclassing
  *) shared-serviceability
<http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shared-serviceability/>
     - The usual code to support another GC
  *) shared-runtime
<http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shared-runtime/>
     - A number of friends declarations to allow Shenandoah
iterators to
hook up with,
       e.g. ClassLoaderData, CodeCache, etc
     - Warning and disabling JFR LeakProfiler
     - fieldDescriptor.hpp added is_stable() accessor, for use in
Shenandoah C2 optimizations
     - Locks initialization in mutexLocker.cpp as usual
     - VM operations defines for Shenandoah's VM ops
     - globalDefinitions.hpp added UINT64_FORMAT_HEX_W for use in
Shenandoah's logging
     - The usual macros in macro.hpp
  *) shared-build
<http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shared-build/>
     - Add shenandoah feature, enabled by default, as agreed with
Vladimir K. beforehand
     - Some flags for shenandoah-enabled compilation to get
SUPPORT_BARRIER_ON_PRIMITIVES
       and SUPPORT_NOT_TO_SPACE_INVARIANT which is required for
Shenandoah's barriers
     - --param inline-unit-growth=1000 settings for 2 shenandoah
source
files, which is
       useful to get the whole marking loop inlined (observed
significant
regression if we
       don't)
  *) shared-tests
<http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shared-tests/>
     - Test infrastructure to support Shenandoah
     - Shenandoah test groups
     - Exclude Shenandoah in various tests that can be run with
selected GC
     - Enable/add configure for Shenandoah for tests that make
sense to
run with it
  *) shenandoah-tests
<http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shenandoah-tests/>
     - Shenandoah specific tests, most reside in gc/shenandoah
subdirectory
     - A couple of tests configurations have been added, e.g.
TestGCBasherWithShenandoah.java
I intentionally left out shared-compiler for now, because we have some
work left to do
there, but if you click around you'll find the patch anyway, in case you
want to take
a peek at it.
   - {Linux} x {x86_64, x86_32, armhf, aarch64, ppc64el, s390x}
   - {Windows} x {x86_64},
   - {MacOS X} x {x86_64}
   - the new Shenandoah tests
   - jcstress with/without aggressive Shenandoah verification
   - specjvm2008 with/without aggressive Shenandoah verification
I'd like to thank my collegues at Red Hat: Christine Flood, she deserves
the credit for being the original inventor of Shenandoah, Aleksey
Shiplëv, Roland Westrelin & Zhengyu Gu for their countless
contributions, everybody else in Red Hat's OpenJDK team for testing,
advice and support, my collegues in Oracle's GC, runtime and compiler
teams for tirelessly helping with and reviewing all the GC
interface and
related changes, and of course the many early adopters for reporting
bugs and success stories and feature requests: we wouldn't be here
without any of you!
Best regards,
Roman

[Per Liden]

Hi Roman,

Post by Roman Kennke
Hi all,
- Verify and gracefully reject dangerous flags combinations that
disables required barriers
- Trim unused code from Shenandoah's SA impl
- Move ShenandoahGCTracer to gc/shenandoah
- Fix ordering of GC names in various files
- Rename UINT64_FORMAT_HEX_W to UINT64_FORMAT_X_W

Thanks for fixing. Looks good to me, except it looks like you missed
adjusting the macro order in the following files:
src/hotspot/share/gc/shared/gc_globals.hpp
src/hotspot/share/gc/shared/vmStructs_gc.hpp

cheers,
Per

Post by Roman Kennke
http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/04/
Thanks everybody for taking time to review this!
Roman

Post by Roman Kennke
Hello all,
Thanks so far for all the reviews and support!
I forgot to update the 'round' yesterday. We are in round 3 now :-)
Also, I fixed the numbering of my webrevs to match the review-round. ;-)
- We moved shenandoah-specific code out of .ad files into our own .ad
files under gc/shenandoah (in shenandoah-gc), how cool is that? This
requires an addition in build machinery though, see
make/hotspot/gensrc/GensrcAdlc.gmk (in shared-build).
- Improved zero-disabling and build-code-simplification as suggested by
Magnus and Per
- Cleaned up some leftovers in C2
- Improved C2 loop opts code by introducing another APIs in
BarrierSetC2. See the new APIs in shared-gc under BarrierSetC2.hpp.
- I don't see where it makes sense to put INCLUDE_SHENANDOAHGC guards now.
- We would all very much prefer to keep ShenandoahXYZNode names, as
noted earlier. This stuff is Shenandoah-specific, so let's just call it
that.
- Rehashed Shenandoah tests (formatting, naming, directory layout, etc)
- Rebased on jdk-12+22
- Question: let us know if you need separate RFE for the new BSC2 APIs?
http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/03/
Thanks,
Roman

Post by Roman Kennke
- The minor issues that Kim reported in shared-gc
- A lot of fixes in shared-tests according to Leonid's review
- Disabled SA heapdumping similar to ZGC as Per suggested
Leonid: test/hotspot/jtreg/gc/TestFullGCCount.java was actually
ExplicitGCInvokesConcurrent at the same time, because that would be
(expectedly) failing. It can run CMS, default GC and any other GC just
fine. Adding the same clause for Shenandoah means the same, and filters
the combination (+UseShenandoahGC)+(+ExplicitGCInvokesConcurrent). I
made the condition a bit clearer by avoiding triple-negation.
http://mail.openjdk.java.net/pipermail/shenandoah-dev/2018-November/008457.html
- test/hotspot/jtreg/serviceability/sa/ClhsdbJhisto.java
- test/hotspot/jtreg/serviceability/sa/TestHeapDumpForLargeArray.java
we filter them for Shenandoah now. I'm wondering: how do you get past
those with ZGC?
http://mail.openjdk.java.net/pipermail/shenandoah-dev/2018-November/008466.html
(Note to Leonid and tests reviewers: I'll add those related filters in
next round).
Vladimir: Roland integrated a bunch of changes to make loop* code look
better. I can tell that we're not done with C2 yet. Can you look over
the code and see what is ok, and especially what is not ok, so that we
can focus our efforts on the relevant parts?
http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/01/
Thanks,
Roman

Post by Roman Kennke
Hi,
This is the first round of changes for including Shenandoah GC into
mainline.
I divided the review into parts that roughly correspond to the mailing lists
that would normally review it, and I divided it into 'shared' code
changes and
'shenandoah' code changes (actually, mostly additions). The intend is to
eventually
push them as single 'combined' changeset, once reviewed.
  https://openjdk.java.net/jeps/189
 https://bugs.openjdk.java.net/browse/JDK-8214259
  http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/
For those who want to see the full change, have a look at the
shenandoah-complete
<http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shenandoah-complete/>
directory,
it contains the full combined webrev. Alternatively, there is the file
shenandoah-master.patch
<http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shenandoah-master.patch>,
which is what I intend to commit (and which should be equivalent to the
'shenandoah-complete' webrev).
 *) shenandoah-gc
<http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shenandoah-gc/>
    - Actual Shenandoah implementation, almost completely residing in
gc/shenandoah
 *) shared-gc
<http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shared-gc/>
    - This is mostly boilerplate that is common to any GC
    - referenceProcessor.cpp has a little change to make one assert not
fail (next to CMS and G1)
    - taskqueue.hpp has some small adjustments to enable subclassing
 *) shared-serviceability
<http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shared-serviceability/>
    - The usual code to support another GC
 *) shared-runtime
<http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shared-runtime/>
    - A number of friends declarations to allow Shenandoah iterators to
hook up with,
      e.g. ClassLoaderData, CodeCache, etc
    - Warning and disabling JFR LeakProfiler
    - fieldDescriptor.hpp added is_stable() accessor, for use in
Shenandoah C2 optimizations
    - Locks initialization in mutexLocker.cpp as usual
    - VM operations defines for Shenandoah's VM ops
    - globalDefinitions.hpp added UINT64_FORMAT_HEX_W for use in
Shenandoah's logging
    - The usual macros in macro.hpp
 *) shared-build
<http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shared-build/>
    - Add shenandoah feature, enabled by default, as agreed with
Vladimir K. beforehand
    - Some flags for shenandoah-enabled compilation to get
SUPPORT_BARRIER_ON_PRIMITIVES
      and SUPPORT_NOT_TO_SPACE_INVARIANT which is required for
Shenandoah's barriers
    - --param inline-unit-growth=1000 settings for 2 shenandoah source
files, which is
      useful to get the whole marking loop inlined (observed significant
regression if we
      don't)
 *) shared-tests
<http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shared-tests/>
    - Test infrastructure to support Shenandoah
    - Shenandoah test groups
    - Exclude Shenandoah in various tests that can be run with selected GC
    - Enable/add configure for Shenandoah for tests that make sense to
run with it
 *) shenandoah-tests
<http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shenandoah-tests/>
    - Shenandoah specific tests, most reside in gc/shenandoah subdirectory
    - A couple of tests configurations have been added, e.g.
TestGCBasherWithShenandoah.java
I intentionally left out shared-compiler for now, because we have some
work left to do
there, but if you click around you'll find the patch anyway, in case you
want to take
a peek at it.
  - {Linux} x {x86_64, x86_32, armhf, aarch64, ppc64el, s390x}
  - {Windows} x {x86_64},
  - {MacOS X} x {x86_64}
  - the new Shenandoah tests
  - jcstress with/without aggressive Shenandoah verification
  - specjvm2008 with/without aggressive Shenandoah verification
I'd like to thank my collegues at Red Hat: Christine Flood, she deserves
the credit for being the original inventor of Shenandoah, Aleksey
Shiplëv, Roland Westrelin & Zhengyu Gu for their countless
contributions, everybody else in Red Hat's OpenJDK team for testing,
advice and support, my collegues in Oracle's GC, runtime and compiler
teams for tirelessly helping with and reviewing all the GC interface and
related changes, and of course the many early adopters for reporting
bugs and success stories and feature requests: we wouldn't be here
without any of you!
Best regards,
Roman

[Roman Kennke]

Hi Per,

Thanks for looking again.

I've fixed the ordering in shenandoah-dev:

http://cr.openjdk.java.net/~rkennke/fix-macro-order/webrev.00/

and it will apear in the next round of webrevs.

Thanks,
Roman

Post by Vladimir Kozlov
Hi Roman,

Post by Roman Kennke
Hi all,
- Verify and gracefully reject dangerous flags combinations that
disables required barriers
- Trim unused code from Shenandoah's SA impl
- Move ShenandoahGCTracer to gc/shenandoah
- Fix ordering of GC names in various files
- Rename UINT64_FORMAT_HEX_W to UINT64_FORMAT_X_W

Thanks for fixing. Looks good to me, except it looks like you missed
 src/hotspot/share/gc/shared/gc_globals.hpp
 src/hotspot/share/gc/shared/vmStructs_gc.hpp
cheers,
Per

Post by Roman Kennke
http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/04/
Thanks everybody for taking time to review this!
Roman

Post by Roman Kennke
Hello all,
Thanks so far for all the reviews and support!
I forgot to update the 'round' yesterday. We are in round 3 now :-)
Also, I fixed the numbering of my webrevs to match the review-round. ;-)
- We moved shenandoah-specific code out of .ad files into our own .ad
files under gc/shenandoah (in shenandoah-gc), how cool is that? This
requires an addition in build machinery though, see
make/hotspot/gensrc/GensrcAdlc.gmk (in shared-build).
- Improved zero-disabling and build-code-simplification as suggested by
Magnus and Per
- Cleaned up some leftovers in C2
- Improved C2 loop opts code by introducing another APIs in
BarrierSetC2. See the new APIs in shared-gc under BarrierSetC2.hpp.
- I don't see where it makes sense to put INCLUDE_SHENANDOAHGC guards now.
- We would all very much prefer to keep ShenandoahXYZNode names, as
noted earlier. This stuff is Shenandoah-specific, so let's just call it
that.
- Rehashed Shenandoah tests (formatting, naming, directory layout, etc)
- Rebased on jdk-12+22
- Question: let us know if you need separate RFE for the new BSC2 APIs?
http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/03/
Thanks,
Roman

Post by Roman Kennke
- The minor issues that Kim reported in shared-gc
- A lot of fixes in shared-tests according to Leonid's review
- Disabled SA heapdumping similar to ZGC as Per suggested
Leonid:  test/hotspot/jtreg/gc/TestFullGCCount.java was actually
ExplicitGCInvokesConcurrent at the same time, because that would be
(expectedly) failing. It can run CMS, default GC and any other GC just
fine. Adding the same clause for Shenandoah means the same, and filters
the combination (+UseShenandoahGC)+(+ExplicitGCInvokesConcurrent). I
made the condition a bit clearer by avoiding triple-negation.
http://mail.openjdk.java.net/pipermail/shenandoah-dev/2018-November/008457.html
- test/hotspot/jtreg/serviceability/sa/ClhsdbJhisto.java
- test/hotspot/jtreg/serviceability/sa/TestHeapDumpForLargeArray.java
we filter them for Shenandoah now. I'm wondering: how do you get past
those with ZGC?
http://mail.openjdk.java.net/pipermail/shenandoah-dev/2018-November/008466.html
(Note to Leonid and tests reviewers: I'll add those related filters in
next round).
Vladimir: Roland integrated a bunch of changes to make loop* code look
better. I can tell that we're not done with C2 yet. Can you look over
the code and see what is ok, and especially what is not ok, so that we
can focus our efforts on the relevant parts?
http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/01/
Thanks,
Roman

Post by Roman Kennke
Hi,
This is the first round of changes for including Shenandoah GC into
mainline.
I divided the review into parts that roughly correspond to the mailing lists
that would normally review it, and I divided it into 'shared' code
changes and
'shenandoah' code changes (actually, mostly additions). The intend is to
eventually
push them as single 'combined' changeset, once reviewed.
   https://openjdk.java.net/jeps/189
  https://bugs.openjdk.java.net/browse/JDK-8214259
   http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/
For those who want to see the full change, have a look at the
shenandoah-complete
<http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shenandoah-complete/>
directory,
it contains the full combined webrev. Alternatively, there is the file
shenandoah-master.patch
<http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shenandoah-master.patch>,
which is what I intend to commit (and which should be equivalent to the
'shenandoah-complete' webrev).
  *) shenandoah-gc
<http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shenandoah-gc/>
     - Actual Shenandoah implementation, almost completely residing in
gc/shenandoah
  *) shared-gc
<http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shared-gc/>
     - This is mostly boilerplate that is common to any GC
     - referenceProcessor.cpp has a little change to make one
assert not
fail (next to CMS and G1)
     - taskqueue.hpp has some small adjustments to enable subclassing
  *) shared-serviceability
<http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shared-serviceability/>
     - The usual code to support another GC
  *) shared-runtime
<http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shared-runtime/>
     - A number of friends declarations to allow Shenandoah
iterators to
hook up with,
       e.g. ClassLoaderData, CodeCache, etc
     - Warning and disabling JFR LeakProfiler
     - fieldDescriptor.hpp added is_stable() accessor, for use in
Shenandoah C2 optimizations
     - Locks initialization in mutexLocker.cpp as usual
     - VM operations defines for Shenandoah's VM ops
     - globalDefinitions.hpp added UINT64_FORMAT_HEX_W for use in
Shenandoah's logging
     - The usual macros in macro.hpp
  *) shared-build
<http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shared-build/>
     - Add shenandoah feature, enabled by default, as agreed with
Vladimir K. beforehand
     - Some flags for shenandoah-enabled compilation to get
SUPPORT_BARRIER_ON_PRIMITIVES
       and SUPPORT_NOT_TO_SPACE_INVARIANT which is required for
Shenandoah's barriers
     - --param inline-unit-growth=1000 settings for 2 shenandoah
source
files, which is
       useful to get the whole marking loop inlined (observed
significant
regression if we
       don't)
  *) shared-tests
<http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shared-tests/>
     - Test infrastructure to support Shenandoah
     - Shenandoah test groups
     - Exclude Shenandoah in various tests that can be run with
selected GC
     - Enable/add configure for Shenandoah for tests that make
sense to
run with it
  *) shenandoah-tests
<http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shenandoah-tests/>
     - Shenandoah specific tests, most reside in gc/shenandoah
subdirectory
     - A couple of tests configurations have been added, e.g.
TestGCBasherWithShenandoah.java
I intentionally left out shared-compiler for now, because we have some
work left to do
there, but if you click around you'll find the patch anyway, in case you
want to take
a peek at it.
   - {Linux} x {x86_64, x86_32, armhf, aarch64, ppc64el, s390x}
   - {Windows} x {x86_64},
   - {MacOS X} x {x86_64}
   - the new Shenandoah tests
   - jcstress with/without aggressive Shenandoah verification
   - specjvm2008 with/without aggressive Shenandoah verification
I'd like to thank my collegues at Red Hat: Christine Flood, she deserves
the credit for being the original inventor of Shenandoah, Aleksey
Shiplëv, Roland Westrelin & Zhengyu Gu for their countless
contributions, everybody else in Red Hat's OpenJDK team for testing,
advice and support, my collegues in Oracle's GC, runtime and compiler
teams for tirelessly helping with and reviewing all the GC
interface and
related changes, and of course the many early adopters for reporting
bugs and success stories and feature requests: we wouldn't be here
without any of you!
Best regards,
Roman

[Roman Kennke]

Round 5 of Shenandoah review includes:
- A fix for the @requires tag in TestFullGCCountTest.java. It should be
correct now. We believe the CMS @requires was also not quite right and
fixed it the same.

It reads now: Don't run this test if:
- Actual GC set by harness is CMS *and* ExplicitGCInvokesConcurrent is
true, as set by harness
- Actual GC set by harness is Shenandoah *and*
ExplicitGCInvokesConcurrent is not set false by harness (it's true by
default in Shenandoah, so this needs to be double-inverteed).

The @requires for CMS was wrong before (we think), because it would also
filter defaultGC + ExplicitGCInvokesConcurrent.

- Sorting of macros was fixed, as was pointed out by Per
- Some stuff was added to SA, as suggested by Jini
- Rebased on most current jdk/jdk code

Webrevs:
http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/05/

I also need reviews from GC reviewers for the CSR:
https://bugs.openjdk.java.net/browse/JDK-8214349

I already got reviews for:
[x] shared-runtime (coleenp)
[x] shared-compiler (kvn)

I got reviews for shared-build, but an earlier version, so maybe makes
sense to look over this again. Erik J, Magnus?

I still need approvals for:
[ ] shared-build (kvn, erikj, ihse, pliden)
[ ] shared-gc (pliden, kbarrett)
[ ] shared-serviceability (jgeorge, pliden)
[ ] shared-tests (lmesnik, pliden)
[ ] shenandoah-gc
[ ] shenandoah-tests


Thanks for your patience and ongoing support!

Cheers,
Roman

Post by Roman Kennke
Hi all,
- Verify and gracefully reject dangerous flags combinations that
disables required barriers
- Trim unused code from Shenandoah's SA impl
- Move ShenandoahGCTracer to gc/shenandoah
- Fix ordering of GC names in various files
- Rename UINT64_FORMAT_HEX_W to UINT64_FORMAT_X_W
http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/04/
Thanks everybody for taking time to review this!
Roman

Post by Roman Kennke
Hello all,
Thanks so far for all the reviews and support!
I forgot to update the 'round' yesterday. We are in round 3 now :-)
Also, I fixed the numbering of my webrevs to match the review-round. ;-)
- We moved shenandoah-specific code out of .ad files into our own .ad
files under gc/shenandoah (in shenandoah-gc), how cool is that? This
requires an addition in build machinery though, see
make/hotspot/gensrc/GensrcAdlc.gmk (in shared-build).
- Improved zero-disabling and build-code-simplification as suggested by
Magnus and Per
- Cleaned up some leftovers in C2
- Improved C2 loop opts code by introducing another APIs in
BarrierSetC2. See the new APIs in shared-gc under BarrierSetC2.hpp.
- I don't see where it makes sense to put INCLUDE_SHENANDOAHGC guards now.
- We would all very much prefer to keep ShenandoahXYZNode names, as
noted earlier. This stuff is Shenandoah-specific, so let's just call it
that.
- Rehashed Shenandoah tests (formatting, naming, directory layout, etc)
- Rebased on jdk-12+22
- Question: let us know if you need separate RFE for the new BSC2 APIs?
http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/03/
Thanks,
Roman

Post by Roman Kennke
- The minor issues that Kim reported in shared-gc
- A lot of fixes in shared-tests according to Leonid's review
- Disabled SA heapdumping similar to ZGC as Per suggested
Leonid: test/hotspot/jtreg/gc/TestFullGCCount.java was actually
ExplicitGCInvokesConcurrent at the same time, because that would be
(expectedly) failing. It can run CMS, default GC and any other GC just
fine. Adding the same clause for Shenandoah means the same, and filters
the combination (+UseShenandoahGC)+(+ExplicitGCInvokesConcurrent). I
made the condition a bit clearer by avoiding triple-negation.
http://mail.openjdk.java.net/pipermail/shenandoah-dev/2018-November/008457.html
- test/hotspot/jtreg/serviceability/sa/ClhsdbJhisto.java
- test/hotspot/jtreg/serviceability/sa/TestHeapDumpForLargeArray.java
we filter them for Shenandoah now. I'm wondering: how do you get past
those with ZGC?
http://mail.openjdk.java.net/pipermail/shenandoah-dev/2018-November/008466.html
(Note to Leonid and tests reviewers: I'll add those related filters in
next round).
Vladimir: Roland integrated a bunch of changes to make loop* code look
better. I can tell that we're not done with C2 yet. Can you look over
the code and see what is ok, and especially what is not ok, so that we
can focus our efforts on the relevant parts?
http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/01/
Thanks,
Roman

Post by Roman Kennke
Hi,
This is the first round of changes for including Shenandoah GC into
mainline.
I divided the review into parts that roughly correspond to the mailing lists
that would normally review it, and I divided it into 'shared' code
changes and
'shenandoah' code changes (actually, mostly additions). The intend is to
eventually
push them as single 'combined' changeset, once reviewed.
  https://openjdk.java.net/jeps/189
 https://bugs.openjdk.java.net/browse/JDK-8214259
  http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/
For those who want to see the full change, have a look at the
shenandoah-complete
<http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shenandoah-complete/>
directory,
it contains the full combined webrev. Alternatively, there is the file
shenandoah-master.patch
<http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shenandoah-master.patch>,
which is what I intend to commit (and which should be equivalent to the
'shenandoah-complete' webrev).
 *) shenandoah-gc
<http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shenandoah-gc/>
    - Actual Shenandoah implementation, almost completely residing in
gc/shenandoah
 *) shared-gc
<http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shared-gc/>
    - This is mostly boilerplate that is common to any GC
    - referenceProcessor.cpp has a little change to make one assert not
fail (next to CMS and G1)
    - taskqueue.hpp has some small adjustments to enable subclassing
 *) shared-serviceability
<http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shared-serviceability/>
    - The usual code to support another GC
 *) shared-runtime
<http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shared-runtime/>
    - A number of friends declarations to allow Shenandoah iterators to
hook up with,
      e.g. ClassLoaderData, CodeCache, etc
    - Warning and disabling JFR LeakProfiler
    - fieldDescriptor.hpp added is_stable() accessor, for use in
Shenandoah C2 optimizations
    - Locks initialization in mutexLocker.cpp as usual
    - VM operations defines for Shenandoah's VM ops
    - globalDefinitions.hpp added UINT64_FORMAT_HEX_W for use in
Shenandoah's logging
    - The usual macros in macro.hpp
 *) shared-build
<http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shared-build/>
    - Add shenandoah feature, enabled by default, as agreed with
Vladimir K. beforehand
    - Some flags for shenandoah-enabled compilation to get
SUPPORT_BARRIER_ON_PRIMITIVES
      and SUPPORT_NOT_TO_SPACE_INVARIANT which is required for
Shenandoah's barriers
    - --param inline-unit-growth=1000 settings for 2 shenandoah source
files, which is
      useful to get the whole marking loop inlined (observed significant
regression if we
      don't)
 *) shared-tests
<http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shared-tests/>
    - Test infrastructure to support Shenandoah
    - Shenandoah test groups
    - Exclude Shenandoah in various tests that can be run with selected GC
    - Enable/add configure for Shenandoah for tests that make sense to
run with it
 *) shenandoah-tests
<http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shenandoah-tests/>
    - Shenandoah specific tests, most reside in gc/shenandoah subdirectory
    - A couple of tests configurations have been added, e.g.
TestGCBasherWithShenandoah.java
I intentionally left out shared-compiler for now, because we have some
work left to do
there, but if you click around you'll find the patch anyway, in case you
want to take
a peek at it.
  - {Linux} x {x86_64, x86_32, armhf, aarch64, ppc64el, s390x}
  - {Windows} x {x86_64},
  - {MacOS X} x {x86_64}
  - the new Shenandoah tests
  - jcstress with/without aggressive Shenandoah verification
  - specjvm2008 with/without aggressive Shenandoah verification
I'd like to thank my collegues at Red Hat: Christine Flood, she deserves
the credit for being the original inventor of Shenandoah, Aleksey
Shiplëv, Roland Westrelin & Zhengyu Gu for their countless
contributions, everybody else in Red Hat's OpenJDK team for testing,
advice and support, my collegues in Oracle's GC, runtime and compiler
teams for tirelessly helping with and reviewing all the GC interface and
related changes, and of course the many early adopters for reporting
bugs and success stories and feature requests: we wouldn't be here
without any of you!
Best regards,
Roman

[Magnus Ihse Bursie]

Post by Roman Kennke
fixed it the same.
- Actual GC set by harness is CMS *and* ExplicitGCInvokesConcurrent is
true, as set by harness
- Actual GC set by harness is Shenandoah *and*
ExplicitGCInvokesConcurrent is not set false by harness (it's true by
default in Shenandoah, so this needs to be double-inverteed).
filter defaultGC + ExplicitGCInvokesConcurrent.
- Sorting of macros was fixed, as was pointed out by Per
- Some stuff was added to SA, as suggested by Jini
- Rebased on most current jdk/jdk code
http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/05/
https://bugs.openjdk.java.net/browse/JDK-8214349
[x] shared-runtime (coleenp)
[x] shared-compiler (kvn)
I got reviews for shared-build, but an earlier version, so maybe makes
sense to look over this again. Erik J, Magnus?

Build changes look good.

/Magnus

Post by Roman Kennke
[ ] shared-build (kvn, erikj, ihse, pliden)
[ ] shared-gc (pliden, kbarrett)
[ ] shared-serviceability (jgeorge, pliden)
[ ] shared-tests (lmesnik, pliden)
[ ] shenandoah-gc
[ ] shenandoah-tests
Thanks for your patience and ongoing support!
Cheers,
Roman

Post by Roman Kennke
Hi all,
- Verify and gracefully reject dangerous flags combinations that
disables required barriers
- Trim unused code from Shenandoah's SA impl
- Move ShenandoahGCTracer to gc/shenandoah
- Fix ordering of GC names in various files
- Rename UINT64_FORMAT_HEX_W to UINT64_FORMAT_X_W
http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/04/
Thanks everybody for taking time to review this!
Roman

Post by Roman Kennke
Hello all,
Thanks so far for all the reviews and support!
I forgot to update the 'round' yesterday. We are in round 3 now :-)
Also, I fixed the numbering of my webrevs to match the review-round. ;-)
- We moved shenandoah-specific code out of .ad files into our own .ad
files under gc/shenandoah (in shenandoah-gc), how cool is that? This
requires an addition in build machinery though, see
make/hotspot/gensrc/GensrcAdlc.gmk (in shared-build).
- Improved zero-disabling and build-code-simplification as suggested by
Magnus and Per
- Cleaned up some leftovers in C2
- Improved C2 loop opts code by introducing another APIs in
BarrierSetC2. See the new APIs in shared-gc under BarrierSetC2.hpp.
- I don't see where it makes sense to put INCLUDE_SHENANDOAHGC guards now.
- We would all very much prefer to keep ShenandoahXYZNode names, as
noted earlier. This stuff is Shenandoah-specific, so let's just call it
that.
- Rehashed Shenandoah tests (formatting, naming, directory layout, etc)
- Rebased on jdk-12+22
- Question: let us know if you need separate RFE for the new BSC2 APIs?
http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/03/
Thanks,
Roman

Post by Roman Kennke
- The minor issues that Kim reported in shared-gc
- A lot of fixes in shared-tests according to Leonid's review
- Disabled SA heapdumping similar to ZGC as Per suggested
Leonid: test/hotspot/jtreg/gc/TestFullGCCount.java was actually
ExplicitGCInvokesConcurrent at the same time, because that would be
(expectedly) failing. It can run CMS, default GC and any other GC just
fine. Adding the same clause for Shenandoah means the same, and filters
the combination (+UseShenandoahGC)+(+ExplicitGCInvokesConcurrent). I
made the condition a bit clearer by avoiding triple-negation.
http://mail.openjdk.java.net/pipermail/shenandoah-dev/2018-November/008457.html
- test/hotspot/jtreg/serviceability/sa/ClhsdbJhisto.java
- test/hotspot/jtreg/serviceability/sa/TestHeapDumpForLargeArray.java
we filter them for Shenandoah now. I'm wondering: how do you get past
those with ZGC?
http://mail.openjdk.java.net/pipermail/shenandoah-dev/2018-November/008466.html
(Note to Leonid and tests reviewers: I'll add those related filters in
next round).
Vladimir: Roland integrated a bunch of changes to make loop* code look
better. I can tell that we're not done with C2 yet. Can you look over
the code and see what is ok, and especially what is not ok, so that we
can focus our efforts on the relevant parts?
http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/01/
Thanks,
Roman

Post by Roman Kennke
Hi,
This is the first round of changes for including Shenandoah GC into
mainline.
I divided the review into parts that roughly correspond to the mailing lists
that would normally review it, and I divided it into 'shared' code
changes and
'shenandoah' code changes (actually, mostly additions). The intend is to
eventually
push them as single 'combined' changeset, once reviewed.
https://openjdk.java.net/jeps/189
https://bugs.openjdk.java.net/browse/JDK-8214259
http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/
For those who want to see the full change, have a look at the
shenandoah-complete
<http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shenandoah-complete/>
directory,
it contains the full combined webrev. Alternatively, there is the file
shenandoah-master.patch
<http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shenandoah-master.patch>,
which is what I intend to commit (and which should be equivalent to the
'shenandoah-complete' webrev).
*) shenandoah-gc
<http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shenandoah-gc/>
- Actual Shenandoah implementation, almost completely residing in
gc/shenandoah
*) shared-gc
<http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shared-gc/>
- This is mostly boilerplate that is common to any GC
- referenceProcessor.cpp has a little change to make one assert not
fail (next to CMS and G1)
- taskqueue.hpp has some small adjustments to enable subclassing
*) shared-serviceability
<http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shared-serviceability/>
- The usual code to support another GC
*) shared-runtime
<http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shared-runtime/>
- A number of friends declarations to allow Shenandoah iterators to
hook up with,
e.g. ClassLoaderData, CodeCache, etc
- Warning and disabling JFR LeakProfiler
- fieldDescriptor.hpp added is_stable() accessor, for use in
Shenandoah C2 optimizations
- Locks initialization in mutexLocker.cpp as usual
- VM operations defines for Shenandoah's VM ops
- globalDefinitions.hpp added UINT64_FORMAT_HEX_W for use in
Shenandoah's logging
- The usual macros in macro.hpp
*) shared-build
<http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shared-build/>
- Add shenandoah feature, enabled by default, as agreed with
Vladimir K. beforehand
- Some flags for shenandoah-enabled compilation to get
SUPPORT_BARRIER_ON_PRIMITIVES
and SUPPORT_NOT_TO_SPACE_INVARIANT which is required for
Shenandoah's barriers
- --param inline-unit-growth=1000 settings for 2 shenandoah source
files, which is
useful to get the whole marking loop inlined (observed significant
regression if we
don't)
*) shared-tests
<http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shared-tests/>
- Test infrastructure to support Shenandoah
- Shenandoah test groups
- Exclude Shenandoah in various tests that can be run with selected GC
- Enable/add configure for Shenandoah for tests that make sense to
run with it
*) shenandoah-tests
<http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shenandoah-tests/>
- Shenandoah specific tests, most reside in gc/shenandoah subdirectory
- A couple of tests configurations have been added, e.g.
TestGCBasherWithShenandoah.java
I intentionally left out shared-compiler for now, because we have some
work left to do
there, but if you click around you'll find the patch anyway, in case you
want to take
a peek at it.
- {Linux} x {x86_64, x86_32, armhf, aarch64, ppc64el, s390x}
- {Windows} x {x86_64},
- {MacOS X} x {x86_64}
- the new Shenandoah tests
- jcstress with/without aggressive Shenandoah verification
- specjvm2008 with/without aggressive Shenandoah verification
I'd like to thank my collegues at Red Hat: Christine Flood, she deserves
the credit for being the original inventor of Shenandoah, Aleksey
Shiplëv, Roland Westrelin & Zhengyu Gu for their countless
contributions, everybody else in Red Hat's OpenJDK team for testing,
advice and support, my collegues in Oracle's GC, runtime and compiler
teams for tirelessly helping with and reviewing all the GC interface and
related changes, and of course the many early adopters for reporting
bugs and success stories and feature requests: we wouldn't be here
without any of you!
Best regards,
Roman

[Roman Kennke]

Hi Magnus,

Post by Magnus Ihse Bursie

Post by Roman Kennke
I got reviews for shared-build, but an earlier version, so maybe makes
sense to look over this again. Erik J, Magnus?

Build changes look good.

Thanks, Magnus!

Roman

[Jini George]

Hi Roman,

Thank you for making the changes. The SA portion looks good to me. One
nit though:

In
src/jdk.hotspot.agent/share/classes/sun/jvm/hotspot/tools/HeapSummary.java,
in printGCAlgorithm(), does displaying the nbr of Parallel GC threads
not make sense for Shenandoah (like it is for G1, ZGC, etc)?

Thank you,
Jini.

Post by Roman Kennke
fixed it the same.
- Actual GC set by harness is CMS *and* ExplicitGCInvokesConcurrent is
true, as set by harness
- Actual GC set by harness is Shenandoah *and*
ExplicitGCInvokesConcurrent is not set false by harness (it's true by
default in Shenandoah, so this needs to be double-inverteed).
filter defaultGC + ExplicitGCInvokesConcurrent.
- Sorting of macros was fixed, as was pointed out by Per
- Some stuff was added to SA, as suggested by Jini
- Rebased on most current jdk/jdk code
http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/05/
https://bugs.openjdk.java.net/browse/JDK-8214349
[x] shared-runtime (coleenp)
[x] shared-compiler (kvn)
I got reviews for shared-build, but an earlier version, so maybe makes
sense to look over this again. Erik J, Magnus?
[ ] shared-build (kvn, erikj, ihse, pliden)
[ ] shared-gc (pliden, kbarrett)
[ ] shared-serviceability (jgeorge, pliden)
[ ] shared-tests (lmesnik, pliden)
[ ] shenandoah-gc
[ ] shenandoah-tests
Thanks for your patience and ongoing support!
Cheers,
Roman

Post by Roman Kennke
Hi all,
- Verify and gracefully reject dangerous flags combinations that
disables required barriers
- Trim unused code from Shenandoah's SA impl
- Move ShenandoahGCTracer to gc/shenandoah
- Fix ordering of GC names in various files
- Rename UINT64_FORMAT_HEX_W to UINT64_FORMAT_X_W
http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/04/
Thanks everybody for taking time to review this!
Roman

Post by Roman Kennke
Hello all,
Thanks so far for all the reviews and support!
I forgot to update the 'round' yesterday. We are in round 3 now :-)
Also, I fixed the numbering of my webrevs to match the review-round. ;-)
- We moved shenandoah-specific code out of .ad files into our own .ad
files under gc/shenandoah (in shenandoah-gc), how cool is that? This
requires an addition in build machinery though, see
make/hotspot/gensrc/GensrcAdlc.gmk (in shared-build).
- Improved zero-disabling and build-code-simplification as suggested by
Magnus and Per
- Cleaned up some leftovers in C2
- Improved C2 loop opts code by introducing another APIs in
BarrierSetC2. See the new APIs in shared-gc under BarrierSetC2.hpp.
- I don't see where it makes sense to put INCLUDE_SHENANDOAHGC guards now.
- We would all very much prefer to keep ShenandoahXYZNode names, as
noted earlier. This stuff is Shenandoah-specific, so let's just call it
that.
- Rehashed Shenandoah tests (formatting, naming, directory layout, etc)
- Rebased on jdk-12+22
- Question: let us know if you need separate RFE for the new BSC2 APIs?
http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/03/
Thanks,
Roman

Post by Roman Kennke
- The minor issues that Kim reported in shared-gc
- A lot of fixes in shared-tests according to Leonid's review
- Disabled SA heapdumping similar to ZGC as Per suggested
Leonid: test/hotspot/jtreg/gc/TestFullGCCount.java was actually
ExplicitGCInvokesConcurrent at the same time, because that would be
(expectedly) failing. It can run CMS, default GC and any other GC just
fine. Adding the same clause for Shenandoah means the same, and filters
the combination (+UseShenandoahGC)+(+ExplicitGCInvokesConcurrent). I
made the condition a bit clearer by avoiding triple-negation.
http://mail.openjdk.java.net/pipermail/shenandoah-dev/2018-November/008457.html
- test/hotspot/jtreg/serviceability/sa/ClhsdbJhisto.java
- test/hotspot/jtreg/serviceability/sa/TestHeapDumpForLargeArray.java
we filter them for Shenandoah now. I'm wondering: how do you get past
those with ZGC?
http://mail.openjdk.java.net/pipermail/shenandoah-dev/2018-November/008466.html
(Note to Leonid and tests reviewers: I'll add those related filters in
next round).
Vladimir: Roland integrated a bunch of changes to make loop* code look
better. I can tell that we're not done with C2 yet. Can you look over
the code and see what is ok, and especially what is not ok, so that we
can focus our efforts on the relevant parts?
http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/01/
Thanks,
Roman

Post by Roman Kennke
Hi,
This is the first round of changes for including Shenandoah GC into
mainline.
I divided the review into parts that roughly correspond to the mailing lists
that would normally review it, and I divided it into 'shared' code
changes and
'shenandoah' code changes (actually, mostly additions). The intend is to
eventually
push them as single 'combined' changeset, once reviewed.
  https://openjdk.java.net/jeps/189
 https://bugs.openjdk.java.net/browse/JDK-8214259
  http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/
For those who want to see the full change, have a look at the
shenandoah-complete
<http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shenandoah-complete/>
directory,
it contains the full combined webrev. Alternatively, there is the file
shenandoah-master.patch
<http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shenandoah-master.patch>,
which is what I intend to commit (and which should be equivalent to the
'shenandoah-complete' webrev).
 *) shenandoah-gc
<http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shenandoah-gc/>
    - Actual Shenandoah implementation, almost completely residing in
gc/shenandoah
 *) shared-gc
<http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shared-gc/>
    - This is mostly boilerplate that is common to any GC
    - referenceProcessor.cpp has a little change to make one assert not
fail (next to CMS and G1)
    - taskqueue.hpp has some small adjustments to enable subclassing
 *) shared-serviceability
<http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shared-serviceability/>
    - The usual code to support another GC
 *) shared-runtime
<http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shared-runtime/>
    - A number of friends declarations to allow Shenandoah iterators to
hook up with,
      e.g. ClassLoaderData, CodeCache, etc
    - Warning and disabling JFR LeakProfiler
    - fieldDescriptor.hpp added is_stable() accessor, for use in
Shenandoah C2 optimizations
    - Locks initialization in mutexLocker.cpp as usual
    - VM operations defines for Shenandoah's VM ops
    - globalDefinitions.hpp added UINT64_FORMAT_HEX_W for use in
Shenandoah's logging
    - The usual macros in macro.hpp
 *) shared-build
<http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shared-build/>
    - Add shenandoah feature, enabled by default, as agreed with
Vladimir K. beforehand
    - Some flags for shenandoah-enabled compilation to get
SUPPORT_BARRIER_ON_PRIMITIVES
      and SUPPORT_NOT_TO_SPACE_INVARIANT which is required for
Shenandoah's barriers
    - --param inline-unit-growth=1000 settings for 2 shenandoah source
files, which is
      useful to get the whole marking loop inlined (observed significant
regression if we
      don't)
 *) shared-tests
<http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shared-tests/>
    - Test infrastructure to support Shenandoah
    - Shenandoah test groups
    - Exclude Shenandoah in various tests that can be run with selected GC
    - Enable/add configure for Shenandoah for tests that make sense to
run with it
 *) shenandoah-tests
<http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shenandoah-tests/>
    - Shenandoah specific tests, most reside in gc/shenandoah subdirectory
    - A couple of tests configurations have been added, e.g.
TestGCBasherWithShenandoah.java
I intentionally left out shared-compiler for now, because we have some
work left to do
there, but if you click around you'll find the patch anyway, in case you
want to take
a peek at it.
  - {Linux} x {x86_64, x86_32, armhf, aarch64, ppc64el, s390x}
  - {Windows} x {x86_64},
  - {MacOS X} x {x86_64}
  - the new Shenandoah tests
  - jcstress with/without aggressive Shenandoah verification
  - specjvm2008 with/without aggressive Shenandoah verification
I'd like to thank my collegues at Red Hat: Christine Flood, she deserves
the credit for being the original inventor of Shenandoah, Aleksey
Shiplëv, Roland Westrelin & Zhengyu Gu for their countless
contributions, everybody else in Red Hat's OpenJDK team for testing,
advice and support, my collegues in Oracle's GC, runtime and compiler
teams for tirelessly helping with and reviewing all the GC interface and
related changes, and of course the many early adopters for reporting
bugs and success stories and feature requests: we wouldn't be here
without any of you!
Best regards,
Roman

[Roman Kennke]

Hi Jini,

Post by Jini George
Thank you for making the changes. The SA portion looks good to me.

Thank you!

Post by Jini George
One
In
src/jdk.hotspot.agent/share/classes/sun/jvm/hotspot/tools/HeapSummary.java,
in printGCAlgorithm(), does displaying the nbr of Parallel GC threads
not make sense for Shenandoah (like it is for G1, ZGC, etc)?

I suppose it does. I will add it.

Thanks,
Roman

Post by Jini George
Thank you,
Jini.

Post by Roman Kennke
fixed it the same.
  - Actual GC set by harness is CMS *and* ExplicitGCInvokesConcurrent is
true, as set by harness
  - Actual GC set by harness is Shenandoah *and*
ExplicitGCInvokesConcurrent is not set false by harness (it's true by
default in Shenandoah, so this needs to be double-inverteed).
filter defaultGC + ExplicitGCInvokesConcurrent.
- Sorting of macros was fixed, as was pointed out by Per
- Some stuff was added to SA, as suggested by Jini
- Rebased on most current jdk/jdk code
http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/05/
https://bugs.openjdk.java.net/browse/JDK-8214349
[x] shared-runtime (coleenp)
[x] shared-compiler (kvn)
I got reviews for shared-build, but an earlier version, so maybe makes
sense to look over this again. Erik J, Magnus?
[ ] shared-build          (kvn, erikj, ihse, pliden)
[ ] shared-gc             (pliden, kbarrett)
[ ] shared-serviceability (jgeorge, pliden)
[ ] shared-tests          (lmesnik, pliden)
[ ] shenandoah-gc
[ ] shenandoah-tests
Thanks for your patience and ongoing support!
Cheers,
Roman

Post by Roman Kennke
Hi all,
- Verify and gracefully reject dangerous flags combinations that
disables required barriers
- Trim unused code from Shenandoah's SA impl
- Move ShenandoahGCTracer to gc/shenandoah
- Fix ordering of GC names in various files
- Rename UINT64_FORMAT_HEX_W to UINT64_FORMAT_X_W
http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/04/
Thanks everybody for taking time to review this!
Roman

Post by Roman Kennke
Hello all,
Thanks so far for all the reviews and support!
I forgot to update the 'round' yesterday. We are in round 3 now :-)
Also, I fixed the numbering of my webrevs to match the review-round. ;-)
- We moved shenandoah-specific code out of .ad files into our own .ad
files under gc/shenandoah (in shenandoah-gc), how cool is that? This
requires an addition in build machinery though, see
make/hotspot/gensrc/GensrcAdlc.gmk (in shared-build).
- Improved zero-disabling and build-code-simplification as suggested by
Magnus and Per
- Cleaned up some leftovers in C2
- Improved C2 loop opts code by introducing another APIs in
BarrierSetC2. See the new APIs in shared-gc under BarrierSetC2.hpp.
- I don't see where it makes sense to put INCLUDE_SHENANDOAHGC guards now.
- We would all very much prefer to keep ShenandoahXYZNode names, as
noted earlier. This stuff is Shenandoah-specific, so let's just call it
that.
- Rehashed Shenandoah tests (formatting, naming, directory layout, etc)
- Rebased on jdk-12+22
- Question: let us know if you need separate RFE for the new BSC2 APIs?
http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/03/
Thanks,
Roman

Post by Roman Kennke
- The minor issues that Kim reported in shared-gc
- A lot of fixes in shared-tests according to Leonid's review
- Disabled SA heapdumping similar to ZGC as Per suggested
Leonid:  test/hotspot/jtreg/gc/TestFullGCCount.java was actually
ExplicitGCInvokesConcurrent at the same time, because that would be
(expectedly) failing. It can run CMS, default GC and any other GC just
fine. Adding the same clause for Shenandoah means the same, and filters
the combination (+UseShenandoahGC)+(+ExplicitGCInvokesConcurrent). I
made the condition a bit clearer by avoiding triple-negation.
http://mail.openjdk.java.net/pipermail/shenandoah-dev/2018-November/008457.html
- test/hotspot/jtreg/serviceability/sa/ClhsdbJhisto.java
- test/hotspot/jtreg/serviceability/sa/TestHeapDumpForLargeArray.java
we filter them for Shenandoah now. I'm wondering: how do you get past
those with ZGC?
http://mail.openjdk.java.net/pipermail/shenandoah-dev/2018-November/008466.html
(Note to Leonid and tests reviewers: I'll add those related filters in
next round).
Vladimir: Roland integrated a bunch of changes to make loop* code look
better. I can tell that we're not done with C2 yet. Can you look over
the code and see what is ok, and especially what is not ok, so that we
can focus our efforts on the relevant parts?
http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/01/
Thanks,
Roman

Post by Roman Kennke
Hi,
This is the first round of changes for including Shenandoah GC into
mainline.
I divided the review into parts that roughly correspond to the mailing lists
that would normally review it, and I divided it into 'shared' code
changes and
'shenandoah' code changes (actually, mostly additions). The intend is to
eventually
push them as single 'combined' changeset, once reviewed.
   https://openjdk.java.net/jeps/189
  https://bugs.openjdk.java.net/browse/JDK-8214259
   http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/
For those who want to see the full change, have a look at the
shenandoah-complete
<http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shenandoah-complete/>
directory,
it contains the full combined webrev. Alternatively, there is the file
shenandoah-master.patch
<http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shenandoah-master.patch>,
which is what I intend to commit (and which should be equivalent to the
'shenandoah-complete' webrev).
  *) shenandoah-gc
<http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shenandoah-gc/>
     - Actual Shenandoah implementation, almost completely
residing in
gc/shenandoah
  *) shared-gc
<http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shared-gc/>
     - This is mostly boilerplate that is common to any GC
     - referenceProcessor.cpp has a little change to make one
assert not
fail (next to CMS and G1)
     - taskqueue.hpp has some small adjustments to enable subclassing
  *) shared-serviceability
<http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shared-serviceability/>
     - The usual code to support another GC
  *) shared-runtime
<http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shared-runtime/>
     - A number of friends declarations to allow Shenandoah
iterators to
hook up with,
       e.g. ClassLoaderData, CodeCache, etc
     - Warning and disabling JFR LeakProfiler
     - fieldDescriptor.hpp added is_stable() accessor, for use in
Shenandoah C2 optimizations
     - Locks initialization in mutexLocker.cpp as usual
     - VM operations defines for Shenandoah's VM ops
     - globalDefinitions.hpp added UINT64_FORMAT_HEX_W for use in
Shenandoah's logging
     - The usual macros in macro.hpp
  *) shared-build
<http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shared-build/>
     - Add shenandoah feature, enabled by default, as agreed with
Vladimir K. beforehand
     - Some flags for shenandoah-enabled compilation to get
SUPPORT_BARRIER_ON_PRIMITIVES
       and SUPPORT_NOT_TO_SPACE_INVARIANT which is required for
Shenandoah's barriers
     - --param inline-unit-growth=1000 settings for 2 shenandoah
source
files, which is
       useful to get the whole marking loop inlined (observed
significant
regression if we
       don't)
  *) shared-tests
<http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shared-tests/>
     - Test infrastructure to support Shenandoah
     - Shenandoah test groups
     - Exclude Shenandoah in various tests that can be run with
selected GC
     - Enable/add configure for Shenandoah for tests that make
sense to
run with it
  *) shenandoah-tests
<http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shenandoah-tests/>
     - Shenandoah specific tests, most reside in gc/shenandoah
subdirectory
     - A couple of tests configurations have been added, e.g.
TestGCBasherWithShenandoah.java
I intentionally left out shared-compiler for now, because we have some
work left to do
there, but if you click around you'll find the patch anyway, in case you
want to take
a peek at it.
   - {Linux} x {x86_64, x86_32, armhf, aarch64, ppc64el, s390x}
   - {Windows} x {x86_64},
   - {MacOS X} x {x86_64}
   - the new Shenandoah tests
   - jcstress with/without aggressive Shenandoah verification
   - specjvm2008 with/without aggressive Shenandoah verification
I'd like to thank my collegues at Red Hat: Christine Flood, she deserves
the credit for being the original inventor of Shenandoah, Aleksey
Shiplëv, Roland Westrelin & Zhengyu Gu for their countless
contributions, everybody else in Red Hat's OpenJDK team for testing,
advice and support, my collegues in Oracle's GC, runtime and compiler
teams for tirelessly helping with and reviewing all the GC interface and
related changes, and of course the many early adopters for reporting
bugs and success stories and feature requests: we wouldn't be here
without any of you!
Best regards,
Roman

[Per Liden]

Hi Roman,

Post by Roman Kennke
fixed it the same.
- Actual GC set by harness is CMS *and* ExplicitGCInvokesConcurrent is
true, as set by harness
- Actual GC set by harness is Shenandoah *and*
ExplicitGCInvokesConcurrent is not set false by harness (it's true by
default in Shenandoah, so this needs to be double-inverteed).
filter defaultGC + ExplicitGCInvokesConcurrent.
- Sorting of macros was fixed, as was pointed out by Per
- Some stuff was added to SA, as suggested by Jini
- Rebased on most current jdk/jdk code
http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/05/
https://bugs.openjdk.java.net/browse/JDK-8214349
[x] shared-runtime (coleenp)
[x] shared-compiler (kvn)
I got reviews for shared-build, but an earlier version, so maybe makes
sense to look over this again. Erik J, Magnus?
[ ] shared-build (kvn, erikj, ihse, pliden)
[ ] shared-gc (pliden, kbarrett)
[ ] shared-serviceability (jgeorge, pliden)
[ ] shared-tests (lmesnik, pliden)

The above parts look good to me. Reviewed.

Just one tiny nit (and I don't need to see a new webrev for this):

In src/hotspot/share/gc/shared/gcCause.cpp you have this:

+ case _shenandoah_allocation_failure_evac:
+ return "Allocation Failure During Evac";
+
+ case _shenandoah_stop_vm:
+ return "Stopping VM";
+
+ case _shenandoah_concurrent_gc:
+ return "Shenandoah Concurrent GC";
+
+ case _shenandoah_traversal_gc:
+ return "Shenandoah Traversal GC";
+
+ case _shenandoah_upgrade_to_full_gc:
+ return "Shenandoah Upgrade To Full GC";
+

And in
src/jdk.hotspot.agent/share/classes/sun/jvm/hotspot/gc/shared/GCCause.java
you have this:

+ _shenandoah_stop_vm ("Stop VM"),
+ _shenandoah_allocation_failure_evac ("Allocation Failure During
Evacuation"),
+ _shenandoah_concurrent_gc ("Concurrent GC"),
+ _shenandoah_traversal_gc ("Traversal GC"),
+ _shenandoah_upgrade_to_full_gc ("Upgrade to Full GC"),

It would be good to have the exact same strings in both places. There
are currently small differences in all of them. "Evac" vs "Evacuation",
"Stop" vs "Stopping", "Shenandoah" vs "", etc.

May I also suggest that you skip "Shenandoah" in things like "Shenandoah
Concurrent GC" as I kind of think it's implied by the context. But I
also know that CMS/G1 isn't consistent on this point. An alternative
would be to add "Shenandoah" to all of the strings to keep things
consistent, but I'm not sure I like that better. You decide.

Post by Roman Kennke
[ ] shenandoah-gc
[ ] shenandoah-tests

I haven't looked very much on these parts, and I didn't plan to do so in
detail right now. I think it's fine of the folks that have been working
on the Shenandoah code reviewed this.

cheers,
Per

Post by Roman Kennke
Thanks for your patience and ongoing support!
Cheers,
Roman

Post by Roman Kennke
Hi all,
- Verify and gracefully reject dangerous flags combinations that
disables required barriers
- Trim unused code from Shenandoah's SA impl
- Move ShenandoahGCTracer to gc/shenandoah
- Fix ordering of GC names in various files
- Rename UINT64_FORMAT_HEX_W to UINT64_FORMAT_X_W
http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/04/
Thanks everybody for taking time to review this!
Roman

Post by Roman Kennke
Hello all,
Thanks so far for all the reviews and support!
I forgot to update the 'round' yesterday. We are in round 3 now :-)
Also, I fixed the numbering of my webrevs to match the review-round. ;-)
- We moved shenandoah-specific code out of .ad files into our own .ad
files under gc/shenandoah (in shenandoah-gc), how cool is that? This
requires an addition in build machinery though, see
make/hotspot/gensrc/GensrcAdlc.gmk (in shared-build).
- Improved zero-disabling and build-code-simplification as suggested by
Magnus and Per
- Cleaned up some leftovers in C2
- Improved C2 loop opts code by introducing another APIs in
BarrierSetC2. See the new APIs in shared-gc under BarrierSetC2.hpp.
- I don't see where it makes sense to put INCLUDE_SHENANDOAHGC guards now.
- We would all very much prefer to keep ShenandoahXYZNode names, as
noted earlier. This stuff is Shenandoah-specific, so let's just call it
that.
- Rehashed Shenandoah tests (formatting, naming, directory layout, etc)
- Rebased on jdk-12+22
- Question: let us know if you need separate RFE for the new BSC2 APIs?
http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/03/
Thanks,
Roman

Post by Roman Kennke
- The minor issues that Kim reported in shared-gc
- A lot of fixes in shared-tests according to Leonid's review
- Disabled SA heapdumping similar to ZGC as Per suggested
Leonid: test/hotspot/jtreg/gc/TestFullGCCount.java was actually
ExplicitGCInvokesConcurrent at the same time, because that would be
(expectedly) failing. It can run CMS, default GC and any other GC just
fine. Adding the same clause for Shenandoah means the same, and filters
the combination (+UseShenandoahGC)+(+ExplicitGCInvokesConcurrent). I
made the condition a bit clearer by avoiding triple-negation.
http://mail.openjdk.java.net/pipermail/shenandoah-dev/2018-November/008457.html
- test/hotspot/jtreg/serviceability/sa/ClhsdbJhisto.java
- test/hotspot/jtreg/serviceability/sa/TestHeapDumpForLargeArray.java
we filter them for Shenandoah now. I'm wondering: how do you get past
those with ZGC?
http://mail.openjdk.java.net/pipermail/shenandoah-dev/2018-November/008466.html
(Note to Leonid and tests reviewers: I'll add those related filters in
next round).
Vladimir: Roland integrated a bunch of changes to make loop* code look
better. I can tell that we're not done with C2 yet. Can you look over
the code and see what is ok, and especially what is not ok, so that we
can focus our efforts on the relevant parts?
http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/01/
Thanks,
Roman

Post by Roman Kennke
Hi,
This is the first round of changes for including Shenandoah GC into
mainline.
I divided the review into parts that roughly correspond to the mailing lists
that would normally review it, and I divided it into 'shared' code
changes and
'shenandoah' code changes (actually, mostly additions). The intend is to
eventually
push them as single 'combined' changeset, once reviewed.
  https://openjdk.java.net/jeps/189
 https://bugs.openjdk.java.net/browse/JDK-8214259
  http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/
For those who want to see the full change, have a look at the
shenandoah-complete
<http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shenandoah-complete/>
directory,
it contains the full combined webrev. Alternatively, there is the file
shenandoah-master.patch
<http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shenandoah-master.patch>,
which is what I intend to commit (and which should be equivalent to the
'shenandoah-complete' webrev).
 *) shenandoah-gc
<http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shenandoah-gc/>
    - Actual Shenandoah implementation, almost completely residing in
gc/shenandoah
 *) shared-gc
<http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shared-gc/>
    - This is mostly boilerplate that is common to any GC
    - referenceProcessor.cpp has a little change to make one assert not
fail (next to CMS and G1)
    - taskqueue.hpp has some small adjustments to enable subclassing
 *) shared-serviceability
<http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shared-serviceability/>
    - The usual code to support another GC
 *) shared-runtime
<http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shared-runtime/>
    - A number of friends declarations to allow Shenandoah iterators to
hook up with,
      e.g. ClassLoaderData, CodeCache, etc
    - Warning and disabling JFR LeakProfiler
    - fieldDescriptor.hpp added is_stable() accessor, for use in
Shenandoah C2 optimizations
    - Locks initialization in mutexLocker.cpp as usual
    - VM operations defines for Shenandoah's VM ops
    - globalDefinitions.hpp added UINT64_FORMAT_HEX_W for use in
Shenandoah's logging
    - The usual macros in macro.hpp
 *) shared-build
<http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shared-build/>
    - Add shenandoah feature, enabled by default, as agreed with
Vladimir K. beforehand
    - Some flags for shenandoah-enabled compilation to get
SUPPORT_BARRIER_ON_PRIMITIVES
      and SUPPORT_NOT_TO_SPACE_INVARIANT which is required for
Shenandoah's barriers
    - --param inline-unit-growth=1000 settings for 2 shenandoah source
files, which is
      useful to get the whole marking loop inlined (observed significant
regression if we
      don't)
 *) shared-tests
<http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shared-tests/>
    - Test infrastructure to support Shenandoah
    - Shenandoah test groups
    - Exclude Shenandoah in various tests that can be run with selected GC
    - Enable/add configure for Shenandoah for tests that make sense to
run with it
 *) shenandoah-tests
<http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shenandoah-tests/>
    - Shenandoah specific tests, most reside in gc/shenandoah subdirectory
    - A couple of tests configurations have been added, e.g.
TestGCBasherWithShenandoah.java
I intentionally left out shared-compiler for now, because we have some
work left to do
there, but if you click around you'll find the patch anyway, in case you
want to take
a peek at it.
  - {Linux} x {x86_64, x86_32, armhf, aarch64, ppc64el, s390x}
  - {Windows} x {x86_64},
  - {MacOS X} x {x86_64}
  - the new Shenandoah tests
  - jcstress with/without aggressive Shenandoah verification
  - specjvm2008 with/without aggressive Shenandoah verification
I'd like to thank my collegues at Red Hat: Christine Flood, she deserves
the credit for being the original inventor of Shenandoah, Aleksey
Shiplëv, Roland Westrelin & Zhengyu Gu for their countless
contributions, everybody else in Red Hat's OpenJDK team for testing,
advice and support, my collegues in Oracle's GC, runtime and compiler
teams for tirelessly helping with and reviewing all the GC interface and
related changes, and of course the many early adopters for reporting
bugs and success stories and feature requests: we wouldn't be here
without any of you!
Best regards,
Roman

[Roman Kennke]

Hi Per,

Post by Per Liden

Post by Roman Kennke
fixed it the same.
  - Actual GC set by harness is CMS *and* ExplicitGCInvokesConcurrent is
true, as set by harness
  - Actual GC set by harness is Shenandoah *and*
ExplicitGCInvokesConcurrent is not set false by harness (it's true by
default in Shenandoah, so this needs to be double-inverteed).
filter defaultGC + ExplicitGCInvokesConcurrent.
- Sorting of macros was fixed, as was pointed out by Per
- Some stuff was added to SA, as suggested by Jini
- Rebased on most current jdk/jdk code
http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/05/
https://bugs.openjdk.java.net/browse/JDK-8214349
[x] shared-runtime (coleenp)
[x] shared-compiler (kvn)
I got reviews for shared-build, but an earlier version, so maybe makes
sense to look over this again. Erik J, Magnus?
[ ] shared-build          (kvn, erikj, ihse, pliden)
[ ] shared-gc             (pliden, kbarrett)
[ ] shared-serviceability (jgeorge, pliden)
[ ] shared-tests          (lmesnik, pliden)

The above parts look good to me. Reviewed.

Great! Thanks!

Post by Per Liden
+      return "Allocation Failure During Evac";
+
+      return "Stopping VM";
+
+      return "Shenandoah Concurrent GC";
+
+      return "Shenandoah Traversal GC";
+
+      return "Shenandoah Upgrade To Full GC";
+
And in
src/jdk.hotspot.agent/share/classes/sun/jvm/hotspot/gc/shared/GCCause.java
+  _shenandoah_stop_vm ("Stop VM"),
+  _shenandoah_allocation_failure_evac ("Allocation Failure During
Evacuation"),
+  _shenandoah_concurrent_gc ("Concurrent GC"),
+  _shenandoah_traversal_gc ("Traversal GC"),
+  _shenandoah_upgrade_to_full_gc ("Upgrade to Full GC"),
It would be good to have the exact same strings in both places. There
are currently small differences in all of them. "Evac" vs "Evacuation",
"Stop" vs "Stopping", "Shenandoah" vs "", etc.
May I also suggest that you skip "Shenandoah" in things like "Shenandoah
Concurrent GC" as I kind of think it's implied by the context. But I
also know that CMS/G1 isn't consistent on this point. An alternative
would be to add "Shenandoah" to all of the strings to keep things
consistent, but I'm not sure I like that better. You decide.

I'm addressing it here:
http://mail.openjdk.java.net/pipermail/shenandoah-dev/2018-December/008572.html

Post by Per Liden

Post by Roman Kennke
[ ] shenandoah-gc
[ ] shenandoah-tests

I haven't looked very much on these parts, and I didn't plan to do so in
detail right now. I think it's fine of the folks that have been working
on the Shenandoah code reviewed this.

Yes, I think so too (except maybe stuff like shenandoah_globals.hpp, but
that was already reviewed I think).

Thanks for reviewing, Per!

Roman

Post by Per Liden
cheers,
Per

Post by Roman Kennke
Thanks for your patience and ongoing support!
Cheers,
Roman

Post by Roman Kennke
Hi all,
- Verify and gracefully reject dangerous flags combinations that
disables required barriers
- Trim unused code from Shenandoah's SA impl
- Move ShenandoahGCTracer to gc/shenandoah
- Fix ordering of GC names in various files
- Rename UINT64_FORMAT_HEX_W to UINT64_FORMAT_X_W
http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/04/
Thanks everybody for taking time to review this!
Roman

Post by Roman Kennke
Hello all,
Thanks so far for all the reviews and support!
I forgot to update the 'round' yesterday. We are in round 3 now :-)
Also, I fixed the numbering of my webrevs to match the review-round. ;-)
- We moved shenandoah-specific code out of .ad files into our own .ad
files under gc/shenandoah (in shenandoah-gc), how cool is that? This
requires an addition in build machinery though, see
make/hotspot/gensrc/GensrcAdlc.gmk (in shared-build).
- Improved zero-disabling and build-code-simplification as suggested by
Magnus and Per
- Cleaned up some leftovers in C2
- Improved C2 loop opts code by introducing another APIs in
BarrierSetC2. See the new APIs in shared-gc under BarrierSetC2.hpp.
- I don't see where it makes sense to put INCLUDE_SHENANDOAHGC guards now.
- We would all very much prefer to keep ShenandoahXYZNode names, as
noted earlier. This stuff is Shenandoah-specific, so let's just call it
that.
- Rehashed Shenandoah tests (formatting, naming, directory layout, etc)
- Rebased on jdk-12+22
- Question: let us know if you need separate RFE for the new BSC2 APIs?
http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/03/
Thanks,
Roman

Post by Roman Kennke
- The minor issues that Kim reported in shared-gc
- A lot of fixes in shared-tests according to Leonid's review
- Disabled SA heapdumping similar to ZGC as Per suggested
Leonid:  test/hotspot/jtreg/gc/TestFullGCCount.java was actually
ExplicitGCInvokesConcurrent at the same time, because that would be
(expectedly) failing. It can run CMS, default GC and any other GC just
fine. Adding the same clause for Shenandoah means the same, and filters
the combination (+UseShenandoahGC)+(+ExplicitGCInvokesConcurrent). I
made the condition a bit clearer by avoiding triple-negation.
http://mail.openjdk.java.net/pipermail/shenandoah-dev/2018-November/008457.html
- test/hotspot/jtreg/serviceability/sa/ClhsdbJhisto.java
- test/hotspot/jtreg/serviceability/sa/TestHeapDumpForLargeArray.java
we filter them for Shenandoah now. I'm wondering: how do you get past
those with ZGC?
http://mail.openjdk.java.net/pipermail/shenandoah-dev/2018-November/008466.html
(Note to Leonid and tests reviewers: I'll add those related filters in
next round).
Vladimir: Roland integrated a bunch of changes to make loop* code look
better. I can tell that we're not done with C2 yet. Can you look over
the code and see what is ok, and especially what is not ok, so that we
can focus our efforts on the relevant parts?
http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/01/
Thanks,
Roman

Post by Roman Kennke
Hi,
This is the first round of changes for including Shenandoah GC into
mainline.
I divided the review into parts that roughly correspond to the mailing lists
that would normally review it, and I divided it into 'shared' code
changes and
'shenandoah' code changes (actually, mostly additions). The intend is to
eventually
push them as single 'combined' changeset, once reviewed.
   https://openjdk.java.net/jeps/189
  https://bugs.openjdk.java.net/browse/JDK-8214259
   http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/
For those who want to see the full change, have a look at the
shenandoah-complete
<http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shenandoah-complete/>
directory,
it contains the full combined webrev. Alternatively, there is the file
shenandoah-master.patch
<http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shenandoah-master.patch>,
which is what I intend to commit (and which should be equivalent to the
'shenandoah-complete' webrev).
  *) shenandoah-gc
<http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shenandoah-gc/>
     - Actual Shenandoah implementation, almost completely
residing in
gc/shenandoah
  *) shared-gc
<http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shared-gc/>
     - This is mostly boilerplate that is common to any GC
     - referenceProcessor.cpp has a little change to make one
assert not
fail (next to CMS and G1)
     - taskqueue.hpp has some small adjustments to enable subclassing
  *) shared-serviceability
<http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shared-serviceability/>
     - The usual code to support another GC
  *) shared-runtime
<http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shared-runtime/>
     - A number of friends declarations to allow Shenandoah
iterators to
hook up with,
       e.g. ClassLoaderData, CodeCache, etc
     - Warning and disabling JFR LeakProfiler
     - fieldDescriptor.hpp added is_stable() accessor, for use in
Shenandoah C2 optimizations
     - Locks initialization in mutexLocker.cpp as usual
     - VM operations defines for Shenandoah's VM ops
     - globalDefinitions.hpp added UINT64_FORMAT_HEX_W for use in
Shenandoah's logging
     - The usual macros in macro.hpp
  *) shared-build
<http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shared-build/>
     - Add shenandoah feature, enabled by default, as agreed with
Vladimir K. beforehand
     - Some flags for shenandoah-enabled compilation to get
SUPPORT_BARRIER_ON_PRIMITIVES
       and SUPPORT_NOT_TO_SPACE_INVARIANT which is required for
Shenandoah's barriers
     - --param inline-unit-growth=1000 settings for 2 shenandoah
source
files, which is
       useful to get the whole marking loop inlined (observed
significant
regression if we
       don't)
  *) shared-tests
<http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shared-tests/>
     - Test infrastructure to support Shenandoah
     - Shenandoah test groups
     - Exclude Shenandoah in various tests that can be run with
selected GC
     - Enable/add configure for Shenandoah for tests that make
sense to
run with it
  *) shenandoah-tests
<http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shenandoah-tests/>
     - Shenandoah specific tests, most reside in gc/shenandoah
subdirectory
     - A couple of tests configurations have been added, e.g.
TestGCBasherWithShenandoah.java
I intentionally left out shared-compiler for now, because we have some
work left to do
there, but if you click around you'll find the patch anyway, in case you
want to take
a peek at it.
   - {Linux} x {x86_64, x86_32, armhf, aarch64, ppc64el, s390x}
   - {Windows} x {x86_64},
   - {MacOS X} x {x86_64}
   - the new Shenandoah tests
   - jcstress with/without aggressive Shenandoah verification
   - specjvm2008 with/without aggressive Shenandoah verification
I'd like to thank my collegues at Red Hat: Christine Flood, she deserves
the credit for being the original inventor of Shenandoah, Aleksey
Shiplëv, Roland Westrelin & Zhengyu Gu for their countless
contributions, everybody else in Red Hat's OpenJDK team for testing,
advice and support, my collegues in Oracle's GC, runtime and compiler
teams for tirelessly helping with and reviewing all the GC interface and
related changes, and of course the many early adopters for reporting
bugs and success stories and feature requests: we wouldn't be here
without any of you!
Best regards,
Roman

[Roman Kennke]

Round 5 of Shenandoah review includes:
- A fix for the @requires tag in TestFullGCCountTest.java. It should be
correct now. We believe the CMS @requires was also not quite right and
fixed it the same.

It reads now: Don't run this test if:
- Actual GC set by harness is CMS *and* ExplicitGCInvokesConcurrent is
true, as set by harness
- Actual GC set by harness is Shenandoah *and*
ExplicitGCInvokesConcurrent is not set false by harness (it's true by
default in Shenandoah, so this needs to be double-inverteed).

The @requires for CMS was wrong before (we think), because it would also
filter defaultGC + ExplicitGCInvokesConcurrent.

- Sorting of macros was fixed, as was pointed out by Per
- Some stuff was added to SA, as suggested by Jini
- Rebased on most current jdk/jdk code

Webrevs:
http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/05/

I also need reviews from GC reviewers for the CSR:
https://bugs.openjdk.java.net/browse/JDK-8214349

I already got reviews for:
[x] shared-runtime (coleenp)
[x] shared-compiler (kvn)

I got reviews for shared-build, but an earlier version, so maybe makes
sense to look over this again. Erik J, Magnus?

I still need approvals for:
[ ] shared-build (kvn, erikj, ihse, pliden)
[ ] shared-gc (pliden, kbarrett)
[ ] shared-serviceability (jgeorge, pliden)
[ ] shared-tests (lmesnik, pliden)
[ ] shenandoah-gc
[ ] shenandoah-tests


Thanks for your patience and ongoing support!

Cheers,
Roman

Post by Roman Kennke
Hi all,
- Verify and gracefully reject dangerous flags combinations that
disables required barriers
- Trim unused code from Shenandoah's SA impl
- Move ShenandoahGCTracer to gc/shenandoah
- Fix ordering of GC names in various files
- Rename UINT64_FORMAT_HEX_W to UINT64_FORMAT_X_W
http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/04/
Thanks everybody for taking time to review this!
Roman

Post by Roman Kennke
Hello all,
Thanks so far for all the reviews and support!
I forgot to update the 'round' yesterday. We are in round 3 now :-)
Also, I fixed the numbering of my webrevs to match the review-round. ;-)
- We moved shenandoah-specific code out of .ad files into our own .ad
files under gc/shenandoah (in shenandoah-gc), how cool is that? This
requires an addition in build machinery though, see
make/hotspot/gensrc/GensrcAdlc.gmk (in shared-build).
- Improved zero-disabling and build-code-simplification as suggested by
Magnus and Per
- Cleaned up some leftovers in C2
- Improved C2 loop opts code by introducing another APIs in
BarrierSetC2. See the new APIs in shared-gc under BarrierSetC2.hpp.
- I don't see where it makes sense to put INCLUDE_SHENANDOAHGC guards now.
- We would all very much prefer to keep ShenandoahXYZNode names, as
noted earlier. This stuff is Shenandoah-specific, so let's just call it
that.
- Rehashed Shenandoah tests (formatting, naming, directory layout, etc)
- Rebased on jdk-12+22
- Question: let us know if you need separate RFE for the new BSC2 APIs?
http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/03/
Thanks,
Roman

Post by Roman Kennke
- The minor issues that Kim reported in shared-gc
- A lot of fixes in shared-tests according to Leonid's review
- Disabled SA heapdumping similar to ZGC as Per suggested
Leonid: test/hotspot/jtreg/gc/TestFullGCCount.java was actually
ExplicitGCInvokesConcurrent at the same time, because that would be
(expectedly) failing. It can run CMS, default GC and any other GC just
fine. Adding the same clause for Shenandoah means the same, and filters
the combination (+UseShenandoahGC)+(+ExplicitGCInvokesConcurrent). I
made the condition a bit clearer by avoiding triple-negation.
http://mail.openjdk.java.net/pipermail/shenandoah-dev/2018-November/008457.html
- test/hotspot/jtreg/serviceability/sa/ClhsdbJhisto.java
- test/hotspot/jtreg/serviceability/sa/TestHeapDumpForLargeArray.java
we filter them for Shenandoah now. I'm wondering: how do you get past
those with ZGC?
http://mail.openjdk.java.net/pipermail/shenandoah-dev/2018-November/008466.html
(Note to Leonid and tests reviewers: I'll add those related filters in
next round).
Vladimir: Roland integrated a bunch of changes to make loop* code look
better. I can tell that we're not done with C2 yet. Can you look over
the code and see what is ok, and especially what is not ok, so that we
can focus our efforts on the relevant parts?
http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/01/
Thanks,
Roman

Post by Roman Kennke
Hi,
This is the first round of changes for including Shenandoah GC into
mainline.
I divided the review into parts that roughly correspond to the mailing lists
that would normally review it, and I divided it into 'shared' code
changes and
'shenandoah' code changes (actually, mostly additions). The intend is to
eventually
push them as single 'combined' changeset, once reviewed.
  https://openjdk.java.net/jeps/189
 https://bugs.openjdk.java.net/browse/JDK-8214259
  http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/
For those who want to see the full change, have a look at the
shenandoah-complete
<http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shenandoah-complete/>
directory,
it contains the full combined webrev. Alternatively, there is the file
shenandoah-master.patch
<http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shenandoah-master.patch>,
which is what I intend to commit (and which should be equivalent to the
'shenandoah-complete' webrev).
 *) shenandoah-gc
<http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shenandoah-gc/>
    - Actual Shenandoah implementation, almost completely residing in
gc/shenandoah
 *) shared-gc
<http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shared-gc/>
    - This is mostly boilerplate that is common to any GC
    - referenceProcessor.cpp has a little change to make one assert not
fail (next to CMS and G1)
    - taskqueue.hpp has some small adjustments to enable subclassing
 *) shared-serviceability
<http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shared-serviceability/>
    - The usual code to support another GC
 *) shared-runtime
<http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shared-runtime/>
    - A number of friends declarations to allow Shenandoah iterators to
hook up with,
      e.g. ClassLoaderData, CodeCache, etc
    - Warning and disabling JFR LeakProfiler
    - fieldDescriptor.hpp added is_stable() accessor, for use in
Shenandoah C2 optimizations
    - Locks initialization in mutexLocker.cpp as usual
    - VM operations defines for Shenandoah's VM ops
    - globalDefinitions.hpp added UINT64_FORMAT_HEX_W for use in
Shenandoah's logging
    - The usual macros in macro.hpp
 *) shared-build
<http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shared-build/>
    - Add shenandoah feature, enabled by default, as agreed with
Vladimir K. beforehand
    - Some flags for shenandoah-enabled compilation to get
SUPPORT_BARRIER_ON_PRIMITIVES
      and SUPPORT_NOT_TO_SPACE_INVARIANT which is required for
Shenandoah's barriers
    - --param inline-unit-growth=1000 settings for 2 shenandoah source
files, which is
      useful to get the whole marking loop inlined (observed significant
regression if we
      don't)
 *) shared-tests
<http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shared-tests/>
    - Test infrastructure to support Shenandoah
    - Shenandoah test groups
    - Exclude Shenandoah in various tests that can be run with selected GC
    - Enable/add configure for Shenandoah for tests that make sense to
run with it
 *) shenandoah-tests
<http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shenandoah-tests/>
    - Shenandoah specific tests, most reside in gc/shenandoah subdirectory
    - A couple of tests configurations have been added, e.g.
TestGCBasherWithShenandoah.java
I intentionally left out shared-compiler for now, because we have some
work left to do
there, but if you click around you'll find the patch anyway, in case you
want to take
a peek at it.
  - {Linux} x {x86_64, x86_32, armhf, aarch64, ppc64el, s390x}
  - {Windows} x {x86_64},
  - {MacOS X} x {x86_64}
  - the new Shenandoah tests
  - jcstress with/without aggressive Shenandoah verification
  - specjvm2008 with/without aggressive Shenandoah verification
I'd like to thank my collegues at Red Hat: Christine Flood, she deserves
the credit for being the original inventor of Shenandoah, Aleksey
Shiplëv, Roland Westrelin & Zhengyu Gu for their countless
contributions, everybody else in Red Hat's OpenJDK team for testing,
advice and support, my collegues in Oracle's GC, runtime and compiler
teams for tirelessly helping with and reviewing all the GC interface and
related changes, and of course the many early adopters for reporting
bugs and success stories and feature requests: we wouldn't be here
without any of you!
Best regards,
Roman

[Leonid Mesnik]

Hi

The shared tests changes looks good for me. Thank you for fixing and testing different combinations.

Leonid

Post by Roman Kennke
fixed it the same.
- Actual GC set by harness is CMS *and* ExplicitGCInvokesConcurrent is
true, as set by harness
- Actual GC set by harness is Shenandoah *and*
ExplicitGCInvokesConcurrent is not set false by harness (it's true by
default in Shenandoah, so this needs to be double-inverteed).
filter defaultGC + ExplicitGCInvokesConcurrent.
- Sorting of macros was fixed, as was pointed out by Per
- Some stuff was added to SA, as suggested by Jini
- Rebased on most current jdk/jdk code
http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/05/
https://bugs.openjdk.java.net/browse/JDK-8214349
[x] shared-runtime (coleenp)
[x] shared-compiler (kvn)
I got reviews for shared-build, but an earlier version, so maybe makes
sense to look over this again. Erik J, Magnus?
[ ] shared-build (kvn, erikj, ihse, pliden)
[ ] shared-gc (pliden, kbarrett)
[ ] shared-serviceability (jgeorge, pliden)
[ ] shared-tests (lmesnik, pliden)
[ ] shenandoah-gc
[ ] shenandoah-tests
Thanks for your patience and ongoing support!
Cheers,
Roman

Post by Roman Kennke
Hi all,
- Verify and gracefully reject dangerous flags combinations that
disables required barriers
- Trim unused code from Shenandoah's SA impl
- Move ShenandoahGCTracer to gc/shenandoah
- Fix ordering of GC names in various files
- Rename UINT64_FORMAT_HEX_W to UINT64_FORMAT_X_W
http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/04/
Thanks everybody for taking time to review this!
Roman

Post by Roman Kennke
Hello all,
Thanks so far for all the reviews and support!
I forgot to update the 'round' yesterday. We are in round 3 now :-)
Also, I fixed the numbering of my webrevs to match the review-round. ;-)
- We moved shenandoah-specific code out of .ad files into our own .ad
files under gc/shenandoah (in shenandoah-gc), how cool is that? This
requires an addition in build machinery though, see
make/hotspot/gensrc/GensrcAdlc.gmk (in shared-build).
- Improved zero-disabling and build-code-simplification as suggested by
Magnus and Per
- Cleaned up some leftovers in C2
- Improved C2 loop opts code by introducing another APIs in
BarrierSetC2. See the new APIs in shared-gc under BarrierSetC2.hpp.
- I don't see where it makes sense to put INCLUDE_SHENANDOAHGC guards now.
- We would all very much prefer to keep ShenandoahXYZNode names, as
noted earlier. This stuff is Shenandoah-specific, so let's just call it
that.
- Rehashed Shenandoah tests (formatting, naming, directory layout, etc)
- Rebased on jdk-12+22
- Question: let us know if you need separate RFE for the new BSC2 APIs?
http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/03/
Thanks,
Roman

Post by Roman Kennke
- The minor issues that Kim reported in shared-gc
- A lot of fixes in shared-tests according to Leonid's review
- Disabled SA heapdumping similar to ZGC as Per suggested
Leonid: test/hotspot/jtreg/gc/TestFullGCCount.java was actually
ExplicitGCInvokesConcurrent at the same time, because that would be
(expectedly) failing. It can run CMS, default GC and any other GC just
fine. Adding the same clause for Shenandoah means the same, and filters
the combination (+UseShenandoahGC)+(+ExplicitGCInvokesConcurrent). I
made the condition a bit clearer by avoiding triple-negation.
http://mail.openjdk.java.net/pipermail/shenandoah-dev/2018-November/008457.html
- test/hotspot/jtreg/serviceability/sa/ClhsdbJhisto.java
- test/hotspot/jtreg/serviceability/sa/TestHeapDumpForLargeArray.java
we filter them for Shenandoah now. I'm wondering: how do you get past
those with ZGC?
http://mail.openjdk.java.net/pipermail/shenandoah-dev/2018-November/008466.html
(Note to Leonid and tests reviewers: I'll add those related filters in
next round).
Vladimir: Roland integrated a bunch of changes to make loop* code look
better. I can tell that we're not done with C2 yet. Can you look over
the code and see what is ok, and especially what is not ok, so that we
can focus our efforts on the relevant parts?
http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/01/
Thanks,
Roman

Post by Roman Kennke
Hi,
This is the first round of changes for including Shenandoah GC into
mainline.
I divided the review into parts that roughly correspond to the mailing lists
that would normally review it, and I divided it into 'shared' code
changes and
'shenandoah' code changes (actually, mostly additions). The intend is to
eventually
push them as single 'combined' changeset, once reviewed.
https://openjdk.java.net/jeps/189
https://bugs.openjdk.java.net/browse/JDK-8214259
http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/
For those who want to see the full change, have a look at the
shenandoah-complete
<http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shenandoah-complete/>
directory,
it contains the full combined webrev. Alternatively, there is the file
shenandoah-master.patch
<http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shenandoah-master.patch>,
which is what I intend to commit (and which should be equivalent to the
'shenandoah-complete' webrev).
*) shenandoah-gc
<http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shenandoah-gc/>
- Actual Shenandoah implementation, almost completely residing in
gc/shenandoah
*) shared-gc
<http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shared-gc/>
- This is mostly boilerplate that is common to any GC
- referenceProcessor.cpp has a little change to make one assert not
fail (next to CMS and G1)
- taskqueue.hpp has some small adjustments to enable subclassing
*) shared-serviceability
<http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shared-serviceability/>
- The usual code to support another GC
*) shared-runtime
<http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shared-runtime/>
- A number of friends declarations to allow Shenandoah iterators to
hook up with,
e.g. ClassLoaderData, CodeCache, etc
- Warning and disabling JFR LeakProfiler
- fieldDescriptor.hpp added is_stable() accessor, for use in
Shenandoah C2 optimizations
- Locks initialization in mutexLocker.cpp as usual
- VM operations defines for Shenandoah's VM ops
- globalDefinitions.hpp added UINT64_FORMAT_HEX_W for use in
Shenandoah's logging
- The usual macros in macro.hpp
*) shared-build
<http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shared-build/>
- Add shenandoah feature, enabled by default, as agreed with
Vladimir K. beforehand
- Some flags for shenandoah-enabled compilation to get
SUPPORT_BARRIER_ON_PRIMITIVES
and SUPPORT_NOT_TO_SPACE_INVARIANT which is required for
Shenandoah's barriers
- --param inline-unit-growth=1000 settings for 2 shenandoah source
files, which is
useful to get the whole marking loop inlined (observed significant
regression if we
don't)
*) shared-tests
<http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shared-tests/>
- Test infrastructure to support Shenandoah
- Shenandoah test groups
- Exclude Shenandoah in various tests that can be run with selected GC
- Enable/add configure for Shenandoah for tests that make sense to
run with it
*) shenandoah-tests
<http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shenandoah-tests/>
- Shenandoah specific tests, most reside in gc/shenandoah subdirectory
- A couple of tests configurations have been added, e.g.
TestGCBasherWithShenandoah.java
I intentionally left out shared-compiler for now, because we have some
work left to do
there, but if you click around you'll find the patch anyway, in case you
want to take
a peek at it.
- {Linux} x {x86_64, x86_32, armhf, aarch64, ppc64el, s390x}
- {Windows} x {x86_64},
- {MacOS X} x {x86_64}
- the new Shenandoah tests
- jcstress with/without aggressive Shenandoah verification
- specjvm2008 with/without aggressive Shenandoah verification
I'd like to thank my collegues at Red Hat: Christine Flood, she deserves
the credit for being the original inventor of Shenandoah, Aleksey
Shiplëv, Roland Westrelin & Zhengyu Gu for their countless
contributions, everybody else in Red Hat's OpenJDK team for testing,
advice and support, my collegues in Oracle's GC, runtime and compiler
teams for tirelessly helping with and reviewing all the GC interface and
related changes, and of course the many early adopters for reporting
bugs and success stories and feature requests: we wouldn't be here
without any of you!
Best regards,
Roman

[Roman Kennke]

Thanks, Leonid, for reviewing!

Roman

Post by Leonid Mesnik
Hi
The shared tests changes looks good for me. Thank you for fixing and testing different combinations.
Leonid

Post by Roman Kennke
fixed it the same.
- Actual GC set by harness is CMS *and* ExplicitGCInvokesConcurrent is
true, as set by harness
- Actual GC set by harness is Shenandoah *and*
ExplicitGCInvokesConcurrent is not set false by harness (it's true by
default in Shenandoah, so this needs to be double-inverteed).
filter defaultGC + ExplicitGCInvokesConcurrent.
- Sorting of macros was fixed, as was pointed out by Per
- Some stuff was added to SA, as suggested by Jini
- Rebased on most current jdk/jdk code
http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/05/
https://bugs.openjdk.java.net/browse/JDK-8214349
[x] shared-runtime (coleenp)
[x] shared-compiler (kvn)
I got reviews for shared-build, but an earlier version, so maybe makes
sense to look over this again. Erik J, Magnus?
[ ] shared-build (kvn, erikj, ihse, pliden)
[ ] shared-gc (pliden, kbarrett)
[ ] shared-serviceability (jgeorge, pliden)
[ ] shared-tests (lmesnik, pliden)
[ ] shenandoah-gc
[ ] shenandoah-tests
Thanks for your patience and ongoing support!
Cheers,
Roman

Post by Roman Kennke
Hi all,
- Verify and gracefully reject dangerous flags combinations that
disables required barriers
- Trim unused code from Shenandoah's SA impl
- Move ShenandoahGCTracer to gc/shenandoah
- Fix ordering of GC names in various files
- Rename UINT64_FORMAT_HEX_W to UINT64_FORMAT_X_W
http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/04/
Thanks everybody for taking time to review this!
Roman

Post by Roman Kennke
Hello all,
Thanks so far for all the reviews and support!
I forgot to update the 'round' yesterday. We are in round 3 now :-)
Also, I fixed the numbering of my webrevs to match the review-round. ;-)
- We moved shenandoah-specific code out of .ad files into our own .ad
files under gc/shenandoah (in shenandoah-gc), how cool is that? This
requires an addition in build machinery though, see
make/hotspot/gensrc/GensrcAdlc.gmk (in shared-build).
- Improved zero-disabling and build-code-simplification as suggested by
Magnus and Per
- Cleaned up some leftovers in C2
- Improved C2 loop opts code by introducing another APIs in
BarrierSetC2. See the new APIs in shared-gc under BarrierSetC2.hpp.
- I don't see where it makes sense to put INCLUDE_SHENANDOAHGC guards now.
- We would all very much prefer to keep ShenandoahXYZNode names, as
noted earlier. This stuff is Shenandoah-specific, so let's just call it
that.
- Rehashed Shenandoah tests (formatting, naming, directory layout, etc)
- Rebased on jdk-12+22
- Question: let us know if you need separate RFE for the new BSC2 APIs?
http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/03/
Thanks,
Roman

Post by Roman Kennke
- The minor issues that Kim reported in shared-gc
- A lot of fixes in shared-tests according to Leonid's review
- Disabled SA heapdumping similar to ZGC as Per suggested
Leonid: test/hotspot/jtreg/gc/TestFullGCCount.java was actually
ExplicitGCInvokesConcurrent at the same time, because that would be
(expectedly) failing. It can run CMS, default GC and any other GC just
fine. Adding the same clause for Shenandoah means the same, and filters
the combination (+UseShenandoahGC)+(+ExplicitGCInvokesConcurrent). I
made the condition a bit clearer by avoiding triple-negation.
http://mail.openjdk.java.net/pipermail/shenandoah-dev/2018-November/008457.html
- test/hotspot/jtreg/serviceability/sa/ClhsdbJhisto.java
- test/hotspot/jtreg/serviceability/sa/TestHeapDumpForLargeArray.java
we filter them for Shenandoah now. I'm wondering: how do you get past
those with ZGC?
http://mail.openjdk.java.net/pipermail/shenandoah-dev/2018-November/008466.html
(Note to Leonid and tests reviewers: I'll add those related filters in
next round).
Vladimir: Roland integrated a bunch of changes to make loop* code look
better. I can tell that we're not done with C2 yet. Can you look over
the code and see what is ok, and especially what is not ok, so that we
can focus our efforts on the relevant parts?
http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/01/
Thanks,
Roman

Post by Roman Kennke
Hi,
This is the first round of changes for including Shenandoah GC into
mainline.
I divided the review into parts that roughly correspond to the mailing lists
that would normally review it, and I divided it into 'shared' code
changes and
'shenandoah' code changes (actually, mostly additions). The intend is to
eventually
push them as single 'combined' changeset, once reviewed.
https://openjdk.java.net/jeps/189
https://bugs.openjdk.java.net/browse/JDK-8214259
http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/
For those who want to see the full change, have a look at the
shenandoah-complete
<http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shenandoah-complete/>
directory,
it contains the full combined webrev. Alternatively, there is the file
shenandoah-master.patch
<http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shenandoah-master.patch>,
which is what I intend to commit (and which should be equivalent to the
'shenandoah-complete' webrev).
*) shenandoah-gc
<http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shenandoah-gc/>
- Actual Shenandoah implementation, almost completely residing in
gc/shenandoah
*) shared-gc
<http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shared-gc/>
- This is mostly boilerplate that is common to any GC
- referenceProcessor.cpp has a little change to make one assert not
fail (next to CMS and G1)
- taskqueue.hpp has some small adjustments to enable subclassing
*) shared-serviceability
<http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shared-serviceability/>
- The usual code to support another GC
*) shared-runtime
<http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shared-runtime/>
- A number of friends declarations to allow Shenandoah iterators to
hook up with,
e.g. ClassLoaderData, CodeCache, etc
- Warning and disabling JFR LeakProfiler
- fieldDescriptor.hpp added is_stable() accessor, for use in
Shenandoah C2 optimizations
- Locks initialization in mutexLocker.cpp as usual
- VM operations defines for Shenandoah's VM ops
- globalDefinitions.hpp added UINT64_FORMAT_HEX_W for use in
Shenandoah's logging
- The usual macros in macro.hpp
*) shared-build
<http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shared-build/>
- Add shenandoah feature, enabled by default, as agreed with
Vladimir K. beforehand
- Some flags for shenandoah-enabled compilation to get
SUPPORT_BARRIER_ON_PRIMITIVES
and SUPPORT_NOT_TO_SPACE_INVARIANT which is required for
Shenandoah's barriers
- --param inline-unit-growth=1000 settings for 2 shenandoah source
files, which is
useful to get the whole marking loop inlined (observed significant
regression if we
don't)
*) shared-tests
<http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shared-tests/>
- Test infrastructure to support Shenandoah
- Shenandoah test groups
- Exclude Shenandoah in various tests that can be run with selected GC
- Enable/add configure for Shenandoah for tests that make sense to
run with it
*) shenandoah-tests
<http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shenandoah-tests/>
- Shenandoah specific tests, most reside in gc/shenandoah subdirectory
- A couple of tests configurations have been added, e.g.
TestGCBasherWithShenandoah.java
I intentionally left out shared-compiler for now, because we have some
work left to do
there, but if you click around you'll find the patch anyway, in case you
want to take
a peek at it.
- {Linux} x {x86_64, x86_32, armhf, aarch64, ppc64el, s390x}
- {Windows} x {x86_64},
- {MacOS X} x {x86_64}
- the new Shenandoah tests
- jcstress with/without aggressive Shenandoah verification
- specjvm2008 with/without aggressive Shenandoah verification
I'd like to thank my collegues at Red Hat: Christine Flood, she deserves
the credit for being the original inventor of Shenandoah, Aleksey
Shiplëv, Roland Westrelin & Zhengyu Gu for their countless
contributions, everybody else in Red Hat's OpenJDK team for testing,
advice and support, my collegues in Oracle's GC, runtime and compiler
teams for tirelessly helping with and reviewing all the GC interface and
related changes, and of course the many early adopters for reporting
bugs and success stories and feature requests: we wouldn't be here
without any of you!
Best regards,
Roman

[c***@oracle.com]

http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/05/shenandoah-gc/src/hotspot/share/gc/shenandoah/vm_operations_shenandoah.cpp.html

Can you rename these to shenandoahVMOperations.hpp/cpp to match the
newly agreed upon naming convention for this?

See 8214791: Consistently name gc files containing VM operations [Was:
Re: RFR (S): 8214791: Rename vm_operations_g1* files to g1VMOperations*]

thanks,
Coleen

Post by Roman Kennke
Thanks, Leonid, for reviewing!
Roman

Post by Leonid Mesnik
Hi
The shared tests changes looks good for me. Thank you for fixing and testing different combinations.
Leonid

Post by Roman Kennke
fixed it the same.
- Actual GC set by harness is CMS *and* ExplicitGCInvokesConcurrent is
true, as set by harness
- Actual GC set by harness is Shenandoah *and*
ExplicitGCInvokesConcurrent is not set false by harness (it's true by
default in Shenandoah, so this needs to be double-inverteed).
filter defaultGC + ExplicitGCInvokesConcurrent.
- Sorting of macros was fixed, as was pointed out by Per
- Some stuff was added to SA, as suggested by Jini
- Rebased on most current jdk/jdk code
http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/05/
https://bugs.openjdk.java.net/browse/JDK-8214349
[x] shared-runtime (coleenp)
[x] shared-compiler (kvn)
I got reviews for shared-build, but an earlier version, so maybe makes
sense to look over this again. Erik J, Magnus?
[ ] shared-build (kvn, erikj, ihse, pliden)
[ ] shared-gc (pliden, kbarrett)
[ ] shared-serviceability (jgeorge, pliden)
[ ] shared-tests (lmesnik, pliden)
[ ] shenandoah-gc
[ ] shenandoah-tests
Thanks for your patience and ongoing support!
Cheers,
Roman

Post by Roman Kennke
Hi all,
- Verify and gracefully reject dangerous flags combinations that
disables required barriers
- Trim unused code from Shenandoah's SA impl
- Move ShenandoahGCTracer to gc/shenandoah
- Fix ordering of GC names in various files
- Rename UINT64_FORMAT_HEX_W to UINT64_FORMAT_X_W
http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/04/
Thanks everybody for taking time to review this!
Roman

Post by Roman Kennke
Hello all,
Thanks so far for all the reviews and support!
I forgot to update the 'round' yesterday. We are in round 3 now :-)
Also, I fixed the numbering of my webrevs to match the review-round. ;-)
- We moved shenandoah-specific code out of .ad files into our own .ad
files under gc/shenandoah (in shenandoah-gc), how cool is that? This
requires an addition in build machinery though, see
make/hotspot/gensrc/GensrcAdlc.gmk (in shared-build).
- Improved zero-disabling and build-code-simplification as suggested by
Magnus and Per
- Cleaned up some leftovers in C2
- Improved C2 loop opts code by introducing another APIs in
BarrierSetC2. See the new APIs in shared-gc under BarrierSetC2.hpp.
- I don't see where it makes sense to put INCLUDE_SHENANDOAHGC guards now.
- We would all very much prefer to keep ShenandoahXYZNode names, as
noted earlier. This stuff is Shenandoah-specific, so let's just call it
that.
- Rehashed Shenandoah tests (formatting, naming, directory layout, etc)
- Rebased on jdk-12+22
- Question: let us know if you need separate RFE for the new BSC2 APIs?
http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/03/
Thanks,
Roman

Post by Roman Kennke
- The minor issues that Kim reported in shared-gc
- A lot of fixes in shared-tests according to Leonid's review
- Disabled SA heapdumping similar to ZGC as Per suggested
Leonid: test/hotspot/jtreg/gc/TestFullGCCount.java was actually
ExplicitGCInvokesConcurrent at the same time, because that would be
(expectedly) failing. It can run CMS, default GC and any other GC just
fine. Adding the same clause for Shenandoah means the same, and filters
the combination (+UseShenandoahGC)+(+ExplicitGCInvokesConcurrent). I
made the condition a bit clearer by avoiding triple-negation.
http://mail.openjdk.java.net/pipermail/shenandoah-dev/2018-November/008457.html
- test/hotspot/jtreg/serviceability/sa/ClhsdbJhisto.java
- test/hotspot/jtreg/serviceability/sa/TestHeapDumpForLargeArray.java
we filter them for Shenandoah now. I'm wondering: how do you get past
those with ZGC?
http://mail.openjdk.java.net/pipermail/shenandoah-dev/2018-November/008466.html
(Note to Leonid and tests reviewers: I'll add those related filters in
next round).
Vladimir: Roland integrated a bunch of changes to make loop* code look
better. I can tell that we're not done with C2 yet. Can you look over
the code and see what is ok, and especially what is not ok, so that we
can focus our efforts on the relevant parts?
http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/01/
Thanks,
Roman

Post by Roman Kennke
Hi,
This is the first round of changes for including Shenandoah GC into
mainline.
I divided the review into parts that roughly correspond to the mailing lists
that would normally review it, and I divided it into 'shared' code
changes and
'shenandoah' code changes (actually, mostly additions). The intend is to
eventually
push them as single 'combined' changeset, once reviewed.
https://openjdk.java.net/jeps/189
https://bugs.openjdk.java.net/browse/JDK-8214259
http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/
For those who want to see the full change, have a look at the
shenandoah-complete
<http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shenandoah-complete/>
directory,
it contains the full combined webrev. Alternatively, there is the file
shenandoah-master.patch
<http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shenandoah-master.patch>,
which is what I intend to commit (and which should be equivalent to the
'shenandoah-complete' webrev).
*) shenandoah-gc
<http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shenandoah-gc/>
- Actual Shenandoah implementation, almost completely residing in
gc/shenandoah
*) shared-gc
<http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shared-gc/>
- This is mostly boilerplate that is common to any GC
- referenceProcessor.cpp has a little change to make one assert not
fail (next to CMS and G1)
- taskqueue.hpp has some small adjustments to enable subclassing
*) shared-serviceability
<http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shared-serviceability/>
- The usual code to support another GC
*) shared-runtime
<http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shared-runtime/>
- A number of friends declarations to allow Shenandoah iterators to
hook up with,
e.g. ClassLoaderData, CodeCache, etc
- Warning and disabling JFR LeakProfiler
- fieldDescriptor.hpp added is_stable() accessor, for use in
Shenandoah C2 optimizations
- Locks initialization in mutexLocker.cpp as usual
- VM operations defines for Shenandoah's VM ops
- globalDefinitions.hpp added UINT64_FORMAT_HEX_W for use in
Shenandoah's logging
- The usual macros in macro.hpp
*) shared-build
<http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shared-build/>
- Add shenandoah feature, enabled by default, as agreed with
Vladimir K. beforehand
- Some flags for shenandoah-enabled compilation to get
SUPPORT_BARRIER_ON_PRIMITIVES
and SUPPORT_NOT_TO_SPACE_INVARIANT which is required for
Shenandoah's barriers
- --param inline-unit-growth=1000 settings for 2 shenandoah source
files, which is
useful to get the whole marking loop inlined (observed significant
regression if we
don't)
*) shared-tests
<http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shared-tests/>
- Test infrastructure to support Shenandoah
- Shenandoah test groups
- Exclude Shenandoah in various tests that can be run with selected GC
- Enable/add configure for Shenandoah for tests that make sense to
run with it
*) shenandoah-tests
<http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shenandoah-tests/>
- Shenandoah specific tests, most reside in gc/shenandoah subdirectory
- A couple of tests configurations have been added, e.g.
TestGCBasherWithShenandoah.java
I intentionally left out shared-compiler for now, because we have some
work left to do
there, but if you click around you'll find the patch anyway, in case you
want to take
a peek at it.
- {Linux} x {x86_64, x86_32, armhf, aarch64, ppc64el, s390x}
- {Windows} x {x86_64},
- {MacOS X} x {x86_64}
- the new Shenandoah tests
- jcstress with/without aggressive Shenandoah verification
- specjvm2008 with/without aggressive Shenandoah verification
I'd like to thank my collegues at Red Hat: Christine Flood, she deserves
the credit for being the original inventor of Shenandoah, Aleksey
Shiplëv, Roland Westrelin & Zhengyu Gu for their countless
contributions, everybody else in Red Hat's OpenJDK team for testing,
advice and support, my collegues in Oracle's GC, runtime and compiler
teams for tirelessly helping with and reviewing all the GC interface and
related changes, and of course the many early adopters for reporting
bugs and success stories and feature requests: we wouldn't be here
without any of you!
Best regards,
Roman

[Roman Kennke]

Hi Coleen,

Post by c***@oracle.com
http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/05/shenandoah-gc/src/hotspot/share/gc/shenandoah/vm_operations_shenandoah.cpp.html
Can you rename these to shenandoahVMOperations.hpp/cpp to match the
newly agreed upon naming convention for this?
Re: RFR (S): 8214791: Rename vm_operations_g1* files to g1VMOperations*]

Doing so:
http://mail.openjdk.java.net/pipermail/shenandoah-dev/2018-December/008595.html

I will integrate this in next round of webrevs.

I expect the next will be the final round of webrevs. I got positive
reviews for all shared-* parts, I'm waiting for shenandoah-* reviews
from Shenandoah devs, plus Zhengyu+Thomas' TaskQueue stuff to arrive in
upstream jdk. The CSR for Shenandoah flags has been approved, and the
JEP should be moved to targeted JDK12 ~tomorrow. I intend/expect to push
Shenandoah during the next couple of days, unless somebody speaks up
until then :-)

Thanks,
Roman

Post by c***@oracle.com
thanks,
Coleen

Post by Roman Kennke
Thanks, Leonid, for reviewing!
Roman

Post by Leonid Mesnik
Hi
The shared tests changes looks good for me. Thank you for fixing and
testing different combinations.
Leonid

Post by Roman Kennke
fixed it the same.
- Actual GC set by harness is CMS *and* ExplicitGCInvokesConcurrent is
true, as set by harness
- Actual GC set by harness is Shenandoah *and*
ExplicitGCInvokesConcurrent is not set false by harness (it's true by
default in Shenandoah, so this needs to be double-inverteed).
filter defaultGC + ExplicitGCInvokesConcurrent.
- Sorting of macros was fixed, as was pointed out by Per
- Some stuff was added to SA, as suggested by Jini
- Rebased on most current jdk/jdk code
http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/05/
https://bugs.openjdk.java.net/browse/JDK-8214349
[x] shared-runtime (coleenp)
[x] shared-compiler (kvn)
I got reviews for shared-build, but an earlier version, so maybe makes
sense to look over this again. Erik J, Magnus?
[ ] shared-build          (kvn, erikj, ihse, pliden)
[ ] shared-gc             (pliden, kbarrett)
[ ] shared-serviceability (jgeorge, pliden)
[ ] shared-tests          (lmesnik, pliden)
[ ] shenandoah-gc
[ ] shenandoah-tests
Thanks for your patience and ongoing support!
Cheers,
Roman

Post by Roman Kennke
Hi all,
- Verify and gracefully reject dangerous flags combinations that
disables required barriers
- Trim unused code from Shenandoah's SA impl
- Move ShenandoahGCTracer to gc/shenandoah
- Fix ordering of GC names in various files
- Rename UINT64_FORMAT_HEX_W to UINT64_FORMAT_X_W
http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/04/
Thanks everybody for taking time to review this!
Roman

Post by Roman Kennke
Hello all,
Thanks so far for all the reviews and support!
I forgot to update the 'round' yesterday. We are in round 3 now :-)
Also, I fixed the numbering of my webrevs to match the
review-round. ;-)
- We moved shenandoah-specific code out of .ad files into our own .ad
files under gc/shenandoah (in shenandoah-gc), how cool is that? This
requires an addition in build machinery though, see
make/hotspot/gensrc/GensrcAdlc.gmk (in shared-build).
- Improved zero-disabling and build-code-simplification as suggested by
Magnus and Per
- Cleaned up some leftovers in C2
- Improved C2 loop opts code by introducing another APIs in
BarrierSetC2. See the new APIs in shared-gc under BarrierSetC2.hpp.
- I don't see where it makes sense to put INCLUDE_SHENANDOAHGC guards now.
- We would all very much prefer to keep ShenandoahXYZNode names, as
noted earlier. This stuff is Shenandoah-specific, so let's just call it
that.
- Rehashed Shenandoah tests (formatting, naming, directory layout, etc)
- Rebased on jdk-12+22
- Question: let us know if you need separate RFE for the new BSC2 APIs?
http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/03/
Thanks,
Roman

Post by Roman Kennke
- The minor issues that Kim reported in shared-gc
- A lot of fixes in shared-tests according to Leonid's review
- Disabled SA heapdumping similar to ZGC as Per suggested
Leonid:  test/hotspot/jtreg/gc/TestFullGCCount.java was actually
ExplicitGCInvokesConcurrent at the same time, because that would be
(expectedly) failing. It can run CMS, default GC and any other GC just
fine. Adding the same clause for Shenandoah means the same, and filters
the combination (+UseShenandoahGC)+(+ExplicitGCInvokesConcurrent). I
made the condition a bit clearer by avoiding triple-negation.
http://mail.openjdk.java.net/pipermail/shenandoah-dev/2018-November/008457.html
- test/hotspot/jtreg/serviceability/sa/ClhsdbJhisto.java
-
test/hotspot/jtreg/serviceability/sa/TestHeapDumpForLargeArray.java
we filter them for Shenandoah now. I'm wondering: how do you get past
those with ZGC?
http://mail.openjdk.java.net/pipermail/shenandoah-dev/2018-November/008466.html
(Note to Leonid and tests reviewers: I'll add those related filters in
next round).
Vladimir: Roland integrated a bunch of changes to make loop* code look
better. I can tell that we're not done with C2 yet. Can you look over
the code and see what is ok, and especially what is not ok, so that we
can focus our efforts on the relevant parts?
http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/01/
Thanks,
Roman

Post by Roman Kennke
Hi,
This is the first round of changes for including Shenandoah GC into
mainline.
I divided the review into parts that roughly correspond to the mailing lists
that would normally review it, and I divided it into 'shared' code
changes and
'shenandoah' code changes (actually, mostly additions). The intend is to
eventually
push them as single 'combined' changeset, once reviewed.
   https://openjdk.java.net/jeps/189
  https://bugs.openjdk.java.net/browse/JDK-8214259
   http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/
For those who want to see the full change, have a look at the
shenandoah-complete
<http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shenandoah-complete/>
directory,
it contains the full combined webrev. Alternatively, there is the file
shenandoah-master.patch
<http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shenandoah-master.patch>,
which is what I intend to commit (and which should be equivalent to the
'shenandoah-complete' webrev).
  *) shenandoah-gc
<http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shenandoah-gc/>
     - Actual Shenandoah implementation, almost completely
residing in
gc/shenandoah
  *) shared-gc
<http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shared-gc/>
     - This is mostly boilerplate that is common to any GC
     - referenceProcessor.cpp has a little change to make one
assert not
fail (next to CMS and G1)
     - taskqueue.hpp has some small adjustments to enable
subclassing
  *) shared-serviceability
<http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shared-serviceability/>
     - The usual code to support another GC
  *) shared-runtime
<http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shared-runtime/>
     - A number of friends declarations to allow Shenandoah
iterators to
hook up with,
       e.g. ClassLoaderData, CodeCache, etc
     - Warning and disabling JFR LeakProfiler
     - fieldDescriptor.hpp added is_stable() accessor, for use in
Shenandoah C2 optimizations
     - Locks initialization in mutexLocker.cpp as usual
     - VM operations defines for Shenandoah's VM ops
     - globalDefinitions.hpp added UINT64_FORMAT_HEX_W for use in
Shenandoah's logging
     - The usual macros in macro.hpp
  *) shared-build
<http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shared-build/>
     - Add shenandoah feature, enabled by default, as agreed with
Vladimir K. beforehand
     - Some flags for shenandoah-enabled compilation to get
SUPPORT_BARRIER_ON_PRIMITIVES
       and SUPPORT_NOT_TO_SPACE_INVARIANT which is required for
Shenandoah's barriers
     - --param inline-unit-growth=1000 settings for 2 shenandoah
source
files, which is
       useful to get the whole marking loop inlined (observed
significant
regression if we
       don't)
  *) shared-tests
<http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shared-tests/>
     - Test infrastructure to support Shenandoah
     - Shenandoah test groups
     - Exclude Shenandoah in various tests that can be run with
selected GC
     - Enable/add configure for Shenandoah for tests that make
sense to
run with it
  *) shenandoah-tests
<http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shenandoah-tests/>
     - Shenandoah specific tests, most reside in gc/shenandoah
subdirectory
     - A couple of tests configurations have been added, e.g.
TestGCBasherWithShenandoah.java
I intentionally left out shared-compiler for now, because we have some
work left to do
there, but if you click around you'll find the patch anyway, in case you
want to take
a peek at it.
   - {Linux} x {x86_64, x86_32, armhf, aarch64, ppc64el, s390x}
   - {Windows} x {x86_64},
   - {MacOS X} x {x86_64}
   - the new Shenandoah tests
   - jcstress with/without aggressive Shenandoah verification
   - specjvm2008 with/without aggressive Shenandoah verification
I'd like to thank my collegues at Red Hat: Christine Flood, she deserves
the credit for being the original inventor of Shenandoah, Aleksey
Shiplëv, Roland Westrelin & Zhengyu Gu for their countless
contributions, everybody else in Red Hat's OpenJDK team for testing,
advice and support, my collegues in Oracle's GC, runtime and compiler
teams for tirelessly helping with and reviewing all the GC interface and
related changes, and of course the many early adopters for reporting
bugs and success stories and feature requests: we wouldn't be here
without any of you!
Best regards,
Roman

[Aleksey Shipilev]

Post by Roman Kennke
http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/05/
[ ] shenandoah-gc
[ ] shenandoah-tests

These two parts look good.

-Aleksey

[Roman Kennke]

Here comes round 6, possibly/hopefully the last round of webrevs for
upstreaming Shenandoah:

http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/06/

It incorporates:
- renames vm_operations_shenandoah* to shenandoahVMOperations*, as
suggested by Coleen
- reshuffles gcCause in shared-gc SA as suggested by Per
- print number of threads in SA, as suggested by Jini
- fixed a problem in webrev generation that did not track move of
CriticalNativeStress.java and CriticalNativeArgs.java

The CSR has been approved, the JEP moved to target jdk12, and I got
positive reviews for all parts. I intend to push this early next week,
unless somebody stops me. If Zhengyu's and Thomas' TaskQueue change goes
in by then, I'll incorporate it.

Thanks everybody for reviewing and reviewing and reviewing again ;-)

Cheers,
Roman

Post by Roman Kennke
fixed it the same.
- Actual GC set by harness is CMS *and* ExplicitGCInvokesConcurrent is
true, as set by harness
- Actual GC set by harness is Shenandoah *and*
ExplicitGCInvokesConcurrent is not set false by harness (it's true by
default in Shenandoah, so this needs to be double-inverteed).
filter defaultGC + ExplicitGCInvokesConcurrent.
- Sorting of macros was fixed, as was pointed out by Per
- Some stuff was added to SA, as suggested by Jini
- Rebased on most current jdk/jdk code
http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/05/
https://bugs.openjdk.java.net/browse/JDK-8214349
[x] shared-runtime (coleenp)
[x] shared-compiler (kvn)
I got reviews for shared-build, but an earlier version, so maybe makes
sense to look over this again. Erik J, Magnus?
[ ] shared-build (kvn, erikj, ihse, pliden)
[ ] shared-gc (pliden, kbarrett)
[ ] shared-serviceability (jgeorge, pliden)
[ ] shared-tests (lmesnik, pliden)
[ ] shenandoah-gc
[ ] shenandoah-tests
Thanks for your patience and ongoing support!
Cheers,
Roman

Post by Roman Kennke
Hi all,
- Verify and gracefully reject dangerous flags combinations that
disables required barriers
- Trim unused code from Shenandoah's SA impl
- Move ShenandoahGCTracer to gc/shenandoah
- Fix ordering of GC names in various files
- Rename UINT64_FORMAT_HEX_W to UINT64_FORMAT_X_W
http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/04/
Thanks everybody for taking time to review this!
Roman

Post by Roman Kennke
Hello all,
Thanks so far for all the reviews and support!
I forgot to update the 'round' yesterday. We are in round 3 now :-)
Also, I fixed the numbering of my webrevs to match the review-round. ;-)
- We moved shenandoah-specific code out of .ad files into our own .ad
files under gc/shenandoah (in shenandoah-gc), how cool is that? This
requires an addition in build machinery though, see
make/hotspot/gensrc/GensrcAdlc.gmk (in shared-build).
- Improved zero-disabling and build-code-simplification as suggested by
Magnus and Per
- Cleaned up some leftovers in C2
- Improved C2 loop opts code by introducing another APIs in
BarrierSetC2. See the new APIs in shared-gc under BarrierSetC2.hpp.
- I don't see where it makes sense to put INCLUDE_SHENANDOAHGC guards now.
- We would all very much prefer to keep ShenandoahXYZNode names, as
noted earlier. This stuff is Shenandoah-specific, so let's just call it
that.
- Rehashed Shenandoah tests (formatting, naming, directory layout, etc)
- Rebased on jdk-12+22
- Question: let us know if you need separate RFE for the new BSC2 APIs?
http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/03/
Thanks,
Roman

Post by Roman Kennke
- The minor issues that Kim reported in shared-gc
- A lot of fixes in shared-tests according to Leonid's review
- Disabled SA heapdumping similar to ZGC as Per suggested
Leonid: test/hotspot/jtreg/gc/TestFullGCCount.java was actually
ExplicitGCInvokesConcurrent at the same time, because that would be
(expectedly) failing. It can run CMS, default GC and any other GC just
fine. Adding the same clause for Shenandoah means the same, and filters
the combination (+UseShenandoahGC)+(+ExplicitGCInvokesConcurrent). I
made the condition a bit clearer by avoiding triple-negation.
http://mail.openjdk.java.net/pipermail/shenandoah-dev/2018-November/008457.html
- test/hotspot/jtreg/serviceability/sa/ClhsdbJhisto.java
- test/hotspot/jtreg/serviceability/sa/TestHeapDumpForLargeArray.java
we filter them for Shenandoah now. I'm wondering: how do you get past
those with ZGC?
http://mail.openjdk.java.net/pipermail/shenandoah-dev/2018-November/008466.html
(Note to Leonid and tests reviewers: I'll add those related filters in
next round).
Vladimir: Roland integrated a bunch of changes to make loop* code look
better. I can tell that we're not done with C2 yet. Can you look over
the code and see what is ok, and especially what is not ok, so that we
can focus our efforts on the relevant parts?
http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/01/
Thanks,
Roman

Post by Roman Kennke
Hi,
This is the first round of changes for including Shenandoah GC into
mainline.
I divided the review into parts that roughly correspond to the mailing lists
that would normally review it, and I divided it into 'shared' code
changes and
'shenandoah' code changes (actually, mostly additions). The intend is to
eventually
push them as single 'combined' changeset, once reviewed.
  https://openjdk.java.net/jeps/189
 https://bugs.openjdk.java.net/browse/JDK-8214259
  http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/
For those who want to see the full change, have a look at the
shenandoah-complete
<http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shenandoah-complete/>
directory,
it contains the full combined webrev. Alternatively, there is the file
shenandoah-master.patch
<http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shenandoah-master.patch>,
which is what I intend to commit (and which should be equivalent to the
'shenandoah-complete' webrev).
 *) shenandoah-gc
<http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shenandoah-gc/>
    - Actual Shenandoah implementation, almost completely residing in
gc/shenandoah
 *) shared-gc
<http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shared-gc/>
    - This is mostly boilerplate that is common to any GC
    - referenceProcessor.cpp has a little change to make one assert not
fail (next to CMS and G1)
    - taskqueue.hpp has some small adjustments to enable subclassing
 *) shared-serviceability
<http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shared-serviceability/>
    - The usual code to support another GC
 *) shared-runtime
<http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shared-runtime/>
    - A number of friends declarations to allow Shenandoah iterators to
hook up with,
      e.g. ClassLoaderData, CodeCache, etc
    - Warning and disabling JFR LeakProfiler
    - fieldDescriptor.hpp added is_stable() accessor, for use in
Shenandoah C2 optimizations
    - Locks initialization in mutexLocker.cpp as usual
    - VM operations defines for Shenandoah's VM ops
    - globalDefinitions.hpp added UINT64_FORMAT_HEX_W for use in
Shenandoah's logging
    - The usual macros in macro.hpp
 *) shared-build
<http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shared-build/>
    - Add shenandoah feature, enabled by default, as agreed with
Vladimir K. beforehand
    - Some flags for shenandoah-enabled compilation to get
SUPPORT_BARRIER_ON_PRIMITIVES
      and SUPPORT_NOT_TO_SPACE_INVARIANT which is required for
Shenandoah's barriers
    - --param inline-unit-growth=1000 settings for 2 shenandoah source
files, which is
      useful to get the whole marking loop inlined (observed significant
regression if we
      don't)
 *) shared-tests
<http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shared-tests/>
    - Test infrastructure to support Shenandoah
    - Shenandoah test groups
    - Exclude Shenandoah in various tests that can be run with selected GC
    - Enable/add configure for Shenandoah for tests that make sense to
run with it
 *) shenandoah-tests
<http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shenandoah-tests/>
    - Shenandoah specific tests, most reside in gc/shenandoah subdirectory
    - A couple of tests configurations have been added, e.g.
TestGCBasherWithShenandoah.java
I intentionally left out shared-compiler for now, because we have some
work left to do
there, but if you click around you'll find the patch anyway, in case you
want to take
a peek at it.
  - {Linux} x {x86_64, x86_32, armhf, aarch64, ppc64el, s390x}
  - {Windows} x {x86_64},
  - {MacOS X} x {x86_64}
  - the new Shenandoah tests
  - jcstress with/without aggressive Shenandoah verification
  - specjvm2008 with/without aggressive Shenandoah verification
I'd like to thank my collegues at Red Hat: Christine Flood, she deserves
the credit for being the original inventor of Shenandoah, Aleksey
Shiplëv, Roland Westrelin & Zhengyu Gu for their countless
contributions, everybody else in Red Hat's OpenJDK team for testing,
advice and support, my collegues in Oracle's GC, runtime and compiler
teams for tirelessly helping with and reviewing all the GC interface and
related changes, and of course the many early adopters for reporting
bugs and success stories and feature requests: we wouldn't be here
without any of you!
Best regards,
Roman

[Roman Kennke]

Zhengyu's and Thomas' TaskQueue stuff landed in jdk/jdk and we
integrated it into shenandoah/jdk and in the master patch and webrevs.
I'm updating the webrevs in-place in:

http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/06/

This is going to be the final webrev that I intend to push on Monday,
unless somebody stops me or we face serious merge conflicts by then.

I've just pushed this to jdk/submit for testing. I will also do more
testing locally over the weekend.

Thanks everybody for reviewing and helping and being patient!

Cheers,
Roman

Post by Roman Kennke
Here comes round 6, possibly/hopefully the last round of webrevs for
http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/06/
- renames vm_operations_shenandoah* to shenandoahVMOperations*, as
suggested by Coleen
- reshuffles gcCause in shared-gc SA as suggested by Per
- print number of threads in SA, as suggested by Jini
- fixed a problem in webrev generation that did not track move of
CriticalNativeStress.java and CriticalNativeArgs.java
The CSR has been approved, the JEP moved to target jdk12, and I got
positive reviews for all parts. I intend to push this early next week,
unless somebody stops me. If Zhengyu's and Thomas' TaskQueue change goes
in by then, I'll incorporate it.
Thanks everybody for reviewing and reviewing and reviewing again ;-)
Cheers,
Roman

Post by Roman Kennke
fixed it the same.
- Actual GC set by harness is CMS *and* ExplicitGCInvokesConcurrent is
true, as set by harness
- Actual GC set by harness is Shenandoah *and*
ExplicitGCInvokesConcurrent is not set false by harness (it's true by
default in Shenandoah, so this needs to be double-inverteed).
filter defaultGC + ExplicitGCInvokesConcurrent.
- Sorting of macros was fixed, as was pointed out by Per
- Some stuff was added to SA, as suggested by Jini
- Rebased on most current jdk/jdk code
http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/05/
https://bugs.openjdk.java.net/browse/JDK-8214349
[x] shared-runtime (coleenp)
[x] shared-compiler (kvn)
I got reviews for shared-build, but an earlier version, so maybe makes
sense to look over this again. Erik J, Magnus?
[ ] shared-build (kvn, erikj, ihse, pliden)
[ ] shared-gc (pliden, kbarrett)
[ ] shared-serviceability (jgeorge, pliden)
[ ] shared-tests (lmesnik, pliden)
[ ] shenandoah-gc
[ ] shenandoah-tests
Thanks for your patience and ongoing support!
Cheers,
Roman

Post by Roman Kennke
Hi all,
- Verify and gracefully reject dangerous flags combinations that
disables required barriers
- Trim unused code from Shenandoah's SA impl
- Move ShenandoahGCTracer to gc/shenandoah
- Fix ordering of GC names in various files
- Rename UINT64_FORMAT_HEX_W to UINT64_FORMAT_X_W
http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/04/
Thanks everybody for taking time to review this!
Roman

Post by Roman Kennke
Hello all,
Thanks so far for all the reviews and support!
I forgot to update the 'round' yesterday. We are in round 3 now :-)
Also, I fixed the numbering of my webrevs to match the review-round. ;-)
- We moved shenandoah-specific code out of .ad files into our own .ad
files under gc/shenandoah (in shenandoah-gc), how cool is that? This
requires an addition in build machinery though, see
make/hotspot/gensrc/GensrcAdlc.gmk (in shared-build).
- Improved zero-disabling and build-code-simplification as suggested by
Magnus and Per
- Cleaned up some leftovers in C2
- Improved C2 loop opts code by introducing another APIs in
BarrierSetC2. See the new APIs in shared-gc under BarrierSetC2.hpp.
- I don't see where it makes sense to put INCLUDE_SHENANDOAHGC guards now.
- We would all very much prefer to keep ShenandoahXYZNode names, as
noted earlier. This stuff is Shenandoah-specific, so let's just call it
that.
- Rehashed Shenandoah tests (formatting, naming, directory layout, etc)
- Rebased on jdk-12+22
- Question: let us know if you need separate RFE for the new BSC2 APIs?
http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/03/
Thanks,
Roman

Post by Roman Kennke
- The minor issues that Kim reported in shared-gc
- A lot of fixes in shared-tests according to Leonid's review
- Disabled SA heapdumping similar to ZGC as Per suggested
Leonid: test/hotspot/jtreg/gc/TestFullGCCount.java was actually
ExplicitGCInvokesConcurrent at the same time, because that would be
(expectedly) failing. It can run CMS, default GC and any other GC just
fine. Adding the same clause for Shenandoah means the same, and filters
the combination (+UseShenandoahGC)+(+ExplicitGCInvokesConcurrent). I
made the condition a bit clearer by avoiding triple-negation.
http://mail.openjdk.java.net/pipermail/shenandoah-dev/2018-November/008457.html
- test/hotspot/jtreg/serviceability/sa/ClhsdbJhisto.java
- test/hotspot/jtreg/serviceability/sa/TestHeapDumpForLargeArray.java
we filter them for Shenandoah now. I'm wondering: how do you get past
those with ZGC?
http://mail.openjdk.java.net/pipermail/shenandoah-dev/2018-November/008466.html
(Note to Leonid and tests reviewers: I'll add those related filters in
next round).
Vladimir: Roland integrated a bunch of changes to make loop* code look
better. I can tell that we're not done with C2 yet. Can you look over
the code and see what is ok, and especially what is not ok, so that we
can focus our efforts on the relevant parts?
http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/01/
Thanks,
Roman

Post by Roman Kennke
Hi,
This is the first round of changes for including Shenandoah GC into
mainline.
I divided the review into parts that roughly correspond to the mailing lists
that would normally review it, and I divided it into 'shared' code
changes and
'shenandoah' code changes (actually, mostly additions). The intend is to
eventually
push them as single 'combined' changeset, once reviewed.
  https://openjdk.java.net/jeps/189
 https://bugs.openjdk.java.net/browse/JDK-8214259
  http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/
For those who want to see the full change, have a look at the
shenandoah-complete
<http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shenandoah-complete/>
directory,
it contains the full combined webrev. Alternatively, there is the file
shenandoah-master.patch
<http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shenandoah-master.patch>,
which is what I intend to commit (and which should be equivalent to the
'shenandoah-complete' webrev).
 *) shenandoah-gc
<http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shenandoah-gc/>
    - Actual Shenandoah implementation, almost completely residing in
gc/shenandoah
 *) shared-gc
<http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shared-gc/>
    - This is mostly boilerplate that is common to any GC
    - referenceProcessor.cpp has a little change to make one assert not
fail (next to CMS and G1)
    - taskqueue.hpp has some small adjustments to enable subclassing
 *) shared-serviceability
<http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shared-serviceability/>
    - The usual code to support another GC
 *) shared-runtime
<http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shared-runtime/>
    - A number of friends declarations to allow Shenandoah iterators to
hook up with,
      e.g. ClassLoaderData, CodeCache, etc
    - Warning and disabling JFR LeakProfiler
    - fieldDescriptor.hpp added is_stable() accessor, for use in
Shenandoah C2 optimizations
    - Locks initialization in mutexLocker.cpp as usual
    - VM operations defines for Shenandoah's VM ops
    - globalDefinitions.hpp added UINT64_FORMAT_HEX_W for use in
Shenandoah's logging
    - The usual macros in macro.hpp
 *) shared-build
<http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shared-build/>
    - Add shenandoah feature, enabled by default, as agreed with
Vladimir K. beforehand
    - Some flags for shenandoah-enabled compilation to get
SUPPORT_BARRIER_ON_PRIMITIVES
      and SUPPORT_NOT_TO_SPACE_INVARIANT which is required for
Shenandoah's barriers
    - --param inline-unit-growth=1000 settings for 2 shenandoah source
files, which is
      useful to get the whole marking loop inlined (observed significant
regression if we
      don't)
 *) shared-tests
<http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shared-tests/>
    - Test infrastructure to support Shenandoah
    - Shenandoah test groups
    - Exclude Shenandoah in various tests that can be run with selected GC
    - Enable/add configure for Shenandoah for tests that make sense to
run with it
 *) shenandoah-tests
<http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shenandoah-tests/>
    - Shenandoah specific tests, most reside in gc/shenandoah subdirectory
    - A couple of tests configurations have been added, e.g.
TestGCBasherWithShenandoah.java
I intentionally left out shared-compiler for now, because we have some
work left to do
there, but if you click around you'll find the patch anyway, in case you
want to take
a peek at it.
  - {Linux} x {x86_64, x86_32, armhf, aarch64, ppc64el, s390x}
  - {Windows} x {x86_64},
  - {MacOS X} x {x86_64}
  - the new Shenandoah tests
  - jcstress with/without aggressive Shenandoah verification
  - specjvm2008 with/without aggressive Shenandoah verification
I'd like to thank my collegues at Red Hat: Christine Flood, she deserves
the credit for being the original inventor of Shenandoah, Aleksey
Shiplëv, Roland Westrelin & Zhengyu Gu for their countless
contributions, everybody else in Red Hat's OpenJDK team for testing,
advice and support, my collegues in Oracle's GC, runtime and compiler
teams for tirelessly helping with and reviewing all the GC interface and
related changes, and of course the many early adopters for reporting
bugs and success stories and feature requests: we wouldn't be here
without any of you!
Best regards,
Roman

[Roman Kennke]

Anybody has an idea what's up with the test below? As far as I can tell,
it passes when I run locally.

Thanks,
Roman

mach5-one-rkennke-JDK-8214259-20181208-1149-13785: FAILED, Failed tests: 1

Build Details: 2018-12-08-1147316.roman.source
1 Failed Test
Test Tier Platform Keywords Description Task
compiler/aot/calls/fromAot/AotInvokeDynamic2AotTest.java tier1
linux-x64-debug othervm driver Exception: java.lang.AssertionError:
duplicate classes for name Ljava/lang/Object;, fingerprints old: ...,
new: ..., klass pointers old: meta{HotSpotType}, new:
meta{HotSpotType} task
Mach5 Tasks Results Summary

FAILED: 0
EXECUTED_WITH_FAILURE: 1
NA: 0
KILLED: 0
PASSED: 75
UNABLE_TO_RUN: 0
Test

1 Executed with failure

tier1-debug-open_test_hotspot_jtreg_tier1_compiler_not_xcomp-linux-x64-debug-37
Results: total: 73, passed: 72, failed: 1, skipped: 0

Post by Roman Kennke
Zhengyu's and Thomas' TaskQueue stuff landed in jdk/jdk and we
integrated it into shenandoah/jdk and in the master patch and webrevs.
http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/06/
This is going to be the final webrev that I intend to push on Monday,
unless somebody stops me or we face serious merge conflicts by then.
I've just pushed this to jdk/submit for testing. I will also do more
testing locally over the weekend.
Thanks everybody for reviewing and helping and being patient!
Cheers,
Roman

Post by Roman Kennke
Here comes round 6, possibly/hopefully the last round of webrevs for
http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/06/
- renames vm_operations_shenandoah* to shenandoahVMOperations*, as
suggested by Coleen
- reshuffles gcCause in shared-gc SA as suggested by Per
- print number of threads in SA, as suggested by Jini
- fixed a problem in webrev generation that did not track move of
CriticalNativeStress.java and CriticalNativeArgs.java
The CSR has been approved, the JEP moved to target jdk12, and I got
positive reviews for all parts. I intend to push this early next week,
unless somebody stops me. If Zhengyu's and Thomas' TaskQueue change goes
in by then, I'll incorporate it.
Thanks everybody for reviewing and reviewing and reviewing again ;-)
Cheers,
Roman

Post by Roman Kennke
fixed it the same.
- Actual GC set by harness is CMS *and* ExplicitGCInvokesConcurrent is
true, as set by harness
- Actual GC set by harness is Shenandoah *and*
ExplicitGCInvokesConcurrent is not set false by harness (it's true by
default in Shenandoah, so this needs to be double-inverteed).
filter defaultGC + ExplicitGCInvokesConcurrent.
- Sorting of macros was fixed, as was pointed out by Per
- Some stuff was added to SA, as suggested by Jini
- Rebased on most current jdk/jdk code
http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/05/
https://bugs.openjdk.java.net/browse/JDK-8214349
[x] shared-runtime (coleenp)
[x] shared-compiler (kvn)
I got reviews for shared-build, but an earlier version, so maybe makes
sense to look over this again. Erik J, Magnus?
[ ] shared-build (kvn, erikj, ihse, pliden)
[ ] shared-gc (pliden, kbarrett)
[ ] shared-serviceability (jgeorge, pliden)
[ ] shared-tests (lmesnik, pliden)
[ ] shenandoah-gc
[ ] shenandoah-tests
Thanks for your patience and ongoing support!
Cheers,
Roman

Post by Roman Kennke
Hi all,
- Verify and gracefully reject dangerous flags combinations that
disables required barriers
- Trim unused code from Shenandoah's SA impl
- Move ShenandoahGCTracer to gc/shenandoah
- Fix ordering of GC names in various files
- Rename UINT64_FORMAT_HEX_W to UINT64_FORMAT_X_W
http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/04/
Thanks everybody for taking time to review this!
Roman

Post by Roman Kennke
Hello all,
Thanks so far for all the reviews and support!
I forgot to update the 'round' yesterday. We are in round 3 now :-)
Also, I fixed the numbering of my webrevs to match the review-round. ;-)
- We moved shenandoah-specific code out of .ad files into our own .ad
files under gc/shenandoah (in shenandoah-gc), how cool is that? This
requires an addition in build machinery though, see
make/hotspot/gensrc/GensrcAdlc.gmk (in shared-build).
- Improved zero-disabling and build-code-simplification as suggested by
Magnus and Per
- Cleaned up some leftovers in C2
- Improved C2 loop opts code by introducing another APIs in
BarrierSetC2. See the new APIs in shared-gc under BarrierSetC2.hpp.
- I don't see where it makes sense to put INCLUDE_SHENANDOAHGC guards now.
- We would all very much prefer to keep ShenandoahXYZNode names, as
noted earlier. This stuff is Shenandoah-specific, so let's just call it
that.
- Rehashed Shenandoah tests (formatting, naming, directory layout, etc)
- Rebased on jdk-12+22
- Question: let us know if you need separate RFE for the new BSC2 APIs?
http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/03/
Thanks,
Roman

Post by Roman Kennke
- The minor issues that Kim reported in shared-gc
- A lot of fixes in shared-tests according to Leonid's review
- Disabled SA heapdumping similar to ZGC as Per suggested
Leonid: test/hotspot/jtreg/gc/TestFullGCCount.java was actually
ExplicitGCInvokesConcurrent at the same time, because that would be
(expectedly) failing. It can run CMS, default GC and any other GC just
fine. Adding the same clause for Shenandoah means the same, and filters
the combination (+UseShenandoahGC)+(+ExplicitGCInvokesConcurrent). I
made the condition a bit clearer by avoiding triple-negation.
http://mail.openjdk.java.net/pipermail/shenandoah-dev/2018-November/008457.html
- test/hotspot/jtreg/serviceability/sa/ClhsdbJhisto.java
- test/hotspot/jtreg/serviceability/sa/TestHeapDumpForLargeArray.java
we filter them for Shenandoah now. I'm wondering: how do you get past
those with ZGC?
http://mail.openjdk.java.net/pipermail/shenandoah-dev/2018-November/008466.html
(Note to Leonid and tests reviewers: I'll add those related filters in
next round).
Vladimir: Roland integrated a bunch of changes to make loop* code look
better. I can tell that we're not done with C2 yet. Can you look over
the code and see what is ok, and especially what is not ok, so that we
can focus our efforts on the relevant parts?
http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/01/
Thanks,
Roman

Post by Roman Kennke
Hi,
This is the first round of changes for including Shenandoah GC into
mainline.
I divided the review into parts that roughly correspond to the mailing lists
that would normally review it, and I divided it into 'shared' code
changes and
'shenandoah' code changes (actually, mostly additions). The intend is to
eventually
push them as single 'combined' changeset, once reviewed.
  https://openjdk.java.net/jeps/189
 https://bugs.openjdk.java.net/browse/JDK-8214259
  http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/
For those who want to see the full change, have a look at the
shenandoah-complete
<http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shenandoah-complete/>
directory,
it contains the full combined webrev. Alternatively, there is the file
shenandoah-master.patch
<http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shenandoah-master.patch>,
which is what I intend to commit (and which should be equivalent to the
'shenandoah-complete' webrev).
 *) shenandoah-gc
<http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shenandoah-gc/>
    - Actual Shenandoah implementation, almost completely residing in
gc/shenandoah
 *) shared-gc
<http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shared-gc/>
    - This is mostly boilerplate that is common to any GC
    - referenceProcessor.cpp has a little change to make one assert not
fail (next to CMS and G1)
    - taskqueue.hpp has some small adjustments to enable subclassing
 *) shared-serviceability
<http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shared-serviceability/>
    - The usual code to support another GC
 *) shared-runtime
<http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shared-runtime/>
    - A number of friends declarations to allow Shenandoah iterators to
hook up with,
      e.g. ClassLoaderData, CodeCache, etc
    - Warning and disabling JFR LeakProfiler
    - fieldDescriptor.hpp added is_stable() accessor, for use in
Shenandoah C2 optimizations
    - Locks initialization in mutexLocker.cpp as usual
    - VM operations defines for Shenandoah's VM ops
    - globalDefinitions.hpp added UINT64_FORMAT_HEX_W for use in
Shenandoah's logging
    - The usual macros in macro.hpp
 *) shared-build
<http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shared-build/>
    - Add shenandoah feature, enabled by default, as agreed with
Vladimir K. beforehand
    - Some flags for shenandoah-enabled compilation to get
SUPPORT_BARRIER_ON_PRIMITIVES
      and SUPPORT_NOT_TO_SPACE_INVARIANT which is required for
Shenandoah's barriers
    - --param inline-unit-growth=1000 settings for 2 shenandoah source
files, which is
      useful to get the whole marking loop inlined (observed significant
regression if we
      don't)
 *) shared-tests
<http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shared-tests/>
    - Test infrastructure to support Shenandoah
    - Shenandoah test groups
    - Exclude Shenandoah in various tests that can be run with selected GC
    - Enable/add configure for Shenandoah for tests that make sense to
run with it
 *) shenandoah-tests
<http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/00/shenandoah-tests/>
    - Shenandoah specific tests, most reside in gc/shenandoah subdirectory
    - A couple of tests configurations have been added, e.g.
TestGCBasherWithShenandoah.java
I intentionally left out shared-compiler for now, because we have some
work left to do
there, but if you click around you'll find the patch anyway, in case you
want to take
a peek at it.
  - {Linux} x {x86_64, x86_32, armhf, aarch64, ppc64el, s390x}
  - {Windows} x {x86_64},
  - {MacOS X} x {x86_64}
  - the new Shenandoah tests
  - jcstress with/without aggressive Shenandoah verification
  - specjvm2008 with/without aggressive Shenandoah verification
I'd like to thank my collegues at Red Hat: Christine Flood, she deserves
the credit for being the original inventor of Shenandoah, Aleksey
Shiplëv, Roland Westrelin & Zhengyu Gu for their countless
contributions, everybody else in Red Hat's OpenJDK team for testing,
advice and support, my collegues in Oracle's GC, runtime and compiler
teams for tirelessly helping with and reviewing all the GC interface and
related changes, and of course the many early adopters for reporting
bugs and success stories and feature requests: we wouldn't be here
without any of you!
Best regards,
Roman

[Kim Barrett]

Post by Roman Kennke
Anybody has an idea what's up with the test below? As far as I can tell,
it passes when I run locally.

It appears to be an instance of https://bugs.openjdk.java.net/browse/JDK-8201248.

Post by Roman Kennke
Thanks,
Roman
mach5-one-rkennke-JDK-8214259-20181208-1149-13785: FAILED, Failed tests: 1
Build Details: 2018-12-08-1147316.roman.source
1 Failed Test
Test Tier Platform Keywords Description Task
compiler/aot/calls/fromAot/AotInvokeDynamic2AotTest.java tier1
duplicate classes for name Ljava/lang/Object;, fingerprints old: ...,
meta{HotSpotType} task
Mach5 Tasks Results Summary
FAILED: 0
EXECUTED_WITH_FAILURE: 1
NA: 0
KILLED: 0
PASSED: 75
UNABLE_TO_RUN: 0
Test
1 Executed with failure
tier1-debug-open_test_hotspot_jtreg_tier1_compiler_not_xcomp-linux-x64-debug-37
Results: total: 73, passed: 72, failed: 1, skipped: 0

[Roland Westrelin]

Post by Roman Kennke
http://cr.openjdk.java.net/~rkennke/shenandoah-upstream/05/

C2 changes look good to me.

Roland.