Default maximum memory setting (-Xmx) for distribution packages

[wiktorn]

Currently for our distribution channels we have following maximum heap memory settings:

distribution channel	maximum memory
josm.jnlp	2048m
Windows Installer	2048m
macOS package	8192m

Two questions:

1. Why for macOS we have 4x more memory than for other platforms?
2. Why do we set this up anyway? As far as I've tested, Java defaults to 25% of physical memory and taking into account current configurations, I'd assume that for 90+% users, this would at least give us the same result or higher. On the other hand, when we have -Xmx occupying high percentage of physical memory, overall performance may be unsatisfying due to high swap usage

For a start, I'd lower the macOS -Xmx to be on par with other platforms.

See: #20443, #20497, #19937

[Stereo]

The macOS maximum is probably just me scratching my own itch because I was often running into memory limits. The cheapest Mac you can buy in 2021 comes with 8GB, and it's indeed not great if JOSM gobbles it all up.

The "modern" way of doing things is to limit to a percentage: ​https://support.cloudbees.com/hc/en-us/articles/204859670-Java-Heap-settings-best-practice . As you've seen, the default is 25%. This isn't something we can change at runtime, is it?

[wiktorn]

No, we can't change that at runtime. Other applications do it, by having separate file that starts JVM, where you can toggle it. But in case of macOS - this file is inside *.dmg and is signed, so no modifications are available..

There is ​JEP that aims to improve this situation and allow runtime changing of maximum heap size, but probably it's a long way, until it's implemented (if at all).

I vote for let the JRE use it's default 25%.

[GerdP]

In JOSM a good reason to use a high -Xmx value is when you want to load large files. Another might be when you load lots of images. For normal mapping activities 2G are more than enough. Always presuming you are not using a method with a memory leak too often ;)

[Don-vip]

Does the JVM really takes 25% by default for JNLP files? I remembered a very low hardcoded value (something like 256Mb)

[wiktorn]

Without testing can't confirm. But as I run jdk1.8.0_171 to launch the jar file, it is setting it to 25% of physical memory. And AFAIR it was not the case earlier with JRE. Hopefully it's the same with JNLP.

If you check this ​documentation it still mentions, that default 25% should be capped at 1GB, but it's not longer the case.

Oh, and found some ​SO answer for that. It's since 6u18, that it's always 25% :-) Maybe it's time to hop on ;-)

[skyper]

Sadly, I do not find a way to use a percentage of physical memory as java option on startup.

[wiktorn]

Replying to skyper:

Sadly, I do not find a way to use a percentage of physical memory as java option on startup.

But can we just move to default by not specifying -Xmx?

[Stereo]

Remember it's a maximum, and that most JOSMs will never swallow up all that RAM. I'd much rather have a high maximum that never gets used than the dreaded JOSM out of memory message while I'm in the middle of a complex task I've already invested a lot of time in. If I was limited to 25% of RAM, I'd get that message all the time.

How does -XX:MaxRAMPercentage 75 sound?

[skyper]

I would go with the default value by not specifying -Xmx. Actually, I use less and do face the "Out of memory" message only after distinct use of validator on many objects.

People, needing more can still specify this in their start script or command.

[wiktorn]

Replying to Stereo:

Remember it's a maximum, and that most JOSMs will never swallow up all that RAM. I'd much rather have a high maximum that never gets used than the dreaded JOSM out of memory message while I'm in the middle of a complex task I've already invested a lot of time in. If I was limited to 25% of RAM, I'd get that message all the time.

Remember, that we're in GC world and it's up to GC to decide, how much memory JOSM will use. I'd just like to point out that here GerdP noticed, that using -Xmx8G on his machine did cause performance problem, while using -Xmx2G did not. Probably due to heap overflowing to swap.

So setting the -Xmx high invites Java GC to use this memory (though AFAIR, Java is able to reclaim memory back to OS).

And the question is - what is better in case of shortage of memory - OutOfMemoryException or unbearble slowness?

[GerdP]

Just to make that clear: I used the 8G value only to reproduce the problem. With 64 bit JRE you can set a value that is even higher than the pysical memory, but the massive swapping of pages makes JOSM (and the whole PC) unusable.

I wouldn't set any limit by default and just document the possible JRE options. A JRE that uses almost all available memory is likely to cause more trouble than just some lost OSM changes.

[Don-vip]

Ticket #10213 has been marked as a duplicate of this ticket.

[mdk]

So setting the -Xmx high invites Java GC to use this memory (though AFAIR, Java is able to reclaim memory back to OS).

You remember right. See details in "4.1 Total Heap" on ​https://www.oracle.com/java/technologies/tuning-garbage-collection-v50-java-virtual-machine.html#0.0.0.%20Total%20Heap|outline

[simon04]

In 17711/josm:

fix #20510 - Default maximum memory setting -Xmx for distribution packages

[gaben]

On the front page, the josm-latest.jnlp file still contains the max-heap-size="2048m" attribute.

[Klumbumbus]

Replying to gaben:

On the front page, the josm-latest.jnlp file still contains the max-heap-size="2048m" attribute.

The tested jnlp too.

[skyper]

Replying to Klumbumbus:

Replying to gaben:

On the front page, the josm-latest.jnlp file still contains the max-heap-size="2048m" attribute.

The tested jnlp too.

That is ok, as the new tested (milestone 21.4) is not released, yet. But the changes need to be made to both, latest and tested, scripts.

[skyper]

How are both *.jnlp updated? Last change for both was in January but I did not find a ticket mentioning the files, so far.

[Don-vip]

It's a manual copy. For a long time the only updates to this file were made every two/three years when we upgraded our version of Java.

[Don-vip]

josm-latest.jnlp updated. I'll update josm.jnlp only when we'll release 21.04. I keep the ticket open to not forget.

[simon04]

Once r17833 is built, it can be released as milestone:21.04. Afterwards we can update the https://josm.openstreetmap.de/download/josm.jnlp file.

[Don-vip]

Great! Thanks for taking care of all other stuff. My rocket is launched in just 28 hours :)

[Don-vip]

Done.

[simon04]

Thank you and all the best for the launch. 🚀