JOSM fails to upload exactly 30000 changes in chunks of 10000

[GerdP]

What steps will reproduce the problem?

1. Have a dataset with exactly 30.000 new nodes (or other objects to upload)
2. Upload, change chunksize to 10.000 (which - as of now - is also exactly the maximum size specified by the API)
3. Start upload

What is the expected result?

All nodes are uploaded

What happens instead?

Only 20.000 nodes are uploaded, no error message is shown for the remaining 10.000.

Please provide any additional information below. Attach a screenshot if possible.

Found this while worrking on #23738. Problem is in this code in UploadPrimitivesTask.handleChangesetFullResponse()

        if (processedPrimitives.size() == toUpload.getSize()) {
            strategy.setPolicy(MaxChangesetSizeExceededPolicy.ABORT);
            return false;
        }

Each time when 10.000 objects are uploaded the changeset is closed. The not-yet-uploaded objects are collected in a new list and the upload is restarted with a new changeset.
After 20.000 objects the number of already uploaded objects is compared with the size of "objects to upload" and the numbers are equal and thus the upload is stopped too early.

Relative:URL: ^/trunk
Repository:UUID: 0c6e7542-c601-0410-84e7-c038aed88b3b
Last:Changed Date: 2024-06-03 18:08:14 +0200 (Mon, 03 Jun 2024)
Revision:19096
Build-Date:2024-06-04 01:31:15
URL:https://josm.openstreetmap.de/svn/trunk

Identification: JOSM/1.5 (19096 en) Windows 10 64-Bit
OS Build number: Windows 10 Pro 22H2 (19045)
Memory Usage: 293 MB / 1888 MB (158 MB allocated, but free)
Java version: 21.0.2+13-LTS, Azul Systems, Inc., OpenJDK 64-Bit Server VM
Look and Feel: com.sun.java.swing.plaf.windows.WindowsLookAndFeel
Screen: \Display0 1920×1080 (scaling 1.00×1.00)
Maximum Screen Size: 1920×1080
Best cursor sizes: 16×16→32×32, 32×32→32×32
System property file.encoding: UTF-8
System property sun.jnu.encoding: Cp1252
Locale info: en_DE
Numbers with default locale: 1234567890 -> 1234567890
VM arguments: [-Djpackage.app-version=1.5.19039, --add-modules=java.scripting,java.sql,javafx.controls,javafx.media,javafx.swing,javafx.web, --add-exports=java.base/sun.security.action=ALL-UNNAMED, --add-exports=java.desktop/com.sun.imageio.plugins.jpeg=ALL-UNNAMED, --add-exports=java.desktop/com.sun.imageio.spi=ALL-UNNAMED, --add-opens=java.base/java.lang=ALL-UNNAMED, --add-opens=java.base/java.nio=ALL-UNNAMED, --add-opens=java.base/jdk.internal.loader=ALL-UNNAMED, --add-opens=java.base/jdk.internal.ref=ALL-UNNAMED, --add-opens=java.desktop/javax.imageio.spi=ALL-UNNAMED, --add-opens=java.desktop/javax.swing.text.html=ALL-UNNAMED, --add-opens=java.prefs/java.util.prefs=ALL-UNNAMED, -Djpackage.app-path=%UserProfile%\AppData\Local\JOSM\JOSM HWConsole.exe]

Plugins:
+ utilsplugin2 (36241)

Last errors/warnings:
- 00000.604 W: extended font config - overriding 'filename.Malgun_Gothic=malgun.ttf' with 'MALGUN.TTF'
- 00000.606 W: extended font config - overriding 'filename.Myanmar_Text=mmrtext.ttf' with 'MMRTEXT.TTF'
- 00000.607 W: extended font config - overriding 'filename.Mongolian_Baiti=monbaiti.ttf' with 'MONBAITI.TTF'
- 00001.141 E: java.security.KeyStoreException: Windows-ROOT not found. Cause: java.security.NoSuchAlgorithmException: Windows-ROOT KeyStore not available

OSM API: https://master.apis.dev.openstreetmap.org/api

[gaben]

Isn't it a problem if the CS obj. size % 10_000 == 0?

[gaben]

Parallel edit fix

[GerdP]

No, for 10.000 and 20.000 it works fine. Probably fails with 30.000, 40.000 and so on. Probably only with exactly 30.000.
Quite a special case!

[gaben]

Ah, I see. So one of the variables is not updated, meaning the sum is more than the CS size.

[GerdP]

Problem is that both collections are updated, one grows, the other shrinks. Always risky to use such a comparison in loop control ;)

[GerdP]

This patch seems to fix the problem. I am now looking at probably dead code in UploadPrimitivesTask.promptUserForPolicy().

[GerdP]

The code is still used when the checkbox "Close changesets after upload" is not enabled and user tries to add changes so that >= 10.000 objects are uploaded, but I think I found another error with this dialog (also) in tested version 19096.

[stoecker]

Milestone closed.

[GerdP]

In 19135/josm:

fix #23754: JOSM fails to upload exactly 30000 changes in chunks of 10000

• don't compare collection sizes() because they are changed in different situations

[taylor.smock]

In 19136/josm:

See #23754: Fix test for UploadPrimitivesTask

[GerdP]

Thanks for the fix!

[taylor.smock]

No problem -- stuff happens.