Modify

Opened 6 years ago

Closed 6 years ago

Last modified 6 years ago

#17291 closed enhancement (fixed)

[Patch] improve handling of return code 404 in multifetch api

Reported by: GerdP Owned by: team
Priority: normal Milestone: 19.03
Component: Core Version:
Keywords: performance Cc:

Description

Please review:
With the current code the MultiFetchServerObjectReader switches to single gets when a list of ids returns a HTTP return code 404. It should instead divide the list into two halves and try again with both halves
unless the lists are very small.

The attached patch implements that approach.
There is also a constant

    /**
     * the max. number of primitives retrieved in one step. Assuming IDs with 7 digits,
     * this leads to a max. request URL of ~ 1600 Bytes ((7 digits +  1 Separator) * 200),
     * which should be safe according to the
     * <a href="http://www.boutell.com/newfaq/misc/urllength.html">WWW FAQ</a>.
     */
    private static final int MAX_IDS_PER_REQUEST = 200;

that looks suspicious as most node ids have 9 or 10 digits now. So maybe the limit is outaged?

Attachments (2)

17291.patch (2.0 KB ) - added by GerdP 6 years ago.
17291-v2.patch (2.7 KB ) - added by GerdP 6 years ago.

Download all attachments as: .zip

Change History (5)

by GerdP, 6 years ago

Attachment: 17291.patch added

by GerdP, 6 years ago

Attachment: 17291-v2.patch added

comment:1 by GerdP, 6 years ago

I've played with the MAX_IDS_PER_REQUEST constant. It seems that the server has no problems when I set it to 500, but with 1000 I see problems. Maybe it also depends on the server load. With the current value of 200 we often produce URLs with ~ 2200 bytes, so I decided to reduce the number to 170 in v2 of the patch.
If I hear no complains I'll commit this next monday.

comment:2 by GerdP, 6 years ago

Resolution: fixed
Status: newclosed

In 14898/josm:

fix #17291: improve handling of return code 404 in multifetch api

  • improve performance when server returns error code 404 (redacted): split workload into two halves instead of requesting each single id
  • reduce MAX_IDS_PER_REQUEST from 200 to 170 (this might decrease performace for large amounts of objects but should be safer)

comment:3 by Don-vip, 6 years ago

Milestone: 19.03

Modify Ticket

Change Properties
Set your email in Preferences
Action
as closed The owner will remain team.
as The resolution will be set.
The resolution will be deleted. Next status will be 'reopened'.

Add Comment


E-mail address and name can be saved in the Preferences .
 
Note: See TracTickets for help on using tickets.