Modify

Opened 4 years ago

Closed 4 years ago

Last modified 4 years ago

#18440 closed enhancement (fixed)

Heads up: Maxar implementing new anti-scraping techniques

Reported by: Stereo Owned by: Don-vip
Priority: normal Milestone: 20.02
Component: Core imagery Version:
Keywords: maxar api key Cc: sommerluk

Description

Attachments (0)

Change History (33)

comment:1 by Boggedy, 4 years ago

This may also require an auto expire feature for the JOSM imagery cache where Maxar and ESRI tiles are all autodeleted on exit.

comment:2 by sommerluk, 4 years ago

Cc: sommerluk added

comment:3 by Don-vip, 4 years ago

Ticket #18473 has been marked as a duplicate of this ticket.

comment:4 by Don-vip, 4 years ago

Keywords: maxar added

comment:5 by Don-vip, 4 years ago

Ticket #18516 has been marked as a duplicate of this ticket.

comment:6 by Don-vip, 4 years ago

See https://josm.openstreetmap.de/ticket/18473#comment:3 comments about the removal from default entries

comment:7 by Don-vip, 4 years ago

Milestone: 20.02
Owner: changed from team to Don-vip
Status: newassigned

comment:8 by Don-vip, 4 years ago

I've received an e-mail from Maxar providing us (JOSM) a new API key to restore access to Maxar imagery. Their only condition is to not make the API key public on the wiki or the source code.

I already began related work in #17744 / [o35017] when the Bing API key was unavailable. I'll work on this tomorrow during GeoFabrik hack week-end.

comment:9 by stoecker, 4 years ago

If the key should not be in the wiki or the source, where should it be?

comment:10 by Don-vip, 4 years ago

I was thinking about providing it at build time and storing it somewhat in JOSM jar. Not completely hidden, but at least not easily crawlable. They seem ok with this.

comment:11 by stoecker, 4 years ago

That's not possible. It would be a license violation of JOSM GPL.

comment:12 by Don-vip, 4 years ago

Ok. Does it work if JOSM retrieves the key from JOSM website using a new endpoint? I'm thinking of a POST request with a user-agent check.

comment:13 by stoecker, 4 years ago

That would work.

comment:14 by stoecker, 4 years ago

If so, please make it expandable to other JOSM specific keys.

comment:15 by Don-vip, 4 years ago

In 15853/josm:

see #18440 - non-blocking warning in case of invalid TMS entries

comment:16 by Don-vip, 4 years ago

JMapViewer updated in [o35320].

in reply to:  14 ; comment:17 by Don-vip, 4 years ago

Replying to stoecker:

If so, please make it expandable to other JOSM specific keys.

What other keys do you have in mind?

in reply to:  17 ; comment:18 by stoecker, 4 years ago

Replying to Don-vip:

Replying to stoecker:

If so, please make it expandable to other JOSM specific keys.

What other keys do you have in mind?

We have a few other JOSM specific map keys. If we setup such a feature it should be possible to support other services as well. It should not be Maxar only.

comment:19 by stoecker, 4 years ago

E.g.

https://josm.openstreetmap.org/mapkey?id=maxar

returns the maxar key and ...

comment:20 by SimonPoole, 4 years ago

@stoecker the GPL in general does not apply to configuration.

in reply to:  20 ; comment:21 by stoecker, 4 years ago

Replying to SimonPoole:

@stoecker the GPL in general does not apply to configuration.

That's not configuration. GPL applies to sources and the tools/toolchain to build an executable from this. So there cannot be any secret in the build process. The code signing we do for JOSM is already a border case.

in reply to:  18 comment:22 by Don-vip, 4 years ago

Replying to stoecker:

If we setup such a feature it should be possible to support other services as well. It should not be Maxar only.

No problem, this is what I had in mind.
EDIT: These are the entries I found with an API key:

  • Bing
  • City_of_Melbourne_Feb_2019
  • City_of_Melbourne_May_2018
  • geoimage.at
  • geovekst-nib
  • IBGE_DF_Addresses
  • IBGE_Nomes_Ruas
  • IBGE_Salvador_Streets
  • lantmateriet-orto1960
  • lantmateriet-orto1975
  • lantmateriet-topowebb
  • LINZ_NZ_Aerial_Imagery
  • LINZ_NZ_Topo50_Gridless_Maps
  • Mapbox
  • mapbox_locator_overlay
  • Maxar-Premium
  • Maxar-Standard
  • New_and_Misaligned_TIGER_Roads-2013
  • opencylemap
  • tf-landscape
  • US_Forest_Service_roads_overlay
  • US-TIGER-Roads-2017
  • US-TIGER-Roads-2018
  • US-TIGER-Roads-2019
  • WSDOT-Roads-2017

Any missing?

Last edited 4 years ago by Don-vip (previous) (diff)

comment:23 by Don-vip, 4 years ago

In 15855/josm:

see #18440 - Fetch imagery API keys from JOSM website. Restore access to Maxar imagery. Requires JMapViewer 2.13

in reply to:  21 comment:24 by anonymous, 4 years ago

Replying to stoecker:

Replying to SimonPoole:

@stoecker the GPL in general does not apply to configuration.

That's not configuration. GPL applies to sources and the tools/toolchain to build an executable from this. So there cannot be any secret in the build process. The code signing we do for JOSM is already a border case.

This not even the interpretation of the FSF.

comment:25 by stoecker, 4 years ago

The source code for a work means the preferred form of the work for making modifications to it. For an executable work, complete source code means all the source code for all modules it contains, plus any associated interface definition files, plus the scripts used to control compilation and installation of the executable. However, as a special exception, the source code distributed need not include anything that is normally distributed (in either source or binary form) with the major components (compiler, kernel, and so on) of the operating system on which the executable runs, unless that component itself accompanies the executable.

I don't care about any "interpretation", especially given by an anonymous commenter. I follow the GPL texts and these are very clear in this fact.

Last edited 4 years ago by stoecker (previous) (diff)

comment:26 by Don-vip, 4 years ago

In 15868/josm:

see #18440 - cache API keys in memory

comment:27 by Don-vip, 4 years ago

Keywords: api key added
Resolution: fixed
Status: assignedclosed

Done for Maxar. Follow-up in #18726

comment:28 by Don-vip, 4 years ago

In 15871/josm:

see #18440 - register API key provider in integration test

comment:29 by stoecker, 4 years ago

Could you change the pattern to lowercase {apikey}? that's not a Java variable name.

in reply to:  29 comment:30 by Don-vip, 4 years ago

Replying to stoecker:

Could you change the pattern to lowercase {apikey}? that's not a Java variable name.

Done in [o35329].

comment:31 by Klumbumbus, 4 years ago

There are at least two ways to display the connectId in JOSM in cleartext. Is that a problem?

in reply to:  31 comment:32 by stoecker, 4 years ago

Replying to Klumbumbus:

There are at least two ways to display the connectId in JOSM in cleartext. Is that a problem?

Hopefully not. Current solution is the best we can do in an OpenSource software.

in reply to:  31 comment:33 by Don-vip, 4 years ago

Replying to Klumbumbus:

There are at least two ways to display the connectId in JOSM in cleartext. Is that a problem?

No, I warned them about this and they're ok. They just don't want the key to be too easily scraped (using a search engine for example).

Modify Ticket

Change Properties
Set your email in Preferences
Action
as closed The owner will remain Don-vip.
as The resolution will be set.
The resolution will be deleted. Next status will be 'reopened'.

Add Comment


E-mail address and name can be saved in the Preferences .
 
Note: See TracTickets for help on using tickets.