Strava Global Heatmap in JOSM

[anonymous]

With the latest changes to the Global Heatmap, street-level detail is only available when logged into a Strava account. A result of this is unfortunately that the Heatmap has become a more or less useless tool for improving Openstreetmap using JOSM. I guess that this was an unintended result of the changes made to the Heatmap.

[stoecker]

Well. You did not say what you want.

[anonymous]

I just want it fixed... I guess this could either be done on the Strava side of things, or by adding the possibility in JOSM to log into the Strava account.

[Wegspotter]

Same problem here. Can this be fixed?

[Don-vip]

iD issue closed as wontfix: ​https://github.com/openstreetmap/iD/issues/4896

There was doubts about the possibility to use Strava for editing OSM, but this has been confirmed by a Strava employee here: ​https://github.com/osmlab/editor-layer-index/pull/373#issuecomment-352544421

OSM wiki: ​https://wiki.openstreetmap.org/wiki/Strava#Global_Heatmap

This would probably require to setup OAuth authentication like with OSM API, see ​https://developers.strava.com/docs/authentication/

It would be nice to make it generic: define in Maps that a layer require OAuth authentication and provide authorization page URL. This way we wouldn't have Strava-specific code.

[alpterlab]

exactly!Since the new Heat Map is being developed under​Mapbox GL they just need to set proper individual ​access tokens in order to associate requests to API resources with each personal STRAVA account. I guess... but... How long would it take ?

[stoecker]

Replying to Don-vip:

This would probably require to setup OAuth authentication like with OSM API, see ​https://developers.strava.com/docs/authentication/

It would be nice to make it generic: define in Maps that a layer require OAuth authentication and provide authorization page URL. This way we wouldn't have Strava-specific code.

It also requires users to understand how OAuth works and to setup OAuth permissions. For OSM that worked not well until the "Fully Automatic" approach, which is site specific.

[Klumbumbus]

Possible relevant: The mapillary plugin uses a login system too.

[wegspotter]

Maybe there is something useful there > ​https://help.openstreetmap.org/questions/62803/strava-heatmap-no-longer-useful-for-osm-mappers-anything-we-can-do?

[solitone]

Replying to wegspotter:

Maybe there is something useful there > ​https://help.openstreetmap.org/questions/62803/strava-heatmap-no-longer-useful-for-osm-mappers-anything-we-can-do?

No luck here with the suggested new URL:
tms[3,17]:​http://globalheat.strava.com/tiles/cycling/color1/{zoom}/{x}/{y}.png

[anonymous]

Replying to solitone:
This works for me: tms[3,17]:​https://heatmap-external-{switch:a,b,c}.strava.com/tiles-auth/both/bluered/{zoom}/{x}/{y}.png?Key-Pair-Id=MYVALUE&Signature=MYVALUE&Policy=MYVALUE where MYVALUE are the respective cookies on your PC value, but the zoom level is the same as before. Of course, you need to have a strava account and you need to be logged in. And once in a while, you have to paste the new cookies values as they expire. But it's worth the effort.

Replying to wegspotter:

Maybe there is something useful there > ​https://help.openstreetmap.org/questions/62803/strava-heatmap-no-longer-useful-for-osm-mappers-anything-we-can-do?

No luck here with the suggested new URL:
tms[3,17]:​http://globalheat.strava.com/tiles/cycling/color1/{zoom}/{x}/{y}.png

[solitone]

Replying to anonymous:

This works for me: tms[3,17]:​https://heatmap-external-{switch:a,b,c}.strava.com/tiles-auth/both/bluered/{zoom}/{x}/{y}.png?Key-Pair-Id=MYVALUE&Signature=MYVALUE&Policy=MYVALUE where MYVALUE are the respective cookies on your PC value, but the zoom level is the same as before. Of course, you need to have a strava account and you need to be logged in. And once in a while, you have to paste the new cookies values as they expire. But it's worth the effort.

Hi & thanks for your feedback. Unfortunately I can't make it work. If I use "tms[3,17] ..." I get "Error: no protocol". If I use "tms[3,11] ..." I get a message complaining that there is "no tile at this zoom level".

And where do you find the fields Key-Pair-Id, Signature, and Policy? I have the following cookies from Strava: _ga, _gid, _strava4_session, ajs_anonymous_id, ajs_group_id, ajs_user_id. But none contains any of those fileds. I've used the content field in the _strava4_session cookie, that is something like: hkseqa5hezhnpd0hclj4of65fqsulbuk.

Thanks again & Regards

[solitone]

Ok, I changed like the following, and now it works. Cheers!

tms[3,15]:​https://heatmap-external-{switch:a,b,c}.strava.com/tiles-auth/both/bluered/{zoom}/{x}/{y}.png?Key-Pair-Id=MYVALUE1&Policy=MYVALUE2&Signature=MYVALUE3

[SHARCRASH]

I tried to add the address with my cooky values as a TMS but i still get an error "No protocol... etc". I noticed that the keys of MYVALUEx start actually with CloudFront- in the developer console (Firefox and Chrome). Can anybody help please?

[solitone]

Replying to SHARCRASH:

I tried to add the address with my cooky values as a TMS but i still get an error "No protocol... etc".

Double check that address. I also had issue, and found out that I needed to specify "Key-Pair-Id", "Policy", and "Signature" in this exact same order.

I noticed that the keys of MYVALUEx start actually with CloudFront- in the developer console (Firefox and Chrome). Can anybody help please?

Yes, that's true, but in the URL that you enter you should remove the "CloudFront-" part, and specify only "Key-Pair-Id", "Policy", and "Signature" as keys.

[solitone]

BTW, to get those three CloudFront-Key-Pair-Id, CloudFront-Policy, and CloudFront-Signature cookies, you need to open the Strava heatmap from your browser--it's not enough to log in Strava.

As explained at [1], the authentication flow is as follows:
1) POST ​https://www.strava.com/session, email=<STRAVA EMAIL>, password=<STRAVA PASSWORD>
-> Sets _strava4_session cookie
2) GET ​https://heatmap-external-a.strava.com/auth with session cookie set
-> Sets CloudFront-Signature, CloudFront-Policy, and CloudFront-Key-Pair-Id cookies
3) GET ​https://heatmap-external-a.strava.com/t ... h/all/hot/{z}/{x}/{y}.png?v=19 with CloudFront cookies set

[1] ​http://duweis.com/forums/viewtopic.php?f=13&t=761

[SHARCRASH]

Found the problem! Actually, JOSM adds another TMS in the address: tms:tms[3,15]:https... instead of tms[3,15]:https...
Thank you Sonitone!

[jflamy]

Still getting "no protocol" error, and "no tiles found at current zoom level" (which is 13).
Proceeded as follows

• used chrome developer mode to clear my strava cookies
• logged on to strava global heat map
• retrieved the three cookies for policy, key-pair-id, and signature
• followed advice regarding ordering of cookie values when passed explicitly
• added an imaging entry at the bottom of the imaging preferences dialog, and saved, as follows.

  tms[3,15]:​https://heatmap-external-{switch:a,b,c}.strava.com/tiles-auth/both/bluered/{zoom}/{x}/{y}.png?Key-Pair-Id=APKAIDPUN4QMG7VUQPSA&Policy=eyJTdGF0ZW1lbnQiOiBbeyJSZXNvdXJjZSI6Imh0dHBzOi8vaGVhdG1hcC1leHRlcm5hbC0qLnN0cmF2YS5jb20vKiIsIkNvbmRpdGlvbiI6eyJEYXRlTGVzc1RoYW4iOnsiQVdTOkVwb2NoVGltZSI6MTUyODIyMDUxMX0sIkRhdGVHcmVhdGVyVGhhbiI6eyJBV1M6RXBvY2hUaW1lIjoxNTI2OTk2NTExfX19XX0_&Signature=YHy7BrWO1~5tm~i~6SXxQhvn4VpVKWlulFk70~sznKuOPl8TMG-~CgH4gELwp3N9fUOejLpk7ef4lwBJIAq1~yBbIDqo5pJsOJgAc3Q~tVxpSQVA5ayKMF3fJjRbmtGAlf5EoHG19INFGhNrBnw3ql9H4zeIzMYEFb9M29Y1IFUf6CAsSNBCjt0CUCZoYWXu4UpZWVVnlu~xLRPs6osyEv7qvZOlxe9AP5B~CScSfDmcEDUcUkmD5sPk0CMPGls6kCgHOsUiuh4hvI-beEdjer5UPBb0Y2UOmAIV1x~Qd8a05cpoIsBDmkZSV6ou1eIBXikUnTJOb8VJGC5VYIY9kA__
  

• checked using the information button that the proper parameters were being used for the layer

Am out of ideas.

[solitone]

Everything seems fine, but a single character can cause problems. Start up again with the default strava entry and change it so that it considers the three cookies. Make sure you have only one "tms" string (as SHARCRASH pointed out).

[csytsma]

I finally got it to work, and am now showing high resolution heatmap data in JOSM, just like it used to. I do have a Premium account, which you have to have to get the high resolution data. A plain login will only display low resolution data.

I'm not sure what I did to exactly get it working, as it didn't initially work after following the steps above. Here are the steps I took to walk through the process to confirm my URL and values were correct:

1. Go to Strava Heatmap (www.strava.com/heatmap) and make sure you are logged in with Premium/Summit account
2. Confirm you can see high-resolution heatmaps
3. In Chrome, right-click and Inspect, or just open Developer Tools
4. Click on the Network tab
5. Zoom into an area, preferably familiar
6. In the left Name column, look for files named something like 1432.png?v=19
7. Click on the png file
8. Click on the Preview tab, and you should see the heatmap image on the right
9. Right-click on the image, and Copy Image Url
10. Open new tab and paste into URL. Confirm that the image loads, so you know you have a good URL.
11. Put the 3 cookie values (Key-Pair-Id, Signature, Policy) on to the end of your url, just like described above. Confirm that you can still view the image with the new URL.
12. Open a new browser that doesn't have the cookie set, and confirm that the new URL works in that browser. If it works, then you know your URL and cookie values are good.
13. In JOSM, create a new TMS image layer and use this new URL, that will have all values hardcoded.
14. If this works, it will show that image duplicated on every tile. But you know it is properly pulling the high-resolution image with the cookie credentials correctly.
15. Now change the hardcoded values back to the variables
16. Here's my final entry: tms[3,15]:​https://heatmap-external-b.strava.com/tiles-auth/all/hot/{zoom}/{x}/{y}.png?Key-Pair-Id=MYVALUE1&Policy=MYVALUE2&Signature=MYVALUE3

• you can only zoom to 15. Anything higher than that and it disappears.

Hopefully this helps someone else. Don't forget, the cookie only lasts a couple weeks I think, so you'll need to get and set the new cookie creds every couple weeks.

[solitone]

I do get hi-res heatmap data although I don't have a premium account.

[drjoshua]

I can also see it without premium.
Is it impossible to make it visible above zoomlevel 15 but in a scaled look? I don't mind a little blurry map!

[drjoshua]

I forgot the first value of '[3,15]' in the URL. Forget my post above!

[daveemtb]

Any chance someone could post a dummies guide on how to obtain the Key-Pair-Id, Signature, Policy values please?

Thanks!

[solitone]

First you need to click on ​https://www.strava.com/heatmap and log in with your Strava credentials (flag the remember option).
Strava server will send some cookies to your browser, and in one of those cookies you'll find the Key-Pair-Id, Signature, and Policy data.
How to find those data depends on your browser. What are you using?

[lists@…]

Since it is a "closed" oAuth process, and they don't seem willing to let just about anybody in on it, what about trying to get JOSM as an authorised app, and a strava authentication tab in preferences? That way, JOSM could take care of retrieving and storing the oAuth keys, and unlocking the strava imagery choices in the menu...

[anonymous]

There was a cool tool, called slide, which allowed mappers to automatically correct ways to Strava heat. The developer since then left Strava and couldn't do much about as it's currently broken. Maybe one day it could work again, who knows.

​https://github.com/paulmach/slide (check issues)
​https://labs.strava.com/slide/

[anonymous]

any recent updates to this thread?

[solitone]

Cookies expire, so every now and then you need to manually edit Strava imagery URLs with up-to-date cookie values. Since it is a tedious activity, I wrote a small python script to automate this process:
​https://github.com/solitone/josm_strava_cookies

You can use it also to set the correct URL string in the first place.

Unfortunately, for the time being it works only under macOS. I hope to extend it to support other operating systems as well.

[solitone]

Replying to anonymous:

There was a cool tool, called slide, which allowed mappers to automatically correct ways to Strava heat. The developer since then left Strava and couldn't do much about as it's currently broken. Maybe one day it could work again, who knows.

That would be a fantastic tool!!

[pangoSE]

We just got permission (again) to use the heatmaps. See ​https://wiki.openstreetmap.org/wiki/Permissions/Strava

Would anyone of the JOSM devs ​register JOSM as an app on strava and try to implement/setup the oauth2 link like we do for OSM.org?

[Don-vip]

To consider after we switch to OAuth2 for osm.org see #20768

[Don-vip]

Ticket #21112 has been marked as a duplicate of this ticket.