Modify

Opened 8 years ago

Closed 8 years ago

Last modified 7 years ago

#14425 closed enhancement (fixed)

[patch] Improve Wikipedia tag validators

Reported by: nyurik Owned by: nyurik
Priority: normal Milestone: 17.03
Component: Core validator Version:
Keywords: , wikipedia Cc:

Description (last modified by nyurik)

Wikipedia tags need a few more validators with fixers:

  • update the list of valid Wikipedia languages
  • auto-replace language be-x-old with be-tarask
  • auto-replace language cz with cs
  • remove space before page title ("en: Foo" -> "en:Foo")
  • replace "_" with spaces ("en:Foo_bar" -> "en:Foo bar")
  • capitalize first letter if it's not "jbo" language ("en:foo" -> "en:Foo")
  • double language link ("en:en:Foo" -> "en:Foo")

This patch also adds a few additional validator functions: trim, upper and lower

Attachments (5)

wikipedia-validations.patch (8.3 KB ) - added by nyurik 8 years ago.
patch to fix this issue
wikipedia-validations.2.patch (10.0 KB ) - added by nyurik 8 years ago.
overrides previous patch
wikipedia-validations.3.patch (10.9 KB ) - added by nyurik 8 years ago.
wikipedia-validations.4.patch (12.3 KB ) - added by nyurik 8 years ago.
wikipedia-validations.5.patch (12.4 KB ) - added by nyurik 8 years ago.

Download all attachments as: .zip

Change History (22)

by nyurik, 8 years ago

Attachment: wikipedia-validations.patch added

patch to fix this issue

comment:1 by nyurik, 8 years ago

Description: modified (diff)

comment:2 by nyurik, 8 years ago

Description: modified (diff)

comment:3 by Don-vip, 8 years ago

Component: CoreCore validator
Keywords: patch validator removed
Milestone: 17.03
Summary: (patch) Improve Wikipedia tag validators[patch] Improve Wikipedia tag validators

by nyurik, 8 years ago

overrides previous patch

comment:4 by nyurik, 8 years ago

Description: modified (diff)

by nyurik, 8 years ago

by nyurik, 8 years ago

comment:5 by Don-vip, 8 years ago

Please don't write this:

 	29	  throwWarning: tr("wikipedia ''be-x-old'' language is obsolete, use ''be-tarask'' instead");
 	36	  throwWarning: tr("wikipedia ''cz'' language is invalid, use ''cs'' instead");

This makes an entry to translate for each value we will list in an individual check. Instead, this only creates a generic string to translate:

 	29	  throwWarning: tr("wikipedia ''{0}'' language is obsolete, use ''{1}'' instead", "be-x-old", "be-tarask");
 	36	  throwWarning: tr("wikipedia ''{0}'' language is invalid, use ''{1}'' instead", "cz", "cs");

I didn't check the other strings, you see the point :)

by nyurik, 8 years ago

comment:6 by nyurik, 8 years ago

@Don-vip, please recheck. I so wish we used git for this. Attaching and revising patches like this is very arcane.

comment:7 by nyurik, 8 years ago

@don-vip, is something still missing for this ticket to be merged?

comment:8 by Don-vip, 8 years ago

Sorry I forgot this one I'll review it tonight :)

comment:9 by Don-vip, 8 years ago

Resolution: fixed
Status: newclosed

In 11756/josm:

fix #14425 - Improve Wikipedia tag validators (patch by nyurik, modified)

comment:10 by Don-vip, 8 years ago

Thanks for the patch! :)
However for the next time, please test it with JOSM first, I had to fix many syntax issues (wrong quote characters, missing semicolons, etc.)

comment:11 by Klumbumbus, 8 years ago

Resolution: fixed
Status: closedreopened

"wikipedia tag is set, but there is no wikidata tag. Wikipedia plugin might help with wikidata id lookups" does this really fit into the category warning? wikidata tag seems to me an optional tag.

comment:12 by Don-vip, 8 years ago

Resolution: fixed
Status: reopenedclosed

In 11771/josm:

fix #14425 - reduce wikidata warning to info level

comment:13 by Klumbumbus, 8 years ago

@nyurik: please add documentation for the new functions at wiki:Help/Styles/MapCSSImplementation

comment:14 by Don-vip, 8 years ago

Resolution: fixed
Status: closedreopened

I didn't notice that the assertions are not met:

GRAVE: Expecting test 'wikipedia tag has no language given, use 'wikipedia'='language:page title'' (i.e., [*[wikipedia][wikipedia'NREGEX'^[-a-z]{2,12}:]]) to not match node wikipedia=en-GB:Foobar (i.e., TagMap[wikipedia=en-GB:Foobar])
GRAVE: Expecting test 'wikipedia tag has an unknown language prefix' (i.e., [*[wikipedia'REGEX'^[-a-z]{2,12}:][wikipedia'NREGEX'^https?:\/\/][wikipedia'NREGEX'^(aa|ab|ace|ady|ady-cyrl|aeb|aeb-arab|aeb-latn|af|ak|aln|als|am|an|ang|anp|ar|arc|arn|arq|ary|arz|as|ase|ast|av|avk|awa|ay|az|azb|ba|ban|bar|bat-smg|bbc|bbc-latn|bcc|bcl|be|be-tarask|be-x-old|bg|bgn|bh|bho|bi|bjn|bm|bn|bo|bpy|bqi|br|brh|bs|bto|bug|bxr|ca|cbk-zam|cdo|ce|ceb|ch|cho|chr|chy|ckb|co|cps|cr|crh|crh-cyrl|crh-latn|cs|csb|cu|cv|cy|cz|da|de|de-at|de-ch|de-formal|din|diq|dsb|dtp|dty|dv|dz|ee|egl|el|eml|en|en-ca|en-gb|eo|es|et|eu|ext|fa|ff|fi|fit|fiu-vro|fj|fo|fr|frc|frp|frr|fur|fy|ga|gag|gan|gan-hans|gan-hant|gd|gl|glk|gn|gom|gom-deva|gom-latn|gor|got|grc|gsw|gu|gv|ha|hak|haw|he|hi|hif|hif-latn|hil|ho|hr|hrx|hsb|ht|hu|hy|hz|ia|id|ie|ig|ii|ik|ike-cans|ike-latn|ilo|inh|io|is|it|iu|ja|jam|jbo|jut|jv|ka|kaa|kab|kbd|kbd-cyrl|kea|kg|khw|ki|kiu|kj|kk|kk-arab|kk-cn|kk-cyrl|kk-kz|kk-latn|kk-tr|kl|km|kn|ko|ko-kp|koi|kr|krc|kri|krj|krl|ks|ks-arab|ks-deva|ksh|ku|ku-arab|ku-latn|kv|kw|ky|la|lad|lb|lbe|lez|lfn|lg|li|lij|liv|lki|lmo|ln|lo|loz|lrc|lt|ltg|lus|luz|lv|lzh|lzz|mai|map-bms|mdf|mg|mh|mhr|mi|min|mk|ml|mn|mo|mr|mrj|ms|mt|mus|mwl|my|myv|mzn|na|nah|nan|nap|nb|nds|nds-nl|ne|new|ng|niu|nl|nl-informal|nn|no|nod|nov|nrm|nso|nv|ny|nys|oc|olo|om|or|os|ota|pa|pag|pam|pap|pcd|pdc|pdt|pfl|pi|pih|pl|pms|pnb|pnt|prg|ps|pt|pt-br|qu|qug|rgn|rif|rm|rmy|rn|ro|roa-rup|roa-tara|ru|rue|rup|ruq|ruq-cyrl|ruq-latn|rw|rwr|sa|sah|sat|sc|scn|sco|sd|sdc|sdh|se|sei|ses|sg|sgs|sh|shi|shi-latn|shi-tfng|shn|si|simple|sje|sk|sl|sli|sm|sma|smj|sn|so|sq|sr|sr-ec|sr-el|srn|srq|ss|st|stq|su|sv|sw|szl|ta|tcy|te|tet|tg|tg-cyrl|tg-latn|th|ti|tk|tl|tly|tn|to|tokipona|tpi|tr|tru|ts|tt|tt-cyrl|tt-latn|tum|tw|ty|tyv|tzm|udm|ug|ug-arab|ug-latn|uk|ur|uz|uz-cyrl|uz-latn|ve|vec|vep|vi|vls|vmf|vo|vot|vro|wa|war|wo|wuu|xal|xh|xmf|yi|yo|yue|za|zea|zh|zh-classical|zh-cn|zh-hans|zh-hant|zh-hk|zh-min-nan|zh-mo|zh-my|zh-sg|zh-tw|zh-yue|zu):]]) to match node wikipedia=X-Y-Z:Foobar (i.e., TagMap[wikipedia=X-Y-Z:Foobar])
GRAVE: Expecting test 'wikipedia title should not have 'wiki/' prefix' (i.e., [*[wikipedia'REGEX'^[-a-z]{2,12}:wiki\/]]) to match node wikipedia=en: foo (i.e., TagMap[wikipedia=en: foo])

Can you please submit a patch that fix them? To check it you can simply run MapCSSTagCheckerTest.testInit() as JUnit test.

comment:15 by Don-vip, 8 years ago

Owner: changed from team to nyurik
Status: reopenedneedinfo

comment:16 by bastiK, 8 years ago

Resolution: fixed
Status: needinfoclosed

In 11800/josm:

fixed #14425 - fix wikipedia tag validator regexes

comment:17 by Klumbumbus, 7 years ago

In 12415/josm:

fix #14947, see #14425 - fix wikipedia warning message

Modify Ticket

Change Properties
Set your email in Preferences
Action
as closed The owner will remain nyurik.
as The resolution will be set.
The resolution will be deleted. Next status will be 'reopened'.

Add Comment


E-mail address and name can be saved in the Preferences .
 
Note: See TracTickets for help on using tickets.