CLDR-11874 Test to verify Subdivisions vs Common

[srl295]

CLDR-11874

• test to verify common/subdivisions vs common/main

• tool to copy from common/main to common/subdivisions

• logKnownIssue on CLDR-18296 for any data in common/subdivisions missing from common/main

• This PR completes the ticket.

ALLOW_MANY_COMMITS=true

[macchiati]

The common/main is primary, so the json generation code should just skip paths in a common/subdivisions locale that have values in the corresponding common/main locale.

[srl295]

The generation code just takes a CLDRFile, it would have to do some acrobatics in order to pull from the other file. We don’t have any thing that does that sort of loading. It might be a lot simpler and better from a data perspective to copy from common/main into subdivisions - would that work?

[macchiati]

that would work. I commented yesterday in response to a slack question. Copying here:

Ah, now I see what the problem is. There is a 4th choice, which is to read from 2 CLDRFiles per locale. Get the merged list of locales. For each locale, load the subdivision/ data into a map<path, value>. Then load on top of that the main/ values filtered to subdivision paths. Then write out the map (if non-empty). The downside is that it uses different loading logic than the normal. (edited)

However, I'm ok with (C); add a CLDRModify step to copy from main/ to subdivisions/. Have a unit test that you run before json generation to verify

[srl295]

approval status copied from common/main (via survey) into subdivisions

[srl295]

an actual change (and approval) from common/main

[srl295]

data in a new locale

[srl295]

with this change cldr json generation works properly.

[macchiati]

Very good.

One request: could you file a ticket to make CLDRFile.make be thread-safe, to avoid

                                synchronized (CLDRFile.class) {
                                    mainF = mainFactory.make(loc, true);
                                    subF = subFactory.make(par.getBaseName(), true);
                                }

I'm sure there are other places in the code that aren't as careful as you are!

[srl295]

Very good.

One request: could you file a ticket to make CLDRFile.make be thread-safe, to avoid

                                synchronized (CLDRFile.class) {
                                    mainF = mainFactory.make(loc, true);
                                    subF = subFactory.make(par.getBaseName(), true);
                                }

I'm sure there are other places in the code that aren't as careful as you are!

Couldn't reproduce the issue now, i thought I had put a stack trace somewhere but can't find it.