Re: Question about Weblate scripts

[Libor Polčák]

Hello Ricardo,


> At this point we could really use some guidance -- are the failures expected? 
> Do they occur because of our changes to the weblate script?
Can you report what you get by:

git remove -v

(You should see a line like:
pagure  ssh://git@pagure.io/JShelter/webextension.git (fetch)
)

I will use $PAGURE as the name in the first column bellow. So in my case 
$PAGURE==pagure, you might see origin or something different.

git fetch $PAGURE

git checkout dd7144fe7803c438f77e55c56dd248b0fea0252b
git switch -c debugbranch
git stash
git status

Please have a look at the list of Untracked files. Remove/backup anything in 
tools/i18n/ or common/_locales/ or any other suspicious file.

$ LC_ALL="C" tools/i18n/weblate2main.py
I suppose that you have a branch weblate mirroring 
https://hosted.weblate.org/git/jshelter/webextension/. Make sure that your 
working tree is clear. Use git stash or similar commands if necessary.

Locales in current branch debugbranch
cs en ru
Press Enter to continue or exit with SIGINT (Ctrl+C)

M       tools/i18n/weblate2main.py
Switched to branch 'weblate'
Write YES if you want to import locale fr ... NO
M       tools/i18n/weblate2main.py
Switched to branch 'debugbranch'

There should be no new commit and the script should not produce any error.

Is that what you get or something different? See also my inline comments below.

> Hi all,
>
> We've been experimenting with Weblate for handling the website Markdown files. 
> Even though their Markdown support is experimental, it's apparently doable. We 
> created a component, website-faq (markdown requires a component per file) to 
> try things out; apparently strings are extracted properly, but we still have to 
> confirm whether the translations are properly applied back.
>
> But we couldn't run the i18n helper scripts. First, there was a python error:
>
> Traceback (most recent call last):
>    File "/home/rlafuente/repos/jshelter/./tools/i18n/weblate2main.py", line 111, in 
> <module>
>      bl[key]["description"] == "" and "description" not in wl[key] and \
>      ~~~~~~~^^^^^^^^^^^^^^^
> KeyError: 'description'

That's weird. I can run weblate2main.py at my computer fine from 
dd7144fe7803c438f77e55c56dd248b0fea0252b.

I tried running that script and not importing the partial French translation of 
one of our users and also with the import. Both times it created a good commit.

One question: Do you have a partial pt translation? What is the output of "ls 
common/_locales/"? At my computer, it is:

cs  en  ru

I tried to create a pt directory and replicate your error, but if I do not 
create the message.json, I get:

    f = open("%s/%s/messages.json" % (LOCALES_PATH, locale))
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
FileNotFoundError: [Errno 2] No such file or directory: 
'common/_locales/pt/messages.json'

When I create that file by copying the English file, everything works.

Have you added anything to common/_locales/en/messages.json?

> The KeyErrors can be dealt with by using .get() instead, so we made these 
> changes which seemed to address the error:
>
>               # Check if the only difference is a missing empty description
>               if bl[key]["message"] == wl[key]["message"] and \
> -                bl[key]["description"] == "" and "description" not in wl[key] 
> and \
> +                not bl[key].get("description") and "description" not in 
> wl[key] and \

Here, I expect that there is always the description key in the base EN JSON.

>                   (("placeholders" not in bl[key] and "placeholders" not in 
> wl[key]) or \
> -                (["placeholders"] in bl[key] and "placeholders" in wl[key] and 
> \
> +                ("placeholders" in bl[key] and "placeholders" in wl[key] and \

I am not sure what bl[key] holds but this looks like a good catch. Can you 
confirm that there is not an array?

> -                bl[key]["placeholders"] == wl[key]["placeholders"])):
> + bl[key].get("placeholders") == wl[key]["placeholders"])):
>                       continue # Skip such strings
>
> After the fix, running ./tools/i18n/weblate2main.py from the root repo 
> directory gives out a string of failures:
>
> I suppose that you have a branch weblate mirroring 
> https://hosted.weblate.org/git/jshelter/webextension/. Make sure that your 
> working tree is clear. Use git stash or similar commands if necessary.
>
> Locales in current branch weblate
> fr cs ru en
> Press Enter to continue or exit with SIGINT (Ctrl+C)

The script asks me whether to import French:
Write YES if you want to import locale fr ... NO

I guess that you imported French before.
> Your branch is up to date with 'origin/weblate'.
> patching file common/_locales/cs/messages.json
> Hunk #1 FAILED at 85.
> Hunk #2 FAILED at 164.
> Hunk #3 FAILED at 192.
> Hunk #4 FAILED at 200.
> Hunk #5 FAILED at 223.
> Hunk #6 FAILED at 245.
> Hunk #7 FAILED at 294.
> Hunk #8 FAILED at 319.
> Hunk #9 FAILED at 355.
> Hunk #10 FAILED at 446.
> Hunk #11 FAILED at 624.
> Hunk #12 FAILED at 661.
> Hunk #13 FAILED at 781.
> Hunk #14 FAILED at 833.
> Hunk #15 FAILED at 845.
> Hunk #16 FAILED at 877.
> Hunk #17 FAILED at 939.
> Hunk #18 FAILED at 955.
> Hunk #19 FAILED at 971.
> Hunk #20 FAILED at 995.
> Hunk #21 FAILED at 1007.
> Hunk #22 FAILED at 1059.
> Hunk #23 FAILED at 1119.
> Hunk #24 FAILED at 1131.
> Hunk #25 FAILED at 1147.
> Hunk #26 FAILED at 1163.
> Hunk #27 FAILED at 1179.
> Hunk #28 FAILED at 1187.
> Hunk #29 FAILED at 1203.
> Hunk #30 FAILED at 1355.
> Hunk #31 FAILED at 1379.
> Hunk #32 FAILED at 1391.
> Hunk #33 FAILED at 1411.
> 33 out of 33 hunks FAILED -- saving rejects to file 
> common/_locales/cs/messages.json.rej

            sp = subprocess.run(["diff", "-Naur", "--ignore-all-space", 
"--ignore-blank-lines",
                                 tmp.name, git_translation_filename],
                                 capture_output=True)
            patch_tmp = tempfile.NamedTemporaryFile(mode="w", suffix = ".patch", prefix = 
"jshelter_%s_" % locale, delete=False)

I guess that you might have a different version of diff that does not support  
"-Naur --ignore-all-space --ignore-blank-lines"?

Hmm, I guess that you are using different locales. Can you try:

LC_ALL="C" tools/i18n/weblate2main.py
?

            patch_tmp.write(sp.stdout.decode("utf-8"))
            patch_tmp.close()
            os.remove(tmp.name)

Maybe try to remove/comment that line and have a look what was created.

At my computer with dd7144fe7803c438f77e55c56dd248b0fea0252b, I do not get to 
that code branch as there is nothing to apply.

            subprocess.run(["patch", "-R", git_translation_filename, 
patch_tmp.name])
 
Best wishes

Libor