Support decoding and encoding of LaTeX characters

[koppor]

There is latex2utf8, which source is https://github.com/fc7/LaTeX-Decode. The opposite is LaTeX::Encode.

We should think of including this in JabRef. Possible in the cleanup functionality.

See also #160.

[koppor]

This is related with sf bug #721. It seems that this functionality is done during import and not at "Cleanup entries". The existing functionality should be checked with the one of latex2utf8. Then, one of it should be chosen and integrated at "Cleanup entries".

[oscargus]

I've browsed the code and (having written most of the current JabRef converter, in HTMLConverter), I'd say that the current one supports more characters (although there may be some missing which are worthwhile adding). Also, the current implementation supports converting from HTML. I would assume that it should be possible to use the same table to do the reverse conversion.

[oscargus]

I started merging the missing characters that were present in latex2utf8 and will provide a PR in a few days.

[oscargus]

There's a huge list at http://www.w3.org/Math/characters/unicode.xml

[koppor]

The Unicode converter converts from Unicode to LaTeX and not vice versa, does it? At my first try, it did not treat the Author field, but at a second try it did. Need to investigate what could have been gone wrong.

[oscargus]

Correct. It should be possible to do it the other way around as well similar to the export formatters XMLChars, RTFChars, and HTMLChars.

Especially, one would like to use the huge list in HTMLConverter for HTMLChars (and maybe XMLChars) as well. I think one major issue here is how to deal with {\"{a}} vs \"{a} vs \"a vs {\"a}, but looking at e.g. HtmlCharsMap, it seems like there is a solution for that in HTMLChars, so probably only a matter of converting the LaTeX commands in HTMLConverter to the same format as in HtmlCharsMap.

This is something that I have been thinking about, but so far not succeeded to find the time/motivation to do.

[oscargus]

There is also a class FormatChars that does Latex to Unicode, which could be extended to cover everything in HTMLConverter.

[oscargus]

With #841 there's a huge step towards having quite good conversion in both directions.

[koppor]

Refs #160

[koppor]

Refs #1013.

[tobiasdiez]

What is the status of this issue? It seems like both conversation direction are present as cleanup operations.

[oscargus]

Agreed. Of course, it can always be improved, but I believe it is one of the better conversions (apart from LaTeX).

[lenhard]

This issue can be closed thanks to the cleanup operations