Question: Explain why special characters from paste bin are not mapped in unicode2latex

[ThiloteE]

Thank you for your tool! We at JabRef are using your unicode2latex converter with great success.

I do have a question, as we recently had some related issues (JabRef/jabref#8673) and people were pondering if it were possible to find a better solution how to deal with LaTeX "special characters" like %, $, _, and so on.

I looked into it a little and found that the following symbols

  "#":                         "\\#",
  "$":                         "\\$",
  "%":                         "\\%",
  "&":                         "\\&",

can be found in the "raw" table located at:

• https://pastebin.com/Q4VkkNr1

but not in your code at:

• https://github.com/plurimath/unicode2latex/blob/master/lib/unicode2latex/unicode2latex.rb

I do know that these are special symbols in LaTeX. $ for example starts math-mode and % starts the comment environment.

For example, when users have a $ sign in their bibliographic entry, utilizing the unicod2latex cleanup action, JabRef behaves as if users wanted to start LaTeX math-mode (= leaving the $ as is), instead of converting the sign to \$, which would be rendered as dollar sign compiling in LaTeX.

I am curious, though, about your reasoning. Was including these special characters forgotten, or was it intentional?

[ronaldtse]

Thank you @ThiloteE for the kind message and for filing this! I believe those symbols were initially omitted because they are not typically used within a math formula.

The point you brought up is certainly valid and after a discussion with @opoudjis he would be able to add them in and let you know.

Thank you again for letting us know!

[ThiloteE]

Thanks for the fast feedback!

This issue is tricky. I am not even sure if including them would be the right way to go, hence I was just asking for clarification. I personally was still in the phase of pondering what the best decision would be, but the following reasoning has led me to believe this should maybe be solved at JabRef level:

For a start (let's take $ as an example), including them means,

• people that had math-mode in their entries will suddenly have entries without math-mode. They will have to go through their entire bibliography and check if everything is rendered correctly.
• people that after the conversion will have an \$ and want to go back to math-mode, will want to detect \$ and remove the \ from the entries. But how to know if users after the conversion wanted to have math-mode or a simple $?

There is just no way to know if users want to use math-mode or simply want to have an ordinary $ sign in their bibliographic data!

I think the same is the case for the other "special characters".

Sorry for the noise!

I think we should better solve this issue at JabRef with two separate converters or even better with an integrity check (a kind of search for "special characters" that notifies the user of their existence in the bibliography) and maybe also add better documentation about what will be converted and what not.

Closing this.

[ronaldtse]

Appreciate the note and it is indeed the case — a naïve replace all in TeX content won’t work with this gem, a replace would have to be context specific to a document element or within math mode.