TI-5: Unicode tags not working.

- Thanks to Sergey Trofimov.
This commit is contained in:
Paul Beckingham 2016-06-06 22:48:53 -04:00
parent 12dfee1bc8
commit 54eb2658a1
3 changed files with 14 additions and 4 deletions

View file

@ -11,6 +11,7 @@ The following submitted code, packages or analysis, and deserve special thanks:
Dirk Deimeke Dirk Deimeke
Wim Schuermann Wim Schuermann
Jelle van der Waa Jelle van der Waa
Sergey Trofimov
Thanks to the following, who submitted detailed bug reports and excellent Thanks to the following, who submitted detailed bug reports and excellent
suggestions: suggestions:

View file

@ -7,6 +7,8 @@
(thanks to Jelle van der Waa). (thanks to Jelle van der Waa).
- TI-2 Two line tags causes false entry in database - TI-2 Two line tags causes false entry in database
(thanks to Sebastian Uharek). (thanks to Sebastian Uharek).
- TI-5 Unicode tags not working.
(thanks to Sergey Trofimov).
- Added 'continue' command. - Added 'continue' command.
- Added 'diagnostics' command. - Added 'diagnostics' command.
- Added 'export' command. - Added 'export' command.

View file

@ -30,6 +30,7 @@
#include <Pig.h> #include <Pig.h>
#include <shared.h> #include <shared.h>
#include <format.h> #include <format.h>
#include <utf8.h>
#include <sstream> #include <sstream>
#include <algorithm> #include <algorithm>
@ -145,10 +146,16 @@ void CLI::entity (const std::string& category, const std::string& name)
void CLI::add (const std::string& argument) void CLI::add (const std::string& argument)
{ {
// Sanitize the input: Convert control charts to spaces. Then trim. // Sanitize the input: Convert control charts to spaces. Then trim.
auto clean = argument; std::string clean;
for (auto& c : clean) std::string::size_type i = 0;
if (c <= 32) int character;
c = ' '; while ((character = utf8_next_char (argument.c_str (), i)))
{
if (character <= 32)
clean += ' ';
else
clean += utf8_character (character);
}
A2 arg (Lexer::trim (clean), Lexer::Type::word); A2 arg (Lexer::trim (clean), Lexer::Type::word);
arg.tag ("ORIGINAL"); arg.tag ("ORIGINAL");