[Maria-developers] JSON libraries
Hi. So i spent quite some time trying to get in use an existing JSON-parsing libraries. But surprisingly enough none seemed to be good enough. Problems i met were like this: 1. Parsers are just slow. - because they copy data - particularly key names and values. - because they use memory allocation/deallocations - because they convert numbers from strings to some numeric formats. Not that i criticise them - it's all mostly because authors of these libraries were concentrating on the ease of use for the inexperienced user. 2. They have to implement the support for different character sets, and cannot use our code for charsets directly. That leads to further slowdown of execution and some inconsistencies with the existing behaviour. 3. Their interface is inconvenient. In order to implement the required JSON functions we'd like to start/stop parsing on various stages, skip parts of JSON, insert something to the found place etc. With the usual interface of the libraries doing all this looks really ugly and inefficient. So that MySQL doesn't even try to implement this - it only uses the library to convert the JSON to internal format and then only operates with that internal. 4. Libraries tend to have their own ways of reporting errors while we'd like use our error messages. Now MySQL just prints the library-generated messages, they cannot translate or somehow modyfy them. 5. We have to parse not only JSON, but also json-paths, as parts of various JSON functions. Libraries don't usually parse the json-path. Partly because these paths can be very different in various applications. Still parts of the path (strings to be specific) should be parsed just as JSON, and it'd be good to use the JSON parser for it, which can require ugly tricks. Like MySQL constructs JSON from that piece of json-path to be able to feed that to the JSON parser. Above all that, i must note that the JSON is pretty simple and compact as a data format. And i seriously belive handling it can be done with great efficiency. So i decided to invest in local JSON parser, taking JSON_CHECKER as a source of inspiration. That JSON_CHECKER was produced by JSON.org itself, is really fast and nice. But it's not a parser - it just checks if the text makes the correct JSON. Well it took some time, but now ready. Now opening it for review. Best regards. HF
On Tue, May 31, 2016 at 03:35:22PM +0400, Alexey Botchkov wrote: ..
That JSON_CHECKER was produced by JSON.org itself, is really fast and nice. But it's not a parser - it just checks if the text makes the correct JSON. Well it took some time, but now ready. Now opening it for review.
So where one can look at it? BR Sergei -- Sergei Petrunia, Software Developer MariaDB Corporation | Skype: sergefp | Blog: http://s.petrunia.net/blog
http://www.json.org/JSON_checker/ 31.05.2016 16:53, Sergey Petrunia wrote:
On Tue, May 31, 2016 at 03:35:22PM +0400, Alexey Botchkov wrote: ..
That JSON_CHECKER was produced by JSON.org itself, is really fast and nice. But it's not a parser - it just checks if the text makes the correct JSON. Well it took some time, but now ready. Now opening it for review. So where one can look at it?
BR Sergei
participants (2)
-
Alexey Botchkov
-
Sergey Petrunia