Page 1 of 1

trying to output JSONL (anybody heard of it?)

Posted: Thu Aug 26, 2021 3:27 pm
by fsuinnc
downloaded Scott's YAJL because I was told a project might be easier if I could output to JSON instead of .csv but in reality I need JSONL which is JSON Lines. a JSON structured data processed a line at a time. If i'm reading things correctly each line is like a complete JSON object (or file whichever is correct). I'm wondering if anyone has used or has advice to get the output this way. So far my attempts create a file that just contains the last record or all records as a single line.
sample of .jsonl -
sample.zip
(421 Bytes) Downloaded 1234 times
thanks.

Re: trying to output JSONL (anybody heard of it?)

Posted: Sat Aug 28, 2021 4:26 am
by Scott Klement
JSONL is basically just a collection of multiple json documents that are stored together in a text file, one per line.

It should be possible to create this in YAJL simply by constructing a separate document for each line, and reading it into a variable. Then write the doc to a text file yourself, with one JSON document per line.

For reading, you'd read it line-by-line, and for each line parse it as a separate document.

Re: trying to output JSONL (anybody heard of it?)

Posted: Mon Aug 30, 2021 6:47 pm
by fsuinnc
belated thanks. will give it a try.

Re: trying to output JSONL (anybody heard of it?)

Posted: Mon Aug 30, 2021 7:55 pm
by jonboy49
Looking at the sample data it looks as if an easy approach would be to use DATA-GEN with the YAJL generator to place the json in a varchar variable, then add the CR/LF (or whatever you need) and write it out to the IFS. Could easily buffer it in a large varchar field.