YAJL

Discussions relating to the ScottKlement.com port of the open source YAJL JSON Reader/Generator. This includes the YAJL tool as well as the YAJLR4, YAJLGEN, YAJLINTO and YAJLDTAGEN add-ons from ScottKlement.com. http://www.scottklement.com/yajl/
Post Reply
Emgalli
Posts: 2
Joined: Thu Mar 03, 2022 1:44 pm

YAJL

Post by Emgalli »

I'm using the YAJL with OS 7.1 to do Json and it works great but I have this situation where a request for a Json format doesn't require using YAJL_BeginArray section each record process is it's own json record. I tried to use yajl.biginObj, yajl.addChar, yajl.savebuf for multiple reads but only got the first set of data to write to the IFS and get this error - "do_genValue: received YAJL generator status 4 for type code 1". I'm not sure how to code it with YAJL
Here is an example: each record starts with meta

{"meta":{"dataDomain":"Store"},"data":{"batchId":"0050568295f01edaad868fc74a390608","dataType":"store","action":"CreateOrUpdate","retailLocationData":{"retailLocationDataType":"RetailLocation","retailLocationId":"1000","translations":{"en":{"name":"APTOS STORE #0"}},"address":{"addressLine1":"16611 CABOT DR STE A","city":"CURTIS BAY","stateOrProvince":"MD","postalCode":"21226-1766","countryCode":"USA"},"phoneNumbers":[{"phoneNumber":"(510)555-2468","typeCode":"business"}],"currencyCode":"USD","primaryLanguage":"en-US","taxZoneId":"2000","active":true}}}
{"meta":{"dataDomain":"Store"},"data":{"batchId":"0050568295f01edaad868fc74a390608","dataType":"store","action":"CreateOrUpdate","retailLocationData":{"retailLocationDataType":"RetailLocation","retailLocationId":"1001","translations":{"en":{"name":"APTOS STORE #1"}},"address":{"addressLine1":"1450 PREMIUM OUTLETS BLVD","city":"HAGERSTOWN","stateOrProvince":"MD","postalCode":"21740-9524","countryCode":"USA"},"phoneNumbers":[{"phoneNumber":"(301)666-2812","typeCode":"business"}],"currencyCode":"USD","primaryLanguage":"en-US","taxZoneId":"2001","active":true}}}
jonboy49
Posts: 200
Joined: Wed Jul 28, 2021 8:18 pm

Re: YAJL

Post by jonboy49 »

If I understand you correctly the data you showed comes from two "records" and each is a complete JSON document. Certainly they are only valid JSON if treated that way.

The easiest approach I can think of is to split the "records" up and process each separately. You don't say if the data is coming at you in a file (in which case the split is at end of record) or in a variable (split on '}}}{' maybe ?) but either way it shouldn't be that difficult to process it in pieces.
Scott Klement
Site Admin
Posts: 636
Joined: Sun Jul 04, 2021 5:12 am

Re: YAJL

Post by Scott Klement »

Here's a site where you can paste in a JSON document and validate it: https://jsonlint.com/ if you copy/paste your document, you'll see that it has a problem with the part where the two objects are back-to-back.

Likewise, you can look at the official JSON specs, here: https://www.json.org/json-en.html

If you read that spec carefully, you'll see that a json document consists of one element. It may be an object, array, string, number or boolean... but it's just one element. The point being: You have two consecutive objects, that's two elements... it's not a valid JSON document.

As Jon points out, you could think of it as two separate documents that have been concatenated... that's about the only way you can consider it valid.

If you wanted to generate it with YAJL, you'd need to generate two documents, save them to strings or files, and then concatenate them together. (DATA-GEN has a feature where it can append to an existing file, so that might be one way.)

Reading them will be more difficult. You can split it up and parse them as two separate documents -- but, figuring out how to split them would be tricky. I suppose you could scan with a regex or similar for } followed by whitespace, followed by {, and then split it between the braces, and parse them as two documents.

But, I wonder why they are doing things this way? I'm not aware of any JSON tool that allows this without you having to do extra work. This seems like a very strange design.
Emgalli
Posts: 2
Joined: Thu Mar 03, 2022 1:44 pm

Re: YAJL

Post by Emgalli »

Thanks Scott after your post it was an export file and I now understood the process, so I basically opened and close the GEN for each record loop and cloned your yajl_saveBuf... to yajl_saveBufRec... in my own program and created and appended each record into the Json file.

Thanks you for your help

Code: Select all

*+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++                           
* yajl_saveBufRec():  Copy YAJL generator buffer to IFS                                      
*                                                                                            
*    stmf = (input) path to stream file in IFS to save to                                    
*    mode = (input) "REPLACE" for creating the file 'APPEND" to append                       
*  errMsg = (output) error that occcured (if any)                                            
*                                                                                            
*  returns the YAJL generator status.                                                        
*+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++                           
 yajl_saveBufRec...                                                                          
                 B                   export                                                  
                 PI            10i 0                                                         
   stmf                      5000a   varying const                                           
   mode                         7a                                                           
D err             s             10i 0 based(p_err)                                            
D fd              s             10i 0                                                         
D rc              s             10i 0                                                         
D jsonBuf         s               *                                                           
D jsonSize        s             10u 0                                                         
 /free                                                                                        
  errMsg = '';                                                                                
                                                                                              
  rc = yajl_getBuf( jsonBuf: jsonSize );                                                      
  if rc <> yajl_gen_status_ok;                                                                
     errMsg = 'Failure reading YAJL generator buffer';                                        
     return rc;                                                                               
  endif;                                                                                      
    if   mode = 'REPLACE';                                                                    
         fd = open( stmf                                                                      
              : O_WRONLY + O_CREAT + O_TRUNC + O_TEXTDATA +                                   
                O_INHERITMODE + O_CCSID                                                       
              : 0                                                                             
              : CCSID_UTF8                                                                    
              : CCSID_UTF8 );                                                                 
         else;                                                                                 
         fd = open( stmf                                                                       
              : O_WRONLY + O_CREAT + O_TEXTDATA +                                              
                O_INHERITMODE + O_CCSID                                                        
              : 0                                                                              
              : CCSID_UTF8                                                                     
              : CCSID_UTF8 );                                                                  
         endif;                                                                                
                                                                                               
  if (fd = -1);                                                                                
     p_err = get_system_errno();                                                               
     errMsg = %str(strerror(err));                                                             
     rc = yajl_gen_open_fail;                                                                  
     return rc;                                                                                
  endif;                                                                                       
                                                                                               
    mem.fds += 1;                                                                              
                                                                                               
    if   MODE = 'APPEND';                                                                      
         rc = lseek(FD: 0: SEEK_END);       
    endif;                                                                     
                                                                               
  if write( fd: jsonBuf: jsonSize ) <> jsonSize;                               
     rc = yajl_gen_write_fail;                                                 
     errMsg = 'Error writing JSON data to disk.';                              
     callp close(fd);                                                          
       mem.fds -= 1;                                                           
     return rc;                                                                
  endif;                                                                       
                                                                               
  callp close(fd);                                                             
    mem.fds -= 1;                                                              
                                                                               
  return yajl_gen_status_ok;                                                   
 /end-free                                                                     
P                 E
[/code[
Post Reply