DATA-INTO Failing (Different Issue)

Discussions relating to the ScottKlement.com port of the open source YAJL JSON Reader/Generator. This includes the YAJL tool as well as the YAJLR4, YAJLGEN, YAJLINTO and YAJLDTAGEN add-ons from ScottKlement.com. http://www.scottklement.com/yajl/
Post Reply
emhill
Posts: 43
Joined: Thu Jul 29, 2021 1:15 pm

DATA-INTO Failing (Different Issue)

Post by emhill »

I had this issue with one program and thanks to JonBoy49 was able to get past it. I'm now getting the same error but it has a little different twist to it.

I'm using a handler due to the size of the incoming JSON. Here is my DS:

Code: Select all

dcl-ds REPAIRINGOUTLETRECORD_t qualified template;               
  REPAIRINGOUTLETBUSINESSPARTNERCODE varchar(10) inz('');        
  REPAIRINGOUTLETNAME varchar(30) inz('');                       
  REPAIRINGOUTLETADDRESSLINE1 varchar(30) inz('');               
  REPAIRINGOUTLETCITY varchar(18) inz('');                       
  REPAIRINGOUTLETSTATE varchar(2) inz('');                       
  REPAIRINGOUTLETCOUNTRY varchar(2) inz('');                     
  REPAIRINGOUTLETPOSTALCODE varchar(10) inz('');                 
  KILOMETERINDICATOR varchar(2) inz('');                         
  CURRENTLABORRATE packed(9:2) inz(0);                           
  CURRENTLABORRATEEFFECTIVEDATE varchar(10) inz('');             
  CURRENTTRAVELDISTANCEREIMBURSEMENTRATE packed(9:2) inz(0);     
  CURRENTTRAVELDISTANCEREIMBURSEMENTRATEEFFECTIVEDATE varchar(10)
      INZ(' ');                                                  
  CURRENTLANDEDCOSTMARKUP packed(9:2) inz(0);                    
  CURRENTLANDEDCOSTMARKUPEFFECTIVEDATE varchar(10) inz('');      
  num_CLAIMPROCESSEDRECORD int(10) inz(0);                       
  dcl-ds CLAIMPROCESSEDRECORD dim(100);                          
    TRACKINGNUMBER varchar(10) inz('');                          
    REPAIRORDERNUMBER varchar(8) inz('');                        
    PRODUCTSERIALNUMBER varchar(20) inz('');                     
    CLAIMNUMBER varchar(20) inz('');                             
    MODELNUMBER varchar(15) inz('');                             
    REPAIRORDEROPENDATE varchar(10) inz('');
    DELIVERYDATE varchar(10) inz('');              
    PRIMARYFAILEDPARTNAME varchar(25) inz('');     
    PRIMARYFAILEDPARTNUMBER varchar(20) inz('');   
    CLAIMSTATUSCODE varchar(3) inz('');            
    CLAIMSTATUSCODEDESCRIPTION varchar(20) inz('');
    CLAIMTYPE varchar(2) inz('');                  
    EXCHANGERATE varchar(10) inz('');              
    PREVIOUSCREDITAMOUNT packed(10:2) inz(0);      
    PARTCREDITTOTAL packed(9:2) inz(0);            
    PARTMARKUPTOTAL packed(9:2) inz(0);            
    LABORCREDITTOTAL packed(10:2) inz(0);          
    NETITEMCREDITTOTAL packed(9:2) inz(0);         
    NETTOTALCREDITDOLLARS packed(9: 2) inz(0);     
    REQUESTEDTOTALDOLLARS packed(9: 2) inz(0);     
    DIFFERENCETOTALDOLLARS packed(9: 2) inz(0);    
    DOCUMENTNUMBER varchar(20) inz('');            
    num_CLAIMLINEDETAILRECORD int(10) inz(0);      
    dcl-ds CLAIMLINEDETAILRECORD dim(150);         
      TRACKINGNUMBER varchar(10) inz('');          
      CLAIMNUMBER varchar(20) inz('');             
      APPROVEDQUANTITY packed(5:1) inz(0);         
      REPLACEDPARTNUMBER varchar(15) inz('');      
      PARTNAME varchar(1) inz('');                 
      LABOROPERATIONCODE varchar(8) inz('');       
      PARTLANDEDMARKUP packed(9:2) inz(0);
      NETITEMCODE varchar(10) inz('');    
      PRORATE packed(9:2) inz(0);         
      TYPE varchar(10) inz('');           
      GROSSCREDIT packed(9:2) inz(0);     
      TOTALCREDIT packed(9:2) inz(0);     
      PROCESSCODE varchar(10) inz('');    
      PROCESSMESSAGE varchar(80) inz(''); 
    end-ds;                               
  end-ds;                                 
end-ds;                                                                 
And here is the data-into:

Code: Select all

options = 'doc=file case=convert countprefix=num_ allowextra=yes ' +
          'path=jsonDoc/CREDITADVICEHEADER/REPAIRINGOUTLETRECORD';  
                                                                    
data-into %Handler( processRepairOutletRecord : totalItems )        
          %DATA( ifsPathname : options )                            
          %PARSER( 'YAJLINTO'                                       
                 : '{ "document_name": "jsonDoc", +                 
                      "number_prefix": "YAJL_" }');                 
                      
The program works great until it gets to a certain point in the JSON. This is a scaled down version of the JSON. It will process through
repairingoutletbusinesspartnercode of 0322400401. Then it will give me the error below (shown after the JSON) when it hits 0322400600 partner code:

Code: Select all

{
  "creditAdviceHeader": {
    "creditAdviceNumber": "202202",
    "distributorCode": "03224",
    "creditAdvicePreparationDate": "2022-01-11T15:47:02.122Z",
    "repairingOutletRecord": [
{
        "repairingOutletBusinessPartnerCode": "0322400400",
        "repairingOutletName": "Freightliner of Philadelphia",
        "repairingOutletAddressline1": "11 Runway Road",
        "repairingOutletCity": "Philadelphia",
        "repairingOutletState": "PA",
        "repairingOutletCountry": "US",
        "repairingOutletPostalcode": "19057",
        "kilometerIndicator": "Mi",
        "currentLaborRate": "155.0",
        "currentLaborRateEffectiveDate": "2020-01-06",
        "currentTravelDistanceReimbursementRate": "1.0",
        "currentTravelDistanceReimbursementRateEffectiveDate": "2011-08-30",
        "currentLandedCostMarkup": 0,
        "currentLandedCostMarkupEffectiveDate": "2020-01-06",
        "workInProgressClaimsRecord": [
          
        ],
        "workInProgressETCRecord": [
          
        ],
        "etcRegistrationRecord": [
          
        ],
        "claimProcessedRecord": [
          
        ]
      },
      {
        "repairingOutletBusinessPartnerCode": "0322400401",
        "repairingOutletName": "Freightliner of Lancaster",
        "repairingOutletAddressline1": "1675 Rohrerstown Road",
        "repairingOutletCity": "Lancaster",
        "repairingOutletState": "PA",
        "repairingOutletCountry": "US",
        "repairingOutletPostalcode": "17601",
        "kilometerIndicator": "Mi",
        "currentLaborRate": "135.0",
        "currentLaborRateEffectiveDate": "2017-08-08",
        "currentTravelDistanceReimbursementRate": "1.0",
        "currentTravelDistanceReimbursementRateEffectiveDate": "2011-11-27",
        "currentLandedCostMarkup": 0,
        "currentLandedCostMarkupEffectiveDate": "2018-05-17",
        "workInProgressClaimsRecord": [
          {
            "trackingNumber": "225704",
            "repairOrderNumber": "225704",
            "claimType": "01",
            "productSerialNumber": "6511643920",
            "claimNumber": "AT0493763",
            "dispositionCode": "",
            "dispositionCodeDescription": "",
            "requestedDollars": 271.74
          },
          {
            "trackingNumber": "226159",
            "repairOrderNumber": "226158",
            "claimType": "01",
            "productSerialNumber": "6511403589",
            "claimNumber": "AT0494714",
            "dispositionCode": "",
            "dispositionCodeDescription": "",
            "requestedDollars": 288.16
          },
          {
            "trackingNumber": "102623",
            "repairOrderNumber": "102623",
            "claimType": "01",
            "productSerialNumber": "9630047747",
            "claimNumber": "AT0494295",
            "dispositionCode": "",
            "dispositionCodeDescription": "",
            "requestedDollars": 4008.31
          },
          {
            "trackingNumber": "821627",
            "repairOrderNumber": "821627",
            "claimType": "01",
            "productSerialNumber": "6311963167",
            "claimNumber": "AT0494321",
            "dispositionCode": "",
            "dispositionCodeDescription": "",
            "requestedDollars": 692.17
          }
        ],
        "workInProgressETCRecord": [
          
        ],
        "etcRegistrationRecord": [
          
        ],
        "claimProcessedRecord": [
          {
            "trackingNumber": "226464",
            "repairOrderNumber": "226464",
            "productSerialNumber": "6311819179",
            "claimNumber": "AT0492161",
            "modelNumber": "2100HS",
            "repairOrderOpenDate": "2021-11-22",
            "deliveryDate": "2020-11-12",
            "primaryFailedPartName": "Unable to Determine Caus",
            "primaryFailedPartNumber": "86",
            "claimStatusCode": "40",
            "claimStatusCodeDescription": "Approved",
            "claimType": "01",
            "exchangeRate": "1.0",
            "previousCreditAmount": 0,
            "partCreditTotal": 2620.01,
            "partMarkupTotal": 524,
            "laborCreditTotal": 769.5,
            "netItemCreditTotal": 108.06,
            "netTotalCreditDollars": 4021.57,
            "requestedTotalDollars": 4025.04,
            "differenceTotalDollars": 3.47,
            "documentNumber": "9900195648",
            "claimLineDetailRecord": [
              {
                "trackingNumber": "226464",
                "claimNumber": "AT0492161",
                "approvedQuantity": 3.8,
                "replacedPartNumber": "",
                "partName": "",
                "laborOperationCode": "00094000",
                "partLandedMarkup": 0,
                "netItemCode": "",
                "proRate": 100,
                "type": "Labor",
                "grossCredit": 513,
                "totalCredit": 513,
                "processCode": "",
                "processMessage": ""
              },
              {
                "trackingNumber": "226464",
                "claimNumber": "AT0492161",
                "approvedQuantity": 0.5,
                "replacedPartNumber": "",
                "partName": "",
                "laborOperationCode": "00094006",
                "partLandedMarkup": 0,
                "netItemCode": "",
                "proRate": 100,
                "type": "Labor",
                "grossCredit": 67.5,
                "totalCredit": 67.5,
                "processCode": "",
                "processMessage": ""
              },
              {
                "trackingNumber": "226464",
                "claimNumber": "AT0492161",
                "approvedQuantity": 0.5,
                "replacedPartNumber": "",
                "partName": "",
                "laborOperationCode": "00096901",
                "partLandedMarkup": 0,
                "netItemCode": "",
                "proRate": 100,
                "type": "Labor",
                "grossCredit": 67.5,
                "totalCredit": 67.5,
                "processCode": "",
                "processMessage": ""
              },
              {
                "trackingNumber": "226464",
                "claimNumber": "AT0492161",
                "approvedQuantity": 0.5,
                "replacedPartNumber": "",
                "partName": "",
                "laborOperationCode": "00096400",
                "partLandedMarkup": 0,
                "netItemCode": "",
                "proRate": 100,
                "type": "Labor",
                "grossCredit": 67.5,
                "totalCredit": 67.5,
                "processCode": "",
                "processMessage": ""
              },
              {
                "trackingNumber": "226464",
                "claimNumber": "AT0492161",
                "approvedQuantity": 0.4,
                "replacedPartNumber": "",
                "partName": "",
                "laborOperationCode": "00094100",
                "partLandedMarkup": 0,
                "netItemCode": "",
                "proRate": 100,
                "type": "Labor",
                "grossCredit": 54,
                "totalCredit": 54,
                "processCode": "",
                "processMessage": ""
              },
              {
                "trackingNumber": "226464",
                "claimNumber": "AT0492161",
                "approvedQuantity": 3,
                "replacedPartNumber": "",
                "partName": "",
                "laborOperationCode": "",
                "partLandedMarkup": 0,
                "netItemCode": "Parts",
                "proRate": 100,
                "type": "Other",
                "grossCredit": 108.06,
                "totalCredit": 108.06,
                "processCode": "",
                "processMessage": ""
              },
              {
                "trackingNumber": "226464",
                "claimNumber": "AT0492161",
                "approvedQuantity": 0,
                "replacedPartNumber": "",
                "partName": "",
                "laborOperationCode": "",
                "partLandedMarkup": 0,
                "netItemCode": "Parts",
                "proRate": 100,
                "type": "Other",
                "grossCredit": 0,
                "totalCredit": 0,
                "processCode": "981",
                "processMessage": "Shop supplies not allowed"
              },
              {
                "trackingNumber": "226464",
                "claimNumber": "AT0492161",
                "approvedQuantity": 1,
                "replacedPartNumber": "29563096",
                "partName": "1000RM RETRAN ASSEMBLY",
                "laborOperationCode": "",
                "partLandedMarkup": 524,
                "netItemCode": "",
                "proRate": 100,
                "type": "Part",
                "grossCredit": 3144.01,
                "totalCredit": 3144.01,
                "processCode": "",
                "processMessage": ""
              }
            ]
          }
        ]
      },
      {
        "repairingOutletBusinessPartnerCode": "0322400600",
        "repairingOutletName": "Faulkner GMC",
        "repairingOutletAddressline1": "705 Auto Park Boulevard",
        "repairingOutletCity": "Chester",
        "repairingOutletState": "PA",
        "repairingOutletCountry": "United States",
        "repairingOutletPostalcode": "19382",
        "kilometerIndicator": "",
        "currentLaborRate": "",
        "currentLaborRateEffectiveDate": "",
        "currentTravelDistanceReimbursementRate": "",
        "currentTravelDistanceReimbursementRateEffectiveDate": "",
        "currentLandedCostMarkup": "",
        "currentLandedCostMarkupEffectiveDate": "",
        "workInProgressClaimsRecord": [
          
        ],
        "workInProgressETCRecord": [
          
        ],
        "etcRegistrationRecord": [
          
        ],
        "claimProcessedRecord": [
          
        ]
      },
      {
        "repairingOutletBusinessPartnerCode": "0322400602",
        "repairingOutletName": "Mohawk Valley Freightliner",
        "repairingOutletAddressline1": "703 Oriskany Boulevard",
        "repairingOutletCity": "Yorkville",
        "repairingOutletState": "NY",
        "repairingOutletCountry": "United States",
        "repairingOutletPostalcode": "13495-0201",
        "kilometerIndicator": "",
        "currentLaborRate": "",
        "currentLaborRateEffectiveDate": "",
        "currentTravelDistanceReimbursementRate": "",
        "currentTravelDistanceReimbursementRateEffectiveDate": "",
        "currentLandedCostMarkup": "",
        "currentLandedCostMarkupEffectiveDate": "",
        "workInProgressClaimsRecord": [
          
        ],
        "workInProgressETCRecord": [
          
        ],
        "etcRegistrationRecord": [
          
        ],
        "claimProcessedRecord": [
          
        ]
      },
      {
        "repairingOutletBusinessPartnerCode": "0322400604",
        "repairingOutletName": "Bentley Truck Services  Inc.",
        "repairingOutletAddressline1": "7777 Essington Avenue",
        "repairingOutletCity": "Philadelphia",
        "repairingOutletState": "PA",
        "repairingOutletCountry": "United States",
        "repairingOutletPostalcode": "19153",
        "kilometerIndicator": "",
        "currentLaborRate": "",
        "currentLaborRateEffectiveDate": "",
        "currentTravelDistanceReimbursementRate": "",
        "currentTravelDistanceReimbursementRateEffectiveDate": "",
        "currentLandedCostMarkup": "",
        "currentLandedCostMarkupEffectiveDate": "",
        "workInProgressClaimsRecord": [
          
        ],
        "workInProgressETCRecord": [
          
        ],
        "etcRegistrationRecord": [
          
        ],
        "claimProcessedRecord": [
          
        ]
       }
    ]
  }
}    
    

Code: Select all

                         Additional Message Information                        
                                                                               
 Message ID . . . . . . :   RNX0356       Severity . . . . . . . :   50        
 Message type . . . . . :   Escape                                             
 Date sent  . . . . . . :   03/10/22      Time sent  . . . . . . :   12:50:10  
                                                                               
 Message . . . . :   The document for the DATA-INTO operation does not match   
   the RPG variable; reason code 5.                                            
 Cause . . . . . :   While parsing a document for the DATA-INTO operation, the 
   parser found that the document does not correspond to RPG variable "PARM"   
   and the options do not allow for this. The reason code is 5. The exact      
   subfield for which the error was detected is "PARM(1)". The options are     
   "doc=file case=convert countprefix=num_ allowextra=yes                      
   path=jsonDoc/CREDITADVICEHEADER/REPAIRINGOUTLETRECORD". The document name is
   /Allison/4Ward/Credit Advice Response Example.json; *N indicates that the   
   document is not an external file. The parser is 'YAJLINTO'. *N indicates    
   that the parser is a procedure pointer.                                     
The top portion of the handler procedure:

Code: Select all

dcl-proc processRepairOutletRecord;                      
dcl-pi *n  int(10);                                      
   count  int(10);                                       
   item   likeDS(REPAIRINGOUTLETRECORD_T)  Dim(1)  Const;
   items  int(10)  Value;                                
end-pi;                                                  
Like I said it works wonderfully until it reaches a certain point in the JSON (which is attached).

Any suggestions? If you need any more information, please let me know.

Thanks in advance!
Scott Klement
Site Admin
Posts: 636
Joined: Sun Jul 04, 2021 5:12 am

Re: DATA-INTO Failing (Different Issue)

Post by Scott Klement »

Eric,

Are you certain that you posted the correct error message? I tried your example, but I get a different one:

Code: Select all

                         Additional Message Information                        
                                                                               
 Message ID . . . . . . :   RNX0356       Severity . . . . . . . :   50        
 Message type . . . . . :   Escape                                             
 Date sent  . . . . . . :   03/10/22      Time sent  . . . . . . :   15:30:43  
                                                                               
 Message . . . . :   The document for the DATA-INTO operation does not match   
   the RPG variable; reason code 8.                                            
 Cause . . . . . :   While parsing a document for the DATA-INTO operation, the 
   parser found that the document does not correspond to RPG variable "PARM"   
   and the options do not allow for this. The reason code is 8. The exact      
   subfield for which the error was detected is "PARM(1).currentlaborrate". The
   options are "doc=file case=convert countprefix=num_ allowextra=yes          
   path=jsonDoc/CREDITADVICEHEADER/REPAIRINGOUTLETRECORD". The document name is
   test/Credit Advice Response Example.json; *N indicates that the document is 
   not an external file. The parser is 'YAJLINTO'. *N indicates that the parser
   is a procedure pointer.                                                     
 Recovery  . . . :   Contact the person responsible for program maintenance to 
                                                                        More...
 Press Enter to continue.                                                      
                                                                               
 F3=Exit   F6=Print   F9=Display message details                               
 F10=Display messages in job log   F12=Cancel   F21=Select assistance level    

Code: Select all

  :
   8. The document contains data that cannot be successfully assigned to the   
     RPG variable. The RPG status code associated with the failure is 105.     
  :
Notice that it says RPG status code 105. So I typed the following command to find out what status code 105 means:

Code: Select all

dspmsgd rnx0105 msgf(qrnxmsg)
 :
A character representation of a numeric value is in error. 
 :
Looking at your JSON document you have this:

Code: Select all

  :
   "currentLaborRate": "",
  :
And in RPG the corresponding field is:

Code: Select all

  :
  CURRENTLABORRATE packed(9:2) inz(0);                           
  :
Firstly, the fact that this shows quote marks means that this is NOT intended to be a numeric field. But, you've defined it as packed, which means that any time they send character data that isn't a valid number, you'll get this error.

Second, there's a somewhat new (PTF was available in Fall 2020) feature in RPG called EXPROPTS(*ALWBLANKNUM) that will allow it to interpret blank values as zero. After I added that to my CTL-OPT, I was able to run the program without errors. For more info on EXPROPTS(*ALWBLANKNUM) see here: https://www.ibm.com/support/pages/node/6342819

With that, no errors -- works perfectly.
emhill
Posts: 43
Joined: Thu Jul 29, 2021 1:15 pm

Post by emhill »

Hey Scott,

I think you told me about the EXPROPTS(*ALWBLANKNUM) before and I do have it in my CTL-OPT specs.

I'm getting the reason code of 5.

I'm going to change that currentLaborRate to character and see if that makes a difference. This company has been notorious for sending me examples of the JSON being sent and then just turn around and make wholesale changes without notification.

I'll play a little more and let you know what happens.

Thanks for the prompt reply.
Scott Klement
Site Admin
Posts: 636
Joined: Sun Jul 04, 2021 5:12 am

Re: DATA-INTO Failing (Different Issue)

Post by Scott Klement »

Changing it to character would make sense if you were getting reason code 8 like I was.

Reason code 5 means that your variable name doesn't match the JSON field name. But, I ran your JSON example with your DS,and didn't have that problem. Any idea why mine wouldn't have the same problem?

Also -- and this is probably unrelated to the error -- in the processRepairOutletRecord procedure you have parameter 'items' defined as int(10), but the docs say it should be uns(10). I suspect it'd work fine as long as there are less than 2 billion items in the array (which should never happen) but you might want to change it to uns(10), anyway... just for the sake of "correctness"
emhill
Posts: 43
Joined: Thu Jul 29, 2021 1:15 pm

Re: DATA-INTO Failing (Different Issue)

Post by emhill »

Wow... I don't know. I'll attach my complete program and maybe there is something else going on there that you see. I'm writing to a flat file called ATDRCVWA that is 350 characters long.

This is a first draft so don't laugh too hard! :-)
Attachments
ATD140.zip
(14.45 KiB) Downloaded 185 times
Scott Klement
Site Admin
Posts: 636
Joined: Sun Jul 04, 2021 5:12 am

Re: DATA-INTO Failing (Different Issue)

Post by Scott Klement »

Eric,
  1. You didn't send your program, you sent a printout created by SEU. This includes extra columns before/after the code as well as heading information, etc, because its designed to be sent to a printer. I can certainly strip all of this off -- but, why not send the source itself rather than a printout of the source, so I don't have to do extra work?
  2. The code calls a program called ATD148, which i don't have.
  3. The code uses the external definitions of a file called ATD140FIL which I don't have.
  4. It does SQL statements against files called WAHDR, WAITM, WALAB, WANET, and ICMST which I don't have.
Frankly, I've blown an hour on this and I haven't even gotten as far as trying to compile it.

So that I don't have to waste any more time, can you please create a copy of the program that reproduces the problem without these extra dependencies? It doesn't have to do anything else useful besides illustrate the problem. Make sure you test it before sending it to verify that it compiles, runs, and reproduces the problem.

I'm glad to help, but I don't want to get into the business of spending hours rewriting your code.
emhill
Posts: 43
Joined: Thu Jul 29, 2021 1:15 pm

Re: DATA-INTO Failing (Different Issue)

Post by emhill »

I"m sorry about that. I'll try to get you something better to work with.
emhill
Posts: 43
Joined: Thu Jul 29, 2021 1:15 pm

Re: DATA-INTO Failing (Different Issue)

Post by emhill »

I found it. I was performing this statement in my handler procedure:

Code: Select all

%subst(rec:203:1) = %subst(item(1).kilometerIndicator:1:1);
That was where it was dying. The JSON for that element looked like this:

Code: Select all

"kilometerIndicator": "",
It was trying to substring a null(?). I just wrapped it up in a MONITOR to clear the resulting field when an error occurs. Now it is humming along.

Sorry I wasted your time with this. I do appreciate you taking a peek...
Post Reply