I'm getting this DATA-INTO error:
========================================
Message . . . . : The document for the DATA-INTO operation does not match
the RPG variable; reason code 5.
Cause . . . . . : While parsing a document for the DATA-INTO operation, the
parser found that the document does not correspond to RPG variable
"advicedoc" and the options do not allow for this. The reason code is 5. The
exact subfield for which the error was detected is
"advicedoc.creditadviceheader". The options are "doc=file case=convert
countprefix=num_". The document name is
/Allison/4WARD/atd_4ward_credit_result_20211118_111555.json; *N indicates
that the document is not an external file. The parser is 'YAJLINTO'. *N
========================================
I've looked for hours and cannot see where the issue is. I know I'm just missing something so I thought I'd let some other eyes have a go at it.
Do you see what I'm missing? I have attached my JSON data that I used with the YAJLGEN command along with the data structure created. I have also edited the created DS to have the valid field lengths and DIM definitions.
DATA-INTO Failing
Re: DATA-INTO Failing
First thing I found was that the size of your work in progress claims record array of 7 was too small. I raised it to 30 and that ran further.
Then I got a decimal conversion error which was apparently caused by a currentlandedcostmarkup value of "" (i.e. an empty string) when you have told RPG that it is a numeric. This can be fixed by using EXPROPTS(*ALWBLANKNUM) on the Ctl-Opt. Be aware though that you may need a compiler and/or run time PTF for this to work depending on how up-to-date your system is. See https://www.ibm.com/support/pages/node/6342819 for details.
You may have other similar problems of size etc. in the future but this particular file runs with these two changes.
Then I got a decimal conversion error which was apparently caused by a currentlandedcostmarkup value of "" (i.e. an empty string) when you have told RPG that it is a numeric. This can be fixed by using EXPROPTS(*ALWBLANKNUM) on the Ctl-Opt. Be aware though that you may need a compiler and/or run time PTF for this to work depending on how up-to-date your system is. See https://www.ibm.com/support/pages/node/6342819 for details.
You may have other similar problems of size etc. in the future but this particular file runs with these two changes.
Re: DATA-INTO Failing
Thanks Jon!!
I don't see the "workinprogressclaimsrecord" issue. I have the DIM defined as 100. Am I looking at the right place?
I think the "markup" field is a bug. Sometimes it has "" and sometimes it has 0. I'm going to get with the other side and see what they say about that. It should be consistent...
I don't see the "workinprogressclaimsrecord" issue. I have the DIM defined as 100. Am I looking at the right place?
Code: Select all
num_WORKINPROGRESSCLAIMSRECORD int(10) inz(0);
dcl-ds WORKINPROGRESSCLAIMSRECORD dim(100);
TRACKINGNUMBER varchar(10) inz('');
REPAIRORDERNUMBER varchar(8) inz('');
CLAIMTYPE varchar(2) inz('');
PRODUCTSERIALNUMBER varchar(20) inz('');
CLAIMNUMBER varchar(20) inz('');
DISPOSITIONCODE varchar(5) inz('');
DISPOSITIONCODEDESCRIPTION varchar(80) inz('');
REQUESTEDDOLLARS packed(9: 2) inz(0);
end-ds; I think the "markup" field is a bug. Sometimes it has "" and sometimes it has 0. I'm going to get with the other side and see what they say about that. It should be consistent...
Re: DATA-INTO Failing
In the version of the RPG you posted the Dim was 7. See here:
Code: Select all
num_WORKINPROGRESSCLAIMSRECORD int(10) inz(0);
dcl-ds WORKINPROGRESSCLAIMSRECORD dim(7);
TRACKINGNUMBER varchar(10) inz('');
REPAIRORDERNUMBER varchar(8) inz('');
Re: DATA-INTO Failing
I needed to change some of my DIM values due to the number of elements being returned. Now I'm getting this error:
============================================================
*RNF0376 30 13 001300 Total length 23473050 of data item REPAIRI... exceeds
16,773,104 bytes.
109 end-ds;
*RNF0501 20 8 000800 Length of character item CREDITA... exceeds 16773104;
length defaults to 16773104.
============================================================
I've set the DIM values pretty much at the bare minimum. Is there a solution for this?:
============================================================
*RNF0376 30 13 001300 Total length 23473050 of data item REPAIRI... exceeds
16,773,104 bytes.
109 end-ds;
*RNF0501 20 8 000800 Length of character item CREDITA... exceeds 16773104;
length defaults to 16773104.
============================================================
I've set the DIM values pretty much at the bare minimum. Is there a solution for this?:
Code: Select all
dcl-ds AdviceDoc qualified;
dcl-ds CREDITADVICEHEADER;
CREDITADVICENUMBER varchar(2) inz('');
DISTRIBUTORCODE varchar(5) inz('');
CREDITADVICEPREPARATIONDATE varchar(24) inz('');
num_REPAIRINGOUTLETRECORD int(10) inz(0);
dcl-ds REPAIRINGOUTLETRECORD dim(150);
REPAIRINGOUTLETBUSINESSPARTNERCODE varchar(10) inz('');
REPAIRINGOUTLETNAME varchar(30) inz('');
REPAIRINGOUTLETADDRESSLINE1 varchar(30) inz('');
REPAIRINGOUTLETCITY varchar(18) inz('');
REPAIRINGOUTLETSTATE varchar(2) inz('');
REPAIRINGOUTLETCOUNTRY varchar(2) inz('');
REPAIRINGOUTLETPOSTALCODE varchar(10) inz('');
KILOMETERINDICATOR varchar(2) inz('');
CURRENTLABORRATE varchar(15) inz('');
CURRENTLABORRATEEFFECTIVEDATE varchar(10) inz('');
CURRENTTRAVELDISTANCEREIMBURSEMENTRATE varchar(15) inz('');
CURRENTTRAVELDISTANCEREIMBURSEMENTRATEEFFECTIVEDATE varchar(10)
INZ(' ');
CURRENTLANDEDCOSTMARKUP packed(10:2) inz(*zero);
CURRENTLANDEDCOSTMARKUPEFFECTIVEDATE varchar(10) inz('');
num_WORKINPROGRESSCLAIMSRECORD int(10) inz(0);
dcl-ds WORKINPROGRESSCLAIMSRECORD dim(40);
TRACKINGNUMBER varchar(10) inz('');
REPAIRORDERNUMBER varchar(8) inz('');
CLAIMTYPE varchar(2) inz('');
PRODUCTSERIALNUMBER varchar(20) inz('');
CLAIMNUMBER varchar(20) inz('');
DISPOSITIONCODE varchar(5) inz('');
DISPOSITIONCODEDESCRIPTION varchar(80) inz('');
REQUESTEDDOLLARS packed(9: 2) inz(0);
end-ds;
num_WORKINPROGRESSETCRECORD int(10) inz(0);
dcl-ds WORKINPROGRESSETCRECORD dim(100);
TRACKINGNUMBER varchar(10) inz('');
REGISTRATIONNUMBER varchar(30) inz('');
PURCHASEORDERNUMBER varchar(15) inz('');
PRODUCTSERIALNUMBER varchar(20) inz('');
ETCPURCHASEDATE varchar(10) inz('');
DISPOSITIONCODE varchar(5) inz('');
DISPOSITIONCODEDESCRIPTION varchar(50) inz('');
ETCPURCHASEPRICE varchar(10) inz('');
end-ds;
num_ETCREGISTRATIONRECORD int(10) inz(0);
dcl-ds ETCREGISTRATIONRECORD dim(10);
TRACKINGNUMBER varchar(10) inz('');
REGISTRATIONNUMBER varchar(30) inz('');
PURCHASEORDERNUMBER varchar(15) inz('');
PRODUCTSERIALNUMBER varchar(20) inz('');
ETCPURCHASEDATE varchar(10) inz('');
MODELNAME varchar(15) inz('');
COVERAGECODE varchar(20) inz('');
COVERAGECODEDESCRIPTION varchar(35) inz('');
EXCHANGERATE varchar(10) inz('');
ETCREGISTRATIONSTATUSCODEDESCRIPTION varchar(35) inz('');
ETCDEBITAMOUNT varchar(10) inz('');
DOCUMENTNUMBER varchar(20) inz('');
end-ds;
num_CLAIMPROCESSEDRECORD int(10) inz(0);
dcl-ds CLAIMPROCESSEDRECORD dim(25);
TRACKINGNUMBER varchar(10) inz('');
REPAIRORDERNUMBER varchar(8) inz('');
PRODUCTSERIALNUMBER varchar(20) inz('');
CLAIMNUMBER varchar(20) inz('');
MODELNUMBER varchar(15) inz('');
REPAIRORDEROPENDATE varchar(10) inz('');
DELIVERYDATE varchar(10) inz('');
PRIMARYFAILEDPARTNAME varchar(25) inz('');
PRIMARYFAILEDPARTNUMBER varchar(20) inz('');
CLAIMSTATUSCODE varchar(3) inz('');
CLAIMSTATUSCODEDESCRIPTION varchar(20) inz('');
CLAIMTYPE varchar(2) inz('');
EXCHANGERATE varchar(10) inz('');
PREVIOUSCREDITAMOUNT packed(10:2) inz(0);
PARTCREDITTOTAL packed(10:2) inz(0);
PARTMARKUPTOTAL packed(10:2) inz(0);
LABORCREDITTOTAL packed(10:2) inz(0);
NETITEMCREDITTOTAL packed(10:2) inz(0);
NETTOTALCREDITDOLLARS packed(10: 2) inz(0);
REQUESTEDTOTALDOLLARS packed(10: 2) inz(0);
DIFFERENCETOTALDOLLARS packed(10: 2) inz(0);
DOCUMENTNUMBER varchar(20) inz('');
num_CLAIMLINEDETAILRECORD int(10) inz(0);
dcl-ds CLAIMLINEDETAILRECORD dim(25);
TRACKINGNUMBER varchar(10) inz('');
CLAIMNUMBER varchar(20) inz('');
APPROVEDQUANTITY packed(10:2) inz(0);
REPLACEDPARTNUMBER varchar(1) inz('');
PARTNAME varchar(1) inz('');
LABOROPERATIONCODE varchar(8) inz('');
PARTLANDEDMARKUP packed(10:2) inz(0);
NETITEMCODE varchar(10) inz('');
PRORATE packed(10:2) inz(0);
TYPE varchar(10) inz('');
GROSSCREDIT packed(10:2) inz(0);
TOTALCREDIT packed(10:2) inz(0);
PROCESSCODE varchar(10) inz('');
PROCESSMESSAGE varchar(80) inz('');
end-ds;
end-ds;
end-ds;
end-ds;
end-ds;
-
Scott Klement
- Site Admin
- Posts: 658
- Joined: Sun Jul 04, 2021 5:12 am
Re: DATA-INTO Failing
I think this message says it all:
RPG has a limit of (just under) 16mb for a single variable. Yours is larger than that.
So you will have a parse the document in pieces if you want to use data-into. Your repairingoutletrecord is almost the entire document, anyway, so it would make sense to process that one element at a time with %handler
Code: Select all
Total length 23473050 of data item REPAIRI... exceeds 16,773,104 bytes.RPG has a limit of (just under) 16mb for a single variable. Yours is larger than that.
So you will have a parse the document in pieces if you want to use data-into. Your repairingoutletrecord is almost the entire document, anyway, so it would make sense to process that one element at a time with %handler
Re: DATA-INTO Failing
Since I already had your code running (and I was bored) I did a quick %Handler version - this should give you an idea of how it has to work.
It is currently set to process just one REPAIRINGOUTLETRECORD at a time - that would allow the individual nested DS to grow significantly and it would all still fit in the 16K limit. But if you wanted to process more than one you would simply have to change the DIM on the procedure interface and then process the data with a "real" index rather than the hardcoded (1) being used right now.
If you need the base data i.e. this bit:
You can either use the raw YAJL APIs and pull the data that way or you could use a separate DATA-INTO instruction with just those fields in the DS and the additional option allowextra=yes.
It is currently set to process just one REPAIRINGOUTLETRECORD at a time - that would allow the individual nested DS to grow significantly and it would all still fit in the 16K limit. But if you wanted to process more than one you would simply have to change the DIM on the procedure interface and then process the data with a "real" index rather than the hardcoded (1) being used right now.
Code: Select all
**free
ctl-opt dftactgrp(*no) EXPROPTS(*ALWBLANKNUM);
dcl-s wait char(1);
dcl-ds REPAIRINGOUTLETRECORD_T Qualified Template ;
REPAIRINGOUTLETBUSINESSPARTNERCODE varchar(10) inz('');
<snip out definitions>
CLAIMPROCESSEDRECORD varchar(1) dim(1); // empty;
end-ds;
dcl-s totalItems int(10);
dcl-s ifsPathName varchar(5000);
dcl-s options varchar(100);
ifsPathName = '/home/paris/jsonstuff/creditadvice.json';
options = 'doc=file case=convert countprefix=num_ +
path=jsonDoc/CREDITADVICEHEADER/REPAIRINGOUTLETRECORD';
data-into %Handler( processRepairOutletRecord : totalItems )
%DATA( ifsPathname : options )
%PARSER( 'YAJLINTO'
: '{ "document_name": "jsonDoc", +
"number_prefix": "YAJL_" }');
Dsply ( 'Total of ' + %Char(totalItems) + ' items processed' ) ' ' wait;
*inlr = *on;
dcl-proc processRepairOutletRecord;
dcl-pi *n int(10);
count int(10);
item likeDS(REPAIRINGOUTLETRECORD_T) Dim(1) Const;
items int(10) Value;
end-pi;
count += 1;
Dsply (' This item for BP ' + item(1).REPAIRINGOUTLETBUSINESSPARTNERCODE );
Return 0;
end-proc;
Code: Select all
"creditAdviceNumber": "45",
"distributorCode": "03224",
"creditAdvicePreparationDate": "2021-11-18T16:16:22.258Z",
Re: DATA-INTO Failing
Scott... yeah I knew I was in trouble with size when I saw that error. Just did not have a clue what the solution(s) was. Jon has given me something to chew on for a bit. Thanks to both for the input on this. I'm still getting my feet wet but this project is teaching me a bunch!
Re: DATA-INTO Failing
Still a little confused...
In my current program I roll through the REPAIROUTLETRECORD array as follows (condensed version):
I'm not sure how to equate that with the code Jon posted. I ran that code against my current JSON input and it is recognizing it and performing the count but I really don't know how to make that work so I can read through all my other objects/arrays for each business partner.
In my current program I roll through the REPAIROUTLETRECORD array as follows (condensed version):
Code: Select all
for a = 1 to advicedoc.creditadviceheader.num_repairingoutletrecord;
for b = 1 to advicedoc.creditadviceheader.repairingoutletrecord(a).
num_workinprogressclaimsrecord;
clear p_error;
if advicedoc.creditadviceheader.repairingoutletrecord(a).
workinprogressclaimsrecord(b).trackingnumber = *blanks;
iter;
endif;
exsr $chk_track#;
if rec_count = *zero;
iter;
endif;
endfor;
endfor;
Re: DATA-INTO Failing
Basically all you need to do is to move that code (slightly simplified) inside the subprocedure that I showed.
Right now your code is iterating through the repairingoutletrecord array. The %Handler process will do that for you. It iterates through the JSON so that you only get one repairingoutletrecord at a time.
I was going to just show your code inside my subproc but I don't know why you are testing for blanks**. This next bit (unlike my earlier code) is NOT tested but you should get the idea of what needs to go in the subproc.
** Other than in exceptional cases you should never have have to worry about blanks in the array entries. Sure the _actual_ RPG array entries will be left blank - but you will be using the num_ fields to control the loops - test for blanks and iter should never be needed as far as I can see.
Right now your code is iterating through the repairingoutletrecord array. The %Handler process will do that for you. It iterates through the JSON so that you only get one repairingoutletrecord at a time.
I was going to just show your code inside my subproc but I don't know why you are testing for blanks**. This next bit (unlike my earlier code) is NOT tested but you should get the idea of what needs to go in the subproc.
Code: Select all
dcl-proc processRepairOutletRecord;
...
for i = 1 to num_repairingoutletrecord;
// process item(1).repairingOutletRecord(i).REPAIRINGOUTLETBUSINESSPARTNERCODE etc.
for j = 1 to item(1).repairingOutletRecord(i).num_WORKINPROGRESSCLAIMSRECORD;
// process item(1).repairingOutletRecord(i).WORKINPROGRESSCLAIMSRECORD(j).TRACKINGNUMBER etc.
endfor;
// Process the other nested arrays and other data.
endfor;