Hi Scott,
I took your advice and just read the IFS file into memory, convert it and write it back out.
I then used this command to get it to a pf:
CPYFRMIMPF FROMSTMF('/magnek/mms2.txt') TOFILE(MAGNEK/MMS) RCDDLM(*CRLF) FLDDLM(*TAB) DECPNT(*PERIOD) RPLNULLVAL(*FLDDFT)
Here is the test program (everything is copied from statements that Scott has done before):
H DFTACTGRP(*NO) ACTGRP(*new) bnddir('QC2LE')
/copy ifsio_h
/copy iconv_h
D ReportError PR
D peMsg 256a varying const
D options(*nopass)
D CCSID_UTF16 c 1200
D fd1 s 10i 0 inz(-1)
D fd2 s 10i 0 inz(-1)
D info ds likeds(statds)
D len s 10i 0
D F1_size s 10I 0
D memblock1 s 16383c based(p_memblock1)
D memblock2 s 16383c based(p_memblock2) ccsid(1200)
D inpptr s *
D outptr s *
D inpleft s 10u 0
D outleft s 10u 0
D outneed s 10u 0
D outsize s 10u 0
D ifs1 s 96a
D ifs2 s 96a
/free
*inlr = *on;
ifs1 = '/magnek/mms.txt'; //UTF-16le unpacked from zip to IFS
ifs2 = '/magnek/mms2.txt'; //UTF-16 (CCSID(1200) manually created.
// --------------------------------------------
// Open the UTF-16le file: Read only
// --------------------------------------------
fd1 = open( %trimr(ifs1) : O_RDONLY );
if (fd1 = -1);
ReportError();
return;
endif;
//---------------------------------------------
//Open the UTF-16 file (CCSID(1200)
//---------------------------------------------
fd2 = open( %trimr(ifs2)
: O_RDWR + O_LARGEFILE
+ O_TEXTDATA + O_CCSID
+ O_SHARE_NONE
: 0
: CCSID_UTF16);
if (fd2 = -1);
ReportError();
return;
endif;
// --------------------------------------------
// Figure out how big the file is (in bytes)
// and reserve a block of memory to load it
// into.
// --------------------------------------------
if fstat(fd1: info) = -1;
callp close(fd1);
ReportError();
return;
endif;
F1_size = info.st_size;
p_memblock1 = %alloc(F1_size);
p_memblock2 = %alloc(F1_size);
// --------------------------------------------
// Read IFS1 into the block of
// memory.
// --------------------------------------------
lseek(fd1: 0: SEEK_SET);
len = read(fd1: p_memblock1: F1_size);
// --------------------------------------------
// Convert from UTF-16le to UTF-16 (CCSID 1200)
// memory.
// --------------------------------------------
inpptr = p_memblock1;
inpleft = len;
outleft = len;
outptr = p_memblock2;
if QlgTransformUCSData( 050042
: inpptr
: inpleft
: outptr
: outleft
: outneed ) <> 0;
ReportError();
return;
endif;
// Write the result to IFS2 file
callp write(fd2: p_memblock2: len);
callp close(fd1);
callp close(fd2);
dealloc p_memblock1;
dealloc p_memblock2;
return;
/end-free
*+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
* ReportError(): Send an escape message explaining any errors
* that occurred.
*
* This function requires binding directory QC2LE in order
* to access the __errno() function.
*+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
P ReportError B
D ReportError PI
D peMsg 256a varying const
D options(*nopass)
D get_errno PR * ExtProc('__errno')
D errno s 10I 0 based(p_errno)
D QMHSNDPM PR ExtPgm('QMHSNDPM')
D MessageID 7A Const
D QualMsgF 20A Const
D MsgData 32767A Const options(*varsize)
D MsgDtaLen 10I 0 Const
D MsgType 10A Const
D CallStkEnt 10A Const
D CallStkCnt 10I 0 Const
D MessageKey 4A
D ErrorCode 8192A options(*varsize)
D ErrorCode DS qualified
D BytesProv 10I 0 inz(0)
D BytesAvail 10I 0 inz(0)
D MsgKey S 4A
D MsgID s 7A
D Msg s 256a varying inz('')
/free
if %parms >= 1;
MsgID = 'CPF9897';
Msg = peMsg;
else;
p_errno = get_errno();
MsgID = 'CPE' + %editc(%dec(errno:4:0):'X');
endif;
QMHSNDPM( MsgID
: 'QCPFMSG *LIBL'
: Msg
: %len(Msg)
: '*ESCAPE'
: '*PGMBDY'
: 1
: MsgKey
: ErrorCode );
/end-free
P E
________________________________
Fra: ftpapi-bounces@xxxxxxxxxxxxxxxxxxxxxx på vegne av Magne Kofoed
Sendt: ti 18.10.2011 08:39
Til: HTTPAPI and FTPAPI Projects
Emne: SV: SV: Converting from UTF16LE to Ascii
Hi Scott,
this is a tab separated file, not xml.
Best regards,
Magne
________________________________
Fra: ftpapi-bounces@xxxxxxxxxxxxxxxxxxxxxx på vegne av Scott Klement
Sendt: ti 18.10.2011 04:58
Til: ftpapi@xxxxxxxxxxxxxxxxxxxxxx
Emne: Re: SV: Converting from UTF16LE to Ascii
Hi Magne,
Frankly, for what you're doing, I wouldn't write the code this way at
all. That code was intended to read one line at a time, looking for
CRLF type delimiters. It makes sense for a CSV file.
For an XML file, I don't think I'd do that, I'd just read the file into
memory, convert it, and write it back out. You don't care if it's
organized into lines...
On 10/17/2011 3:02 PM, Magne Kofoed wrote:
> Hi,
>
> I changed one stmt (the data variable) in the readlineUTF procedure:
> if pos = 0;
> data += inpbuf.buf;
> inpbuf.pos += %len(inpbuf.buf);
> else;
> len = (pos - inpbuf.pos) + %len(EOL);
> // data += %subst(inpbuf.buf:inpbuf.pos:len);
> data = %subst(inpbuf.buf:inpbuf.pos:len);
> inpbuf.pos += len;
> endif;
>
> Works better now, but there is still one more bug.
> Seems like some data is missing in the beginning of the last data record in every buffer.
> Or the first record after a new read into the buffer that is missing the data.
> Have to debug more...
>
> Mvh
> Magne
>
>
> ________________________________
>
> Fra: ftpapi-bounces@xxxxxxxxxxxxxxxxxxxxxx på vegne av Magne Kofoed
> Sendt: ma 17.10.2011 18:49
> Til: HTTPAPI and FTPAPI Projects
> Emne: SV: Converting from UTF16LE to Ascii
>
>
> Hi Jan,
>
> thanks, yes, I found the example program and use it to write to a pf.
> Scott, thank you for this example program.
>
> But I beleave there is a bug here, sometimes the variable UTFLINE has the whole buffer when the program writes to the pf. I'll try to debug and return to the list if I find out what's wrong.
>
> Best regards,
> Magne
>
>
> ________________________________
>
> Fra: ftpapi-bounces@xxxxxxxxxxxxxxxxxxxxxx på vegne av Jan Grove Vejlstrup
> Sendt: ma 17.10.2011 14:55
> Til: HTTPAPI and FTPAPI Projects
> Emne: Re: Converting from UTF16LE to Ascii
>
>
>
> Hi Magne
>
> Scott Klement wrote a little conversion-program for this.
>
> You can find it here:
>
> http://forums.systeminetwork.com/isnetforums/showthread.php?t=145571
>
>
> Best regards
>
> Jan
> Am 14.10.2011 19:06, schrieb Scott Klement:
>> Hi Magne,
>>
>> The file you have is UTF-16LE, but CCSID 1200 is UTF-16. The "LE"
>> stands for "little endian", whereas CCSID 1200 is "big endian."
>>
>> IBM does not officially support UTF-16LE (CCSID 1202) anywhere on the
>> system as far as I know -- the only IBM-approved way to deal with
>> UTF-16LE is by converting it from UTF-16LE to UTF-16 (CCSID 1200) with
>> the QlgTransformUCSData() API.
>>
>> Once converted to UTF-16 (which is fully supported), you can mark it as
>> CCSID 1200 and deal with it normally.
>>
>>
>> On 10/14/2011 3:57 AM, Magne Kofoed wrote:
>>> Hi Scott,
>>>
>>>
>>> I got a file in Unicode (UTF16LE) and tries to import to a pf.
>>>
>>> I have read a couple of your articles about this and got a few leads
>>> on how to do this, but could not find how to get from Unicode to
>>> ascii.
>>>
>>>
>>> 1. I store the file in IFS, it gets CCSID = 1252. I can view the
>>> file with wrklnk and option 5, but I guess it's still Unicode since
>>> I've just stored it.
>>>
>>> 2. I tried to use these commands with no success:
>>>
>>> CHGATR OB(`/magnek/mms1.txt') ATR(*CCSID) VALUE(1200)
>>>
>>> I can not see any characters with wrklnk and option 5. But wordpad
>>> shows the file ok.
>>>
>>> CHGATR OB(`/magnek/mms1.txt') ATR(*CCSID) VALUE(819)
>>>
>>> Same result as in nbr 1.
>>>
>>> 3. I tried to use CPY fromccsid(1200) toccsid(819). Same result
>>> as in nbr 1.
>>>
>>> 4. I tried to use CPYFRMSTMF and CPYFRMIMPF with no success.
>>>
>>> How can I get this UTF16LE file from the IFS to a PF?
>>>
>>>
>>> Best regards,
>>>
>>> Magne
>>>
>> -----------------------------------------------------------------------
>> This is the FTPAPI mailing list. To unsubscribe, please go to:
>> http://www.scottklement.com/mailman/listinfo/ftpapi
>> -----------------------------------------------------------------------
>>
>>
>
>
>
>
> -----------------------------------------------------------------------
> This is the FTPAPI mailing list. To unsubscribe, please go to:
> http://www.scottklement.com/mailman/listinfo/ftpapi
> -----------------------------------------------------------------------
-----------------------------------------------------------------------
This is the FTPAPI mailing list. To unsubscribe, please go to:
http://www.scottklement.com/mailman/listinfo/ftpapi
-----------------------------------------------------------------------
<<winmail.dat>>
----------------------------------------------------------------------- This is the FTPAPI mailing list. To unsubscribe, please go to: http://www.scottklement.com/mailman/listinfo/ftpapi -----------------------------------------------------------------------