Page 1 of 1
HTTPAPI & UTF-8 strings
Posted: Wed Dec 08, 2021 11:14 am
by ChristLarsen
Hi all:
I'm having problems sending utf-8 strings to a rest server using http_req.
I'm not sure if I'm doing it well:
I have a pf with dds like this:
Code: Select all
UNIQUE
R RTEXT1
LANG 1A
NUMBER 5S 0
TEXT 50G CCSID(1200 *NOCONVERT)
K LANG
K NUMBER
TEXT. contains data in different languages (spanish, engligh, russian, greek, and sometimes data in mixed languages...).
I have been coding something like this:
Code: Select all
chain ('G':1) rtext1;
if not %found;
clear *all rtext1;
endif;
profile.name = x_text;
data-gen profile %data(jsonVAR) %gen('YAJL/YAJLDTAGEN');
(I'm generating a JSON like this: { "name" : "xxxxx" }. where xxxxx is the information I have in the file).
jsonVAR is defined as a varchar(255) ccsid(*utf8). data-gen and YAJL is doing well. I have used HTTPPUTCLOB with that string, and is working fine. But when I try it with HTTPAPI, something is not working well:
Code: Select all
http_setOption('local-ccsid': '0');
http_setOption('network-ccsid': '1208');
http_setOption('content-type':'application/json');
monitor;
http_req('PUT':
profileURL:
*omit:
response:
*omit:
jsonVAR:
'application/json; charset=utf-8');
on-error;
dsply 'Error';
endmon;
What am I doing bad???
Thanks
Re: HTTPAPI & UTF-8 strings
Posted: Thu Dec 09, 2021 8:48 pm
by Scott Klement
It looks like you're converting the data to your job's EBCDIC?! Won't that cause problems if you have data in many languages?
You didn't provide the definition of 'profile.name', but you'll need to be sure that this is Unicode, not EBCDIC.
Then, you are translating jsonVAR from *utf8 to EBCDIC, and sending it as local-ccsid 0 (the job's EBCDIC)... so it's translated to EBCDIC, then translated back to 1208 (UTF-8). Won't you lose characters this way? Seems like a bad choice.
Re: HTTPAPI & UTF-8 strings
Posted: Fri Dec 10, 2021 8:26 am
by ChristLarsen
Hi Scott:
I have tried it many ways...
The definition of "profile" is like this:
Code: Select all
dcl-ds profile qualified;
name varchar(100) ccsid(*utf8);
end-ds;
The JSON is generated OK. I have tried to send it to the server with SQL, and it works perfectly.
The problem is, I think, with the http_setOption. I don't know what to put there. My job ccsid is 284. If I put local-ccsid to 0, I know that HTTPAPI will convert the information I'm sending to UTF8 (because I use "network-ccsid"=1208), but If I put local-ccsid=1208, the server I have errors sending the information to the rest server (error 400).
Re: HTTPAPI & UTF-8 strings
Posted: Fri Dec 10, 2021 2:23 pm
by Scott Klement
There are two problems that I see:
1) The parameter on the prototype for http_req() does NOT have ccsid(*utf8), so RPG is automatically converting from UTF-8 to EBCDIC when you pass the parameter to it.
2) You are specifying local-ccsid = 0, which means EBCDIC.
Re: HTTPAPI & UTF-8 strings
Posted: Fri Dec 10, 2021 3:24 pm
by ChristLarsen
Ok,
Is there any other procedure in httpapi that can handle utf8 strings in the same way that http_req do?
Re: HTTPAPI & UTF-8 strings
Posted: Fri Dec 10, 2021 3:54 pm
by Scott Klement
http_req or http_string would work, but you'd have to use a variable that does not have the CCSID keyword specified on it. HTTP_persist_req() would also work, but is more complex to use.
I will try to find time later today (it is early morning where I live) to make an example.
Re: HTTPAPI & UTF-8 strings
Posted: Fri Dec 10, 2021 6:16 pm
by Scott Klement
Here's the example that I built (I had to guess at some parts of how your program worked, but this should give you the idea):
Code: Select all
**free
ctl-opt dftactgrp(*no) bnddir('HTTPAPI') option(*srcstmt:*nodebugio)
ccsid(*UCS2: 1200);
dcl-f RTEXT disk keyed usage(*input);
/copy httpapi_h
dcl-ds data qualified;
jsonVAR varchar(255) ccsid(*utf8) inz('');
buffer varchar(255) overlay(jsonVAR);
end-ds;
dcl-ds profile qualified;
name varchar(100) ccsid(*utf8);
end-ds;
dcl-s profileURL varchar(1000);
dcl-s response varchar(10000:4);
dcl-s status int(10);
dcl-s msg char(52);
chain ('G': 1) rtext1;
if not %found;
clear *all rtext1;
endif;
profile.name = text;
data-gen profile %data(data.jsonVAR) %gen('YAJL/YAJLDTAGEN');
// NOTE:
// http_req() can accept data in any character set. However,
// if you code a variable as char/varchar with ccsid(*utf8)
// RPG will translate it to EBCDIC before passing it to http_req because
// it assumes all char parameters are EBCDIC.
// SOLUTION:
// Overlay the Unicode field with a char field in a data structure
// and pass the char field.
// ALSO:
// Since the RPG data is already in UTF-8, we must tell HTTPAPI that
// our local CCSID is 1208 (UTF-8)
http_setOption('local-ccsid': '1208');
http_setOption('network-ccsid': '1208');
profileURL = 'https://profoundjs.com/run/scottklement/christlarsen/wsapi/testput';
status = http_req( 'PUT'
: profileURL
: *omit
: response
: *omit
: data.buffer
: 'application/json; charset=utf-8');
if status <> 1;
msg = 'Error' + %char(status);
dsply msg;
msg = http_error();
dsply msg;
endif;
// You may view https://profoundjs.com/run/scottklement/christlarsen/ to
// see the name that was received.
*inlr = *on;
Re: HTTPAPI & UTF-8 strings
Posted: Sat Dec 11, 2021 11:03 am
by ChristLarsen
Thanks Scott.
This works fine!
I could not "put" to your URL, because it gives me an error:
"(GSKit) Access to the key database is not allowed. "
But, I have tested it with my own rest server, and it works!
Thanks again
Re: HTTPAPI & UTF-8 strings
Posted: Mon Dec 13, 2021 2:41 am
by Scott Klement
This is a copy/paste from the README file included with HTTPAPI:
Code: Select all
GRANTING ORDINARY USERS PERMISSION TO RUN SSL APPLICATIONS
---------------------------------------------------------------------
When you get an error using SSL like the following:
"(GSKit) Access to the key database is not allowed"
This is because the end-user doesn't have authority to the files
in the IFS needed by the Global Secure Toolkit (GSKit) in the
operating system. This is the component in the OS used for SSL/TLS.
To solve that problem, grant authority as follows...
In this example, I'm giving a user named SCOTTK access to the
files. (Change SCOTTK to the proper userid when you do it)
CHGAUT OBJ('/') +
USER(SCOTTK) DTAAUT(*RX)
CHGAUT OBJ('/QIBM') +
USER(SCOTTK) DTAAUT(*RX)
CHGAUT OBJ('/QIBM/UserData') +
USER(SCOTTK) DTAAUT(*RX)
CHGAUT OBJ('/QIBM/UserData/ICSS') +
USER(SCOTTK) DTAAUT(*RX)
CHGAUT OBJ('/QIBM/UserData/ICSS/CERT') +
USER(SCOTTK) DTAAUT(*RX)
CHGAUT OBJ('/QIBM/UserData/ICSS/CERT/SERVER')
USER(SCOTTK) DTAAUT(*RX)
CHGAUT OBJ('/QIBM/UserData/ICSS/CERT/SERVER/DEFAULT.KDB')
USER(SCOTTK) DTAAUT(*R)
CHGAUT OBJ('/QIBM/UserData/ICSS/CERT/SERVER/DEFAULT.RDB')
USER(SCOTTK) DTAAUT(*R)
If you wish to give all users access to run SSL programs, then
you should change USER(SCOTTK) to USER(*PUBLIC). You can also
use an AUTL if you like by specifying AUTL(your-autl) instead
of USER(your-user)
NOTE: It's okay to give extra access. For example, if your
users currently have *RWX to the IFS root, there's no
need to change it to *RX, *RWX will also work. The
above authorities are the minimum levels needed.
NOTE: Adopted authority does not work in the IFS. Please
grant permissions by the actual userid, not the adopted
one.