HTTPAPI & UTF-8 strings

[ChristLarsen]

Hi all:

I'm having problems sending utf-8 strings to a rest server using http_req.

I'm not sure if I'm doing it well:

I have a pf with dds like this:

Code: Select all

                                             UNIQUE                               
                 R RTEXT1                                                         
                   LANG           1A                                              
                   NUMBER         5S 0                                            
                   TEXT          50G         CCSID(1200 *NOCONVERT)               
                 K LANG                                                           
                 K NUMBER        

TEXT. contains data in different languages (spanish, engligh, russian, greek, and sometimes data in mixed languages...).

I have been coding something like this:

Code: Select all

chain ('G':1) rtext1;
if not %found;
    clear *all rtext1;
endif;

profile.name = x_text;

data-gen profile %data(jsonVAR) %gen('YAJL/YAJLDTAGEN');

(I'm generating a JSON like this: { "name" : "xxxxx" }. where xxxxx is the information I have in the file).

jsonVAR is defined as a varchar(255) ccsid(*utf8). data-gen and YAJL is doing well. I have used HTTPPUTCLOB with that string, and is working fine. But when I try it with HTTPAPI, something is not working well:

Code: Select all

http_setOption('local-ccsid': '0');
http_setOption('network-ccsid': '1208');
http_setOption('content-type':'application/json');

monitor;
    http_req('PUT':
            profileURL:
            *omit:
            response:
            *omit:
            jsonVAR:
            'application/json; charset=utf-8');
on-error;
    dsply 'Error';
endmon;

What am I doing bad???


Thanks

[Scott Klement]

It looks like you're converting the data to your job's EBCDIC?! Won't that cause problems if you have data in many languages?

You didn't provide the definition of 'profile.name', but you'll need to be sure that this is Unicode, not EBCDIC.

Then, you are translating jsonVAR from *utf8 to EBCDIC, and sending it as local-ccsid 0 (the job's EBCDIC)... so it's translated to EBCDIC, then translated back to 1208 (UTF-8). Won't you lose characters this way? Seems like a bad choice.

[ChristLarsen]

Hi Scott:
I have tried it many ways...

The definition of "profile" is like this:

Code: Select all

dcl-ds profile qualified;
    name varchar(100) ccsid(*utf8);
end-ds;

The JSON is generated OK. I have tried to send it to the server with SQL, and it works perfectly.
The problem is, I think, with the http_setOption. I don't know what to put there. My job ccsid is 284. If I put local-ccsid to 0, I know that HTTPAPI will convert the information I'm sending to UTF8 (because I use "network-ccsid"=1208), but If I put local-ccsid=1208, the server I have errors sending the information to the rest server (error 400).

[Scott Klement]

There are two problems that I see:

1) The parameter on the prototype for http_req() does NOT have ccsid(*utf8), so RPG is automatically converting from UTF-8 to EBCDIC when you pass the parameter to it.

2) You are specifying local-ccsid = 0, which means EBCDIC.

[ChristLarsen]

Ok,

Is there any other procedure in httpapi that can handle utf8 strings in the same way that http_req do?

[Scott Klement]

http_req or http_string would work, but you'd have to use a variable that does not have the CCSID keyword specified on it. HTTP_persist_req() would also work, but is more complex to use.

I will try to find time later today (it is early morning where I live) to make an example.

[Scott Klement]

Here's the example that I built (I had to guess at some parts of how your program worked, but this should give you the idea):

Code: Select all

**free
  ctl-opt dftactgrp(*no) bnddir('HTTPAPI') option(*srcstmt:*nodebugio)
          ccsid(*UCS2: 1200);

  dcl-f RTEXT disk keyed usage(*input);

  /copy httpapi_h

  dcl-ds data qualified;
    jsonVAR varchar(255) ccsid(*utf8) inz('');
    buffer  varchar(255) overlay(jsonVAR);
  end-ds;

  dcl-ds profile qualified;
    name varchar(100) ccsid(*utf8);
  end-ds;

  dcl-s profileURL varchar(1000);
  dcl-s response   varchar(10000:4);
  dcl-s status     int(10);
  dcl-s msg        char(52);

  chain ('G': 1) rtext1;
  if not %found;
    clear *all rtext1;
  endif;

  profile.name = text;
  data-gen profile %data(data.jsonVAR) %gen('YAJL/YAJLDTAGEN');

  // NOTE:
  //   http_req() can accept data in any character set.  However,
  //   if you code a variable as char/varchar with ccsid(*utf8)
  //   RPG will translate it to EBCDIC before passing it to http_req because
  //   it assumes all char parameters are EBCDIC.
  // SOLUTION:
  //   Overlay the Unicode field with a char field in a data structure
  //   and pass the char field.
  // ALSO:
  //   Since the RPG data is already in UTF-8, we must tell HTTPAPI that
  //   our local CCSID is 1208 (UTF-8)

  http_setOption('local-ccsid': '1208');
  http_setOption('network-ccsid': '1208');

  profileURL = 'https://profoundjs.com/run/scottklement/christlarsen/wsapi/testput';

  status = http_req( 'PUT'
                   : profileURL
                   : *omit
                   : response
                   : *omit
                   : data.buffer
                   : 'application/json; charset=utf-8');

  if status <> 1;
    msg = 'Error' + %char(status);
    dsply msg;
    msg = http_error();
    dsply msg;
  endif;

  // You may view https://profoundjs.com/run/scottklement/christlarsen/ to
  // see the name that was received.

  *inlr = *on; 
  

[ChristLarsen]

Thanks Scott.

This works fine!

I could not "put" to your URL, because it gives me an error:

"(GSKit) Access to the key database is not allowed. "

But, I have tested it with my own rest server, and it works!

Thanks again

[Scott Klement]

This is a copy/paste from the README file included with HTTPAPI:

Code: Select all

GRANTING ORDINARY USERS PERMISSION TO RUN SSL APPLICATIONS
---------------------------------------------------------------------
When you get an error using SSL like the following:
  "(GSKit) Access to the key database is not allowed"
This is because the end-user doesn't have authority to the files
in the IFS needed by the Global Secure Toolkit (GSKit) in the
operating system. This is the component in the OS used for SSL/TLS.

To solve that problem, grant authority as follows...
In this example, I'm giving a user named SCOTTK access to the
files.  (Change SCOTTK to the proper userid when you do it)

 CHGAUT OBJ('/') +
        USER(SCOTTK) DTAAUT(*RX)
 CHGAUT OBJ('/QIBM') +
        USER(SCOTTK) DTAAUT(*RX)
 CHGAUT OBJ('/QIBM/UserData') +
        USER(SCOTTK) DTAAUT(*RX)
 CHGAUT OBJ('/QIBM/UserData/ICSS') +
        USER(SCOTTK) DTAAUT(*RX)
 CHGAUT OBJ('/QIBM/UserData/ICSS/CERT') +
        USER(SCOTTK) DTAAUT(*RX)
 CHGAUT OBJ('/QIBM/UserData/ICSS/CERT/SERVER')
        USER(SCOTTK) DTAAUT(*RX)
 CHGAUT OBJ('/QIBM/UserData/ICSS/CERT/SERVER/DEFAULT.KDB')
        USER(SCOTTK) DTAAUT(*R)
 CHGAUT OBJ('/QIBM/UserData/ICSS/CERT/SERVER/DEFAULT.RDB')
        USER(SCOTTK) DTAAUT(*R)

If you wish to give all users access to run SSL programs, then
you should change USER(SCOTTK) to USER(*PUBLIC).  You can also
use an AUTL if you like by specifying AUTL(your-autl) instead
of USER(your-user)

NOTE: It's okay to give extra access. For example, if your
      users currently have *RWX to the IFS root, there's no
      need to change it to *RX, *RWX will also work. The
      above authorities are the minimum levels needed.

NOTE: Adopted authority does not work in the IFS.  Please
      grant permissions by the actual userid, not the adopted
      one.