HTTP 500 Error - HTTPAPI vs POSTMAN

Discussions related to HTTPAPI (An HTTP Client Package for RPG programming.) http://www.scottklement.com/httpapi/
Post Reply
jvondolteren
Posts: 4
Joined: Tue Mar 29, 2022 3:00 pm

HTTP 500 Error - HTTPAPI vs POSTMAN

Post by jvondolteren »

We are attempting to call UKG ( Kronos Software ) Import API to post Wage Change Information for an employee. We have been able to successfully test the API via POSTMAN API tester. However, when we attempt to use a different API tester and/or call the API from within an RPGLE program using HTTPAPI, we are not successful and receive a 500 error response from UKG. Unfortunately their resources are unable to support and resolve our issue. From their perspective, if the API call works in POSTMAN then they have no further technical obligation.

I am including images of both POSTMAN and REQBIN requests ( setups and results ). I am also including text from the httpapi debug log when the request is executed from within RPGLE code. We are not sure what is causing the error and how to resolve it. If anyone has thoughts or ideas of what steps we might take next, we would be very grateful if you could share those with us. Also, please let us know if there is additional information we can provide.

Thank you,
Jim Von Dolteren

RPG Snippet:
-----------------
// post body
// ----------
BODY = '--' + %trim( NAME ) + CRLF
+ 'Content-Disposition: form-data; name="File"; filename="' + %trim( NAME ) + '.csv"' + CRLF
+ 'Content-Type: application/octet-stream' + CRLF + CRLF
+ %trim( DATA_HDR ) + CRLF
+ %trim( DATA_ROW ) + CRLF
+ '--' + %trim( NAME ) + '--' ;

// make http request
// ------------------
RTN_CODE = http_req( 'POST'
: %trim( API_URL )
: *omit
: RESPONSE
: *omit
: %trim( BODY )
: 'multipart/form-data' ) ;


HTTPAPI Debug Log:
-------------------------
HTTPAPI Ver 1.45 released 2021-09-20
NTLM Ver 1.4.0 released 2014-12-22
OS/400 Ver V7R4M0

New iconv() objects set, PostRem=1208. PostLoc=0. ProtRem=819. ProtLoc=0
http_persist_open(): entered
http_long_ParseURL(): entered
DNS resolver retrans: 2
DNS resolver retry : 2
DNS resolver options: x'00000136'
DNS default domain: HEARTLANDCOOP.COM
DNS server found: 10.1.244.1
DNS server found: 10.1.254.1
Nagle's algorithm (TCP_NODELAY) disabled.
SNI hostname set to: secure.saashr.com
-------------------------------------------------------------------------------------
Dump of server-side certificate information:
-------------------------------------------------------------------------------------
Cert Validation Code = 0
-----BEGIN CERTIFICATE-----
MIIGozCCBYugAwIBAgIQDCkaxKqdwYOKck+P8a4jGzANBgkqhkiG9w0BAQsFADBP
MQswCQYDVQQGEwJVUzEVMBMGA1UEChMMRGlnaUNlcnQgSW5jMSkwJwYDVQQDEyBE
aWdpQ2VydCBUTFMgUlNBIFNIQTI1NiAyMDIwIENBMTAeFw0yMjAyMTAwMDAwMDBa
Fw0yMzAzMTEyMzU5NTlaMFoxCzAJBgNVBAYTAlVTMRAwDgYDVQQIEwdGbG9yaWRh
MQ8wDQYDVQQHEwZXZXN0b24xETAPBgNVBAoTCFVLRyBJbmMuMRUwEwYDVQQDDAwq
LlNhYVNIUi5jb20wggEiMA0GCSqGSIb3DQEBAQUAA4IBDwAwggEKAoIBAQC+ayIC
eAIW7FbysQQ4zf94xOjkbea+gmzZ0mOOTsoRdcmI4N+hcCdGx+H0A/79Rn3Tboed
gxyfuFi5+QtCsgiLjxCzlYUoJ+bND90j7AF3qwjln0ovO3hpO+fx872Bw4Ea0Zkb
9o8oUIqlp79huEyVC81AtLyFGN3pm3hOy/LawykaC1rVIo780f4RA2aO+/3/WdpV
rgpCbhOyoME5h/7SuItAV/O17B59jOoKNvJ0tTUwSygDOtzV4dxJdEBl65TSzP2/
Bz3tKLoIehwVbxdc7GHNH/gQqljqBKgTR8I79/NGsj7fwR4mevxdtWT30xWQqQIt
W3kZU4BxcaAU05cBAgMBAAGjggNuMIIDajAfBgNVHSMEGDAWgBS3a6LqqKqEjHnq
tNoPmLLFlXa59DAdBgNVHQ4EFgQUnuxvs/O+Ehf09CvncGcb3P6iCDswFwYDVR0R
BBAwDoIMKi5TYWFTSFIuY29tMA4GA1UdDwEB/wQEAwIFoDAdBgNVHSUEFjAUBggr
BgEFBQcDAQYIKwYBBQUHAwIwgY8GA1UdHwSBhzCBhDBAoD6gPIY6aHR0cDovL2Ny
bDMuZGlnaWNlcnQuY29tL0RpZ2lDZXJ0VExTUlNBU0hBMjU2MjAyMENBMS00LmNy
bDBAoD6gPIY6aHR0cDovL2NybDQuZGlnaWNlcnQuY29tL0RpZ2lDZXJ0VExTUlNB
U0hBMjU2MjAyMENBMS00LmNybDA+BgNVHSAENzA1MDMGBmeBDAECAjApMCcGCCsG
AQUFBwIBFhtodHRwOi8vd3d3LmRpZ2ljZXJ0LmNvbS9DUFMwfwYIKwYBBQUHAQEE
czBxMCQGCCsGAQUFBzABhhhodHRwOi8vb2NzcC5kaWdpY2VydC5jb20wSQYIKwYB
BQUHMAKGPWh0dHA6Ly9jYWNlcnRzLmRpZ2ljZXJ0LmNvbS9EaWdpQ2VydFRMU1JT
QVNIQTI1NjIwMjBDQTEtMS5jcnQwDAYDVR0TAQH/BAIwADCCAX0GCisGAQQB1nkC
BAIEggFtBIIBaQFnAHYA6D7Q2j71BjUy51covIlryQPTy9ERa+zraeF3fW0GvW4A
AAF+5YWrbwAABAMARzBFAiAWqLOvabCrUwS8P/KUI9I0gOwG/6Y5pHpuyyshujiP
9gIhAL6KlGE54tmjdu465mrAe3IbGh2iAN8sjky/L9Jso9KDAHUANc8ZG7+xbFe/
D61MbULLu7YnICZR6j/hKu+oA8M71kwAAAF+5YWrqQAABAMARjBEAiAsTFzqxUP/
+XLYg1BvkuniaFyubWkc1iKX22bkTAQSxgIgOm1gpfbFUD+iUbec0HWw5M/2yD8e
Lgi0NSYnZokGCBIAdgCzc3cH4YRQ+GOG1gWp3BEJSnktsWcMC4fc8AMOeTalmgAA
AX7lhavFAAAEAwBHMEUCIAsR7I7+MLtnXvn68M1Lv0RZnaVIRLPjwAaxvSTvTVjj
AiEAzPoDva/sHEuUfUZLU3iOIswN5MPAZA4eEbpzvwj02dIwDQYJKoZIhvcNAQEL
BQADggEBACE9mLEW8fNT5jI10LKjdzcdpgasRNlGMDUqgiMIRbZw8bTjb6aAk1/R
A6VM4IrE5sLFhOiL6fexjnoU4Fycy40yxaXSPzd6qcoMwqOX+pandNkIeOtmP87i
aYSfnPcCBZbD1CbCW/IZpscOkZhNir5BYDOJS0eefS6T9uAPWeEsygJfB2R1sdGp
YR2sex6hlQZp/T7crP68tL+CKbzsDQ9rOV7tBnswQhiDlnD6lcb4P2h05UzO96YI
+KrZluwDnZqgc6/l7bAiL6ppLSAiKORQcDJo4BFJdRcY6YHKsigC/3yKvzl8MFON
4HR1Xbv+tXAzoYzkIrqkAxRkk4TQxLY=
-----END CERTIFICATE-----
Serial Number: 0C:29:1A:C4:AA:9D:C1:83:8A:72:4F:8F:F1:AE:23:1B
Common Name: *.SaaSHR.com
Country: US
State/Province: Florida
Locality: Weston
Org Unit: UKG Inc.
Issuer CN: DigiCert TLS RSA SHA256 2020 CA1
Issuer Country: US
Issuer Org: DigiCert Inc
Version: 3
not before: 20220209180000
Unknown Field: 18:00:00 09-02-2022
not after: 20230311175959
Unknown Field: 17:59:59 11-03-2023
pub key alg: 1.2.840.113549.1.1.1
signature algorithm: 1.2.840.113549.1.1.11
Unknown Field: 0382010F003082010A0282010100BE6B2202780216EC56F2B10438CDFF78C4E8E46DE6BE826CD9D2638E4ECA1175C988E0DFA1702746C7E1F403FEFD467DD36E879D831C9FB858B9F90B42B2088B8F10B395852827E6CD0FDD23EC0177AB08E59F4A2F3B78693BE7F1F3BD81C3811AD1991BF68F28508AA5A7BF61B84C950BCD40B4BC8518DDE99B784ECBF2DAC3291A0B5AD5228EFCD1FE1103668EFBFDFF59DA55AE0A426E13B2A0C13987FED2B88B4057F3B5EC1E7D8CEA0A36F274B535304B28033ADCD5E1DC49744065EB94D2CCFDBF073DED28BA087A1C156F175CEC61CD1FF810AA58EA04A81347C23BF7F346B23EDFC11E267AFC5DB564F7D31590A9022D5B791953807171A014D397010203010001
Unknown Field: 2048
Unknown Field: 8ECF7CA8CC3D557CF642106337C57E3C
Unknown Field: 1.2.840.113549.2.5
Unknown Field: 2F3C6B4A3B5B6D5357E3CEB370C994FE70438670
Unknown Field: 2B527EFEBEF7048981B48CEDC7F7B53573BE40CAEBD511714549BEB02FD46046
Unknown Field: 5
Unknown Field: *.SaaSHR.com
Unknown Field: 0
Unknown Field: 1.3.6.1.5.5.7.3.2
Unknown Field: 1.3.6.1.5.5.7.3.1
Unknown Field: 2.23.140.1.2.2
Unknown Field: http://ocsp.digicert.com

Protocol Used: TLS Version 1.3
http_persist_req(POST) entered.
http_long_ParseURL(): entered
http_long_ParseURL(): entered
do_oper(POST): entered
There are 2 cookies in the cache
cookie=lbSession not sent (wrong path or domain)
POST /ta/rest/v1/import/122?force_upload=true HTTP/1.1
Host: secure.saashr.com
User-Agent: http-api/1.45
Content-Type: multipart/form-data
Content-Length: 351
Cookie: $Version=0; lbSession=232ea538abb2bf440f67ddb104121f0a; $Path=/;
Content-Type: multipart/form-data; boundary=JVONDOLTER20220329143157
Authentication: Bearer eyJhbGciOiJIUzUxMiJ9.eyJleHAiOjE2NDg1ODU5MTgsImlhdCI6MTY0ODU4MjMxOCwic2lkIjoiMTkzNzA2ODk1MDMiLCJhaWQiOiIxMjk1Nzg2Nzk5NiIsImNpZCI6IjY3MTMwNjM2IiwidHlwZSI6InIifQ.5w_mnE63fmhGB7qS_QLKaXneNEDR8vVSqLatvo3lP4gNaceaphzQYtvTu9_At92be8RmUaiqhzPTHAG2z3xphQ


senddoc(): entered
--JVONDOLTER20220329143157
Content-Disposition: form-data; name="File"; filename="JVONDOLTER20220329143157.csv"
Content-Type: application/octet-stream

Employee Id,Start Date,Amount,Amount Period,Hours,Hours Period,Num_PP_In_Year,Change Reason Code
1234,01/10/2022,3299.28,Pay Period,80,Pay Period,26,Merit Increase,
--JVONDOLTER20220329143157--
recvresp(): entered
HTTP/1.1 500
Date: Tue, 29 Mar 2022 19:32:01 GMT
Content-Length: 0
Set-Cookie: lbSession=6321e05bca1d09df22885aa684a79894; secure; samesite=none; path=/
Referrer-Policy: strict-origin
X-Content-Type-Options: nosniff
X-XSS-Protection: 1; mode=block
Set-Cookie: lbSession=232ea538abb2bf440f67ddb104121f0a; secure; samesite=none; path=/
Referrer-Policy: no-referrer
X-Content-Type-Options: nosniff
X-XSS-Protection: 1; mode=block
RequestId: 1052852720132
X-Frame-Options: DENY
Content-Security-Policy: default-src 'none'
Strict-Transport-Security: max-age=86400;
X-CallLimit-Threshold: 250
X-CallLimit-CurrentCalls: 1
Content-Language: en-US
Alt-Svc: clear
Via: 1.1 google, 1.1 google
Alt-Svc: clear


SetError() #13: HTTP/1.1 500
recvresp(): end with 500
recvdoc parms: identity 0
header_load_cookies() entered
cookie_parse() entered
cookie = lbSession=6321e05bca1d09df22885aa684a79894; secure; samesite=none; path=/
cookie attr lbSession=6321e05bca1d09df22885aa684a79894
cookie attr secure=
cookie attr samesite=none
cookie attr path=/
cookie_parse() entered
cookie = lbSession=232ea538abb2bf440f67ddb104121f0a; secure; samesite=none; path=/
cookie attr lbSession=232ea538abb2bf440f67ddb104121f0a
cookie attr secure=
cookie attr samesite=none
cookie attr path=/
recvdoc(): entered
SetError() #0:
recvdoc(): Receiving 0 bytes.
recvdoc(): Nothing to receive, exiting...
SetError() #13: HTTP/1.1 500
http_close(): entered
Attachments
REQBIN API Tester - UKG Import API.png
REQBIN API Tester - UKG Import API.png (150.48 KiB) Viewed 3196 times
POSTMAN - UKG Import API.png
POSTMAN - UKG Import API.png (106.95 KiB) Viewed 3196 times
Scott Klement
Site Admin
Posts: 635
Joined: Sun Jul 04, 2021 5:12 am

Re: HTTP 500 Error - HTTPAPI vs POSTMAN

Post by Scott Klement »

Hi Jim,

HTTPAPI contains a multipart form-data encoder, but you aren't using it -- you're building this form yourself by hand. Is there a particular reason for this?

As to why it's failing... it's hard to know for certain because all it's sending is a 500 response, it's not sending any sort of error message.

However, I do see one problem in your code. Your content-type is missing the boundary string. Notice the POSTMAN screenshot has a boundary string in the content-type, but neither your HTTPAPI nor the REQBIN (whatever that is) has this. The two that are missing this piece of data are both failing with 500 responses.
jvondolteren
Posts: 4
Joined: Tue Mar 29, 2022 3:00 pm

Re: HTTP 500 Error - HTTPAPI vs POSTMAN

Post by jvondolteren »

Good afternoon, Scott. Thank your for your prompt rely to our inquiry. In response to the points brought up in your reply:

1) We were not aware of a process within HTTPAPI that provides a multipart form-data encoder. We will have to research this option further. Can you tell us if there is online documentation for this functionality please?

2) The 500 error code is frustrating. We are not able to get more detail / messaging from UKG of what might be causing this issue on their server. They have provided a report that provides vague details for each 500 error encountered from our attempted requests. The report documents date, time, server / hostname, request type and a couple other minor details that do nothing to help us resolve the issue.

3) The boundary string is included in the Content-Type header record for the API request. I did not include the code snippet showing how this is added in program using http_xproc( HTTP_POINT_ADDL_HEADER : %paddr( ADD_HEADERS ) ). The HTTPAPI debug log ( nifty little document ) does show the boundary string which is 'boundary=JVONDOLTER20220329143157'. REQBIN is an online API tester like POSTMAN. We like to use more than one tester. The debug log, and image files were meant to show that we set up the request the same in all testers and in code. We still don't understand why POSTMAN was successful, but we weren't successful via REQBIN or our rpgle coding. Also worth noting is that we are able to successfully call other APIs for UKG with varying methods. The difference here is we are imbedding flat file data within the body of the request.

Again, thank you for taking a look at this. We will experiment with HTTPAPI and its multipart form-data encoder. Perhaps this will be the tool to yield positive results.

Jim Von Dolteren
Scott Klement
Site Admin
Posts: 635
Joined: Sun Jul 04, 2021 5:12 am

Re: HTTP 500 Error - HTTPAPI vs POSTMAN

Post by Scott Klement »

Ohhhh... I missed that entirely. You have this:

Code: Select all

User-Agent: http-api/1.45
Content-Type: multipart/form-data  <-- no boundary
Content-Length: 351
Cookie: $Version=0; lbSession=232ea538abb2bf440f67ddb104121f0a; $Path=/; 
Content-Type: multipart/form-data; boundary=JVONDOLTER20220329143157  <-- with boundary
You can't set the content-type with HTTP_POINT_ADDL_HEADER -- this is not supported. Content-Type is required, and must be handled by HTTPAPI, not your own code. That's why you are ending up with two of them. I'm pretty sure this is a big part of the reason the server is rejecting the request, it is violating HTTP standards.

Please remove the Content-Type from the HTTP_POINT_ADDL_HEADER and pass it directly to http_req

Code: Select all

RTN_CODE = http_req( 'POST'
                   : %trim( API_URL )
                   : *omit
                   : RESPONSE
                   : *omit
                   : %trim( BODY )
                   : 'multipart/form-data; boundary=JVONDOLTER20220329143157' ) ;
jvondolteren
Posts: 4
Joined: Tue Mar 29, 2022 3:00 pm

Re: HTTP 500 Error - HTTPAPI vs POSTMAN

Post by jvondolteren »

Once again, thank you for your observations. This is a learning process for me and I appreciate the guidance of someone much more experienced than I.

I modified code within the program and have begun testing the process once more. The request failed once again, but now the error indicates that length is required -- error 411. This seems to me to be incorrect. A review of the current log indicates that the content length header was passed in the request. I have passed that log onto UKG with an expectation they can tell me why this error is being generated. Here is log excerpt.

Again, thanks.
Jim Von Dolteren

POST /ta/rest/v1/import/122 HTTP/1.1
Host: secure.saashr.com
User-Agent: http-api/1.45
Content-Type: multipart/form-data; boundary=JVONDOLTER20220331090544
Content-Length: 351
Authentication: Bearer eyJhbGciOiJIUzUxMiJ9.eyJleHAiOjE2NDg3MzkxNDQsImlhdCI6MTY0ODczNTU0NCwic2lkIjoiMTkzNzI2NjEzNDgiLCJhaWQiOiIxMjk1Nzg2Nzk5NiIsImNpZCI6IjY3MTMwNjM2IiwidHlwZSI6InIifQ.RkV9PLtkgq8pt-F_jKXVELWmhn1yGXoTF1ORKZ7VEnVMDPmU0li82sG8DKnF4CMXVD70fNHQqW0Kjqc46N39gg

senddoc(): entered
--JVONDOLTER20220331090544
Content-Disposition: form-data; name="File"; filename="JVONDOLTER20220331090544.csv"
Content-Type: application/octet-stream

Employee Id,Start Date,Amount,Amount Period,Hours,Hours Period,Num_PP_In_Year,Change Reason Code
1234,01/10/2022,3299.28,Pay Period,80,Pay Period,26,Merit Increase,
--JVONDOLTER20220331090544--
recvresp(): entered
HTTP/1.0 411 Length Required
Content-Type: text/html; charset=UTF-8
Referrer-Policy: no-referrer
Content-Length: 286
Date: Thu, 31 Mar 2022 14:05:45 GMT
Scott Klement
Site Admin
Posts: 635
Joined: Sun Jul 04, 2021 5:12 am

Re: HTTP 500 Error - HTTPAPI vs POSTMAN

Post by Scott Klement »

So now you have "HTTP/1.0 411 Length Required". That's strange. "411 Length Required" is supposed to be sent when the Content-Length is missing -- but, as you pointed out, I can clearly see it being sent! I have not received that error before.

Furthermore, why is it HTTP/1.0? The request was sent as 1.1, and the previous logs you sent all showed 1.1. Why is it suddenly treating this as a 1.0 request?

Are there any special or unusual characters in the request that are hard to see here in the forum post? For example, unprintable characters in the URL, boundary or bearer token? Or extra spaces?
jvondolteren
Posts: 4
Joined: Tue Mar 29, 2022 3:00 pm

Re: HTTP 500 Error - HTTPAPI vs POSTMAN

Post by jvondolteren »

Good afternoon, Scott.

An interesting observation regarding HTTP/1.1 v HTTP/1.0. The HTTP/1.0 seem to only happen with a response of 411. That being said, I decided to use HTTP_POST_URL to make the API call. One of the parameters passed in this procedure is the size of the data payload. I thought perhaps this is what was 'missing'. And even a blind squirrel finds a nut once in while... the procedure change results in a successful API call. I get back expected return code of 202 along with URL to a status report that gives me information about the import itself.

So, at this point I consider the issue resolved. I want to thank you again for making available the HTTPAPI library and for the time you have spent reviewing my issue and providing your insight. I know without your first response regarding content-type, the API call would still be failing.

Thank you,
Jim Von Dolteren
Scott Klement
Site Admin
Posts: 635
Joined: Sun Jul 04, 2021 5:12 am

Re: HTTP 500 Error - HTTPAPI vs POSTMAN

Post by Scott Klement »

jvondolteren wrote: Thu Mar 31, 2022 9:55 pm I decided to use HTTP_POST_URL to make the API call. One of the parameters passed in this procedure is the size of the data payload.
That's because http_url_post() takes a pointer to the payload, so it has no way to know how many bytes to read from the position that the pointer points to aside from having a separate parameter.

http_req() uses the same underlying routine, but gets its data with a VARCHAR parameter. A VARCHAR parameter contains 4 bytes fo the length of the data followed by X bytes for the actual data... so you are indeed passing the length to http_req(), as well -- the only difference is that RPG is calculating the length for you and putting into the length portion of the VARCHAR.

You can see in the log that http_req() was, indeed, passing the length. Does the log look different for http_url_post()?

My guess is that the problem is somewhere else completely... maybe forcing you to re-code the procedure call (because you switched to a different procedure) forced you to do something that fixed the problem. But, I can't imagine it's because of the length being a separate parameter.
Post Reply