From: craigs@dekko.com
To: Scott Klement <sk@scottklement.com>
Date: 3 May 2006  14:13 PM
Subject: Re: FTPAPI FTP SSL posting

To be honest, I was struggling to understand the underlying principles of
session switching.  I gave it my best stab and I opted to use global
variables just to get something to work.  In many cases, I was just trying
to get something that works.  For example, on the RecvLine, I checked for
the GSK_WOULD_BLOCK before doing the timeout but not on the other reads.  I
don't know why I did it but it seemed to work just there.  Without it there
would be a timeout.  I figured that I was getting the underlying general
structures created and then it could be improved.  I know you really
understand the program and I highly respect your comments.  I feel like I
got something to work but almost none of it was coded correctly for the
masses.  You can take it or leave it.  Scrap all of it if you want.  The
sole reason I was considering SSL FTPAPI was for using my own application
id.  It turns out IBM development is planning on having application id user
specific in the future.  Then, there is the unpublished API "Update
Certificate Usage" that is very questionable.  I may or may not use what I
have in the short term as it seems JP Morgan Chase may or may not be going
to certificates instead of the VPN.  We are already hogging the FTP
client's SSL with the Wells Fargo bank.  HTTPAPI works for the "put" but
for the functions like "mget", "delete", etc just get seemingly impossible
except through FTP.  Side note: Doing the http_url_get() to the Wells Fargo
https which uses Java script returns "xpgusrf1 Virtual user logged on".  I
was hoping for a list of files or something.  Oh well, the FTP client works
for now and I can let the emergency happen if we have bad luck.

Responses:
a) Just trying to get it to work.  If I understood the switching I would
have done that.  At least I learned the right point to set the passive
default (opened session).
b) I totally agree.  Actually, someone was looking for what I did.  They
were having trouble getting a certificate imported.  Based on their
response, it seemed they didn't trust higher certificate authorities rather
than anything to do with their client.  I pointed them at how they could
look at their certificate path and make sure they trust higher certificate
authorities.  I gave them my version but I gave a huge disclaimer saying
that it should only be used in this particular case whatever that is and
should not be distributed, and it is not guaranteed to work as expected.  I
gave him my version but just set wkPassive *on if wkSslUse was *on and
deleted the whole DftPassive procedure.  If wkPassive were set *on by
default for SSL in the ftp_open() why check for wkSslUse for every pasvcmd?
c) I didn't know.  Okay, probably implement like HTTPAPI.  I went the easy
and less time consuming route which worked best for a newbie.   :) 
d) They should probably all be in the session then.  Global was easy but I
understand how it could be bad for other connections.
e) I fully realize that and I was hoping you would catch that.  I also
moved the "220" check from ftp_login to ftp_open which you may not like.  I
was thinking about making those protection values the default then have a
procedure they could execute like ftp_dataprotect(<'P' or 'C'>) for protect
or clear to override.  There may be a better way and you probably know
best.
f) I thought since I added the new subprocedures I should keep the previous
signature just in general as a new version.  I don't need the *PRV?  Maybe
if you aren't changing the order, just adding, you don't need *PRV?
Unfortunately, I don't work with service programs much.
g) Makes sense to me.  I was thinking like HTTPAPI and I didn't quite
understand sessions.  I understand there is a control session and a data
session but sessions for different profiles I get lost.  I made the "A"
handle the control session and "B" handle the data session.  The important
thing is you understand sessions.  Putting parms on the ftp_open() AND
ftp_conn() for SSL use and application ID make sense to me.  I think
keeping the force protocol separate is good.
h) I didn't know.  Sounds good.

Makes sense.  I hope I didn't push you back rather than help.  I don't have
the time to go through all this code and understand it all but I'm sure you
understand your code.
Thank you for your consideration and patience!

Craig Strong



                                                                           
             Scott Klement                                                 
             <sk@scottklement.                                             
             com>                                                       To 
                                       craigs@dekko.com                    
             05/03/2006 01:14                                           cc 
             PM                                                            
                                                                   Subject 
                                       Re: FTPAPI FTP SSL posting          
                                                                           
                                                                           
Hi Craig,

I looked over your SSL additions to FTPAPI, and I have several concerns:

a) wkSSLuse, wkEnvh, wkSSLAh, wkSSLBh, wkSockA, wkSockB are all outside of
the session structure.  That means that a change to one of them would
affect all of the open SSL sessions, wouldn't it?

If I have two sessions running, the first one non-SSL and the second one
SSL, wouldn't that cause some very nasty problems?  Wouldn't the first
session become unusable as soon as I started the second session?

b) The PassiveDft subprocedure is not what I had in mind at all.  I never
intended to mimic the behavior of IBM's FTP client.  Worse, I CERTAINLY
don't want to use IBM's data area!   IF you like IBM's client so
much, go use it.  Don't make my software mimic theirs.

Instead, all that's needed is to force passive to be used when an SSL
session is established.  Instead of doing this:

       c                   if        wkPassive = *On
       c                   eval      wwSock = pasvcmd(peSocket)
       c                   else
       c                   eval      wwSock = portcmd(peSocket)
       c                   endif

Do this:

       c                   if        wkPassive = *On
       c                               or wkSslUse = *On
       c                   eval      wwSock = pasvcmd(peSocket)
       c                   else
       c                   eval      wwSock = portcmd(peSocket)
       c                   endif


No need for the whole PassiveDft monstrosity.

c) Your code won't compile on pre-V5R1 systems since you call APIs that
didn't exist in V4R5 and earlier.   That's why all of the SSL related code
in HTTPAPI is either in it's own module (which can simply be left off on
an earlier system) or it's conditionally compiled.

You have code checking if wkOsRls < 040200 and that point is moot since
the code won't even compile on V4R2.  You also have code that does
if wkOsRls <= 0405000 / else ..    the problem is the V5R1 code is still
compiled on V4R5, even though it's not called at runtime.  So the V4R5
check is pointless since the program won't compile on V4R5.

Actually, I'm wrong when I say compile... it'll compile, it just won't
bind.  CRTRPGMOD will work, CRTSRVPGM won't.

d) You use a lot of global variables where they're not necessary.
Subprocedures should return values or pass parameters, not set global
variables.  (With the exception of the session stuff -- but that's
pseudo-non-global)

e) You have all of the SSL parameters hard-coded (PBSZ, PROT) hard-coded.

f) You adding an extra *PRV group in the binding source. (This was totally
unnecessary.)

g) The way it's coded, all sessions use the same environment and therefore
the same application ID.  That means you have to unload the activation
group and re-load it in order to switch application profiles and therefore
client certificates.

It's my opinion that this should be session-specific.  Without unloading
the service program, or even closing the first FTP session, I should be
able to create a second session that uses different a different SSL
profile.

Consequently, the "ftps_init" subprocedure won't work.  I suggest that
parameters be added to FTP_open().  An indicator that turns on SSL
support, and another that specifies the SSL application ID.  Both can be
optional parameters.  If the SSL parameter isn't passed, default it to
*OFF.  If the application ID is not passed, use the *SYSTEM store instead
of an application profile.  I don't think we need to be able to force
different SSL protocol versions, do you?  (Does anyone need to do that?)
But if we do, you could export the SSL_force_protocol() subprocedure as
something like FTP_force_ssl_version() rather than adding more parameters
to FTP_open().

h) Instead of adding util_diag() to FTPAPI, you should simply call the
existing DiagMsg() subprocdure.  No reason to maintain two subprocedures
that do the same thing!

Make sense?

---
Scott Klement  http://www.scottklement.com


On Wed, 26 Apr 2006, craigs@dekko.com wrote:

> > Thaks for your response!  Wow!  I didn't realize thousands of shops were
> > using FTPAPI.  Leaving the non-SSL default the way it is makes sense for
> > backward compatibility with so many shops.  I would probably do the same
> > with so many people.   :)   Prompting on the install makes sense to me too.
> > That would kind of serve as documentation so people know which default is
> > being set.  Documentation is my main concern because the mode may not be
> > apparent to newbies (like myself).  It makes sense to me not caring what
> > IBM does.  I guess in that case, it might make the most sense to always
> > default to passive for SSL only (unless prompted on install).  No
checking
> > release levels and a data area.  Well, the release level would still be
> > used for the code making the gskit act as a client >= V5R1.
> >
> > As far as the example ex8urlssl, yes, it should be "SCK_FTPAPI_EX8URLSSL"
> > for APP_ID and in the comments.  I should have been more descriptive.  I
> > realize I just included APP_ID in the D-specs without use because HTTPAPI
> > EXAMPLE3 does the same thing without use so I don't care.  It is
commented
> > out just like example 3.  I also realize ftp.freebsd.org is unsecure and
> > doesn't work with SSL.  That is why it is called EX8URLSSL and uncompiled
> > and not TESTURLSSL and compiled.  I thought that was why you had the "EX"
> > members as ones for examples but should not be compiled and executed.  I
> > could be wrong.  If you find an SSL site that doesn't need certificates,
> > you can put it in there and start the member with "TEST" for people to
run.
> > Maybe I should have called it "<ftp site>".  Again, I don't care, just so
> > the example is there to see and allow copying.
> >
> > Craig Strong
> >
> >
> >
> >             Scott Klement
> >             <sk@scottklement.
> >             com>                                                       To
> >                                       craigs@dekko.com
> >             04/26/2006 01:56                                           cc
> >             PM
> >                                                                   Subject
> >                                       Re: FTPAPI FTP SSL posting
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
>> >> I posted some questions I had about FTP SSL passive connections to
>> >> midrange.com.  I will see the responses I get but FTP with SSL passive
>> >> seems to be recommended (maybe even mandatory) instead of standard FTP
> > from
>> >> what I see in Google searches.  If FTP SSL should never be done with
>> >> standard FTP, I don't see any reason to waste any time trying to program
>> >> it.  Hopefully it is some help to you.
> >
> >
> > Again, the reason that FTPAPI is set up with standard as it's default
> > (rather than passive) is to preserve backward compatibility.  There may
be
> > software packages that use FTPAPI that coded the following:
> >
> >       x = FTP_Conn( whatever );
> >       FTP_get( x: whatever: whatever);
> >       FTP_Close(X);
> >
> > If I changed the default from standard to passive, all of those programs
> > would have to have coding changes made to them.  Remember, there are
> > thousands of shops using FTPAPI, and many of them have been doing so for
> > quite awhile.
> >
> > I've always told them that they can just install the new FTPAPI, and
their
> > programs will not need any sort of changes or recompiles, that it'll
> > always be compatible.
> >
> > I don't want to break that compatibility -- especially for a DEFAULT that
> > can easily be overwritten by a programmer.
> >
> > I have no problem with the default for SSL being different, however.
And,
> > in fact, I have no problem with having the install program ask them
> > whether they prefer standard or passive for a default.  That way, if they
> > do prefer passive, it could be done.
> >
> > It doesn't matter to me at all what IBM did with their FTP client, or
what
> > data areas IBM asks people to create, or whether a majority are using
> > passive or standard.  What matters to me is that backward compatility be
> > retained, and the only way that'll happen is by leaving the default
alone.
> > (or, at least giving the option to leave it alone in the instlal program)
> >
> >
>> >> I was about to say that you could just comment out the passive default
>> >> stuff so SSL defaults to standard too (less confusion).  But, if passive
>> >> is recommended for SSL like it seems, it may be good to leave it at the
>> >> different default than non-SSL.
> >
> > (Shrug) I don't understand why one or the other would be "recommended".
> > Depending on which side of the firewall you're on, one or the other might
> > work or not work.
> >
>> >> Also, so I don't look like I don't pay attention, in the ILE RPG member
>> >> ex8urlssl it still says https_init() instead of ftps_init().  I also
>> >> forgot to change the APP_ID to have ex8urlssl instead of example3 which
>> >> is the constant and in the comments.
> >
> > You didn't use the APP_ID, though, you just included it in the D-specs.
> > and it's 'SCK_HTTPAPI_EXAMPLE3' so you'd want to change both the
'HTTPAPI'
> > and 'EXAMPLE3' parts.
> >
> > What I don't understand about EX8URLSSL is that it connects to
> > ftp.freebsd.org.  I wasn't aware that they allowed SSL connections?  Do
> > they?
> >
> > It'll be awhile before I've completely looked over the code, since I have
> > about 10,000 other things to do, but I'll let you know what I think.
> >
> >

