[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
Re: which example can i use to access this webpage
Okay. I've attached an example that I hope will point you in the right
direction.
This type of coding is hard, because this site isn't intended to be
called by a computer program -- it's intended to be called by a web
browser. Accessing a web site (as opposed to a web service) requires
you to have a pretty strong knowledge of how a programmer wrote the web
page. And, figuring out how to read the output is challenging, because
the output is designed to dictate a screen layout, it's not designed to
identify what each field is and what it's for (as would be the case with
a web service.) So what you're looking for is possible, but it's
hard. Not because of the tool, but because the site just wasn't meant
to be used this way.
But, the attached example does work. It's just harder than it would be
if it were a web service.
1) You connect to the initial web page, and it sets cookies that it uses
to identify your browser session. HTTPAPI will manage the cookie for
you -- but make sure you're running version 1.24 or newer, because there
have been bugs fixed recently in the cookie support.
2) You create a web form containing the fields in the <input> tags in
the HTML. Web sites can potentially modify this stuff using JavaScript
on the page, so the <input> tags are a good starting point, but you
shouldn't rely on them 100%. Instead, use a tool like the "Live HTTP
Headers" plugin for Firefox to see exactly what's sent/received, then
copy that in HTTPAPI.
3) After submitting the login form, the site receives your session
cookie and your login credentials (user/pass) and validates them. Once
that's done, it sets your session ID's status (stored in a file on the
server) to "logged in". From here on, you must re-submit the cookie
with each request, or it won't know you're logged in. That's okay,
though, HTTPAPI manages the cookies and resubmits them as long as you're
still running in the same activation group.
4) The server redirects you to a new page buy sending a 302 HTTP
response, and a new URL. Your code can call http_redir_loc to get the
new URL, and one of the http_get routines to follow the redirect.
You'll see that in the sample code. I always like to limit the number
of redirects to prevent the program gettting stuck in a loop if the
redirect points to another redirect, et al.
5) Submit the form containing the zip code query. I coded the program
to take the zip code as a parameter and send it as a query. Again, I
looked at the <input> html tags on the page, and used Live HTTP Headers
to make sure I was sending the right things. The only thing that I made
a variable is the zip code, and you supply it like this:
CALL PGM(MYFIRTEST) PARM(71635) (where 71635 is the zip code)
6) Finally, the response is received (as an HTML document, explaining
how to format data on the browser's screen) containing the list of
foreclosures. I simply displayed the raw HTML on the screen -- I'll
leave it up to you to figure out how to get the data you need out of
that page (by %scan, %subst, etc)
Good luck!
On 5/17/2012 6:00 PM, tim.dclinc@xxxxxxxxx wrote:
The site in question is http://www.myfir.com/myFir/login.asp
you can use tim.dclinc@xxxxxxxxx as user, and "password" as password.
Its a public site which anyone can join...i just wanted to
programmatically "check" the site.
H DFTACTGRP(*NO) ACTGRP('KLEMENT') BNDDIR('HTTPAPI')
/define WEBFORMS
/copy HTTPAPI_H
D MYFIRTEST PR ExtPgm('MYFIRTEST')
D peZipCode 15p 5 const
*ENTRY PLIST-->
D MYFIRTEST PI
D peZipCode 15p 5 const
D QUILNGTX PR ExtPgm('QUILNGTX')
D text 65535a const options(*varsize)
D length 10i 0 const
D msgid 7a const
D qualmsgf 20a const
D errorCode 20i 0 const
D stringReader PR 10i 0
D fd 10i 0 value
D data 65535a options(*varsize)
D len 10i 0 value
D form s like(WEBFORM)
D formData s *
D formDataLen s 10i 0
D url s 1024a varying
D rc s 10i 0
D x s 10i 0
D response s 65535a varying
/free
if %parms < 1;
dsply ('Usage: CALL PGM(MYFIRTEST) PARM(zip-code)');
return;
endif;
http_debug(*on: '/tmp/myfir_debug.txt');
http_use_cookies(*on);
// -------------------------------------------------
// Go to the login page. Even though we don't
// use this page, we must visit it since the
// site sets a cookie at this time.
// -------------------------------------------------
rc = http_url_get_raw( 'http://www.myfir.com/myFir/login.asp'
: 0
: %paddr(stringReader));
if (rc <> 1);
http_crash();
endif;
// -------------------------------------------------
// Create a web form containing login data.
// -------------------------------------------------
form = WEBFORM_open();
WEBFORM_setVar(form: 'loginEmail': 'tim.dclinc@xxxxxxxxx');
WEBFORM_setVar(form: 'loginPassword': 'password');
WEBFORM_setVar(form: 'Log In': 'Log In');
WEBFORM_postData(form: formData: formDataLen);
// -------------------------------------------------
// Send the form to the web site
// -------------------------------------------------
rc = http_url_post_raw(
'http://www.myfir.com+
/myFir/login.asp?checkLogin=true'
: formData
: formDataLen
: 0
: %paddr(stringReader)
: 30
: *omit
: 'application/x-www-form-urlencoded' );
WEBFORM_close(form);
if rc <> 1 and rc<>302;
http_crash();
endif;
// -------------------------------------------------
// Successful login will redirect us to another
// page (the search page). So we'll get a rc=302
// upon success, and must follow the redirect.
// -------------------------------------------------
x = 0;
dow rc=302 and x<5;
x += 1;
url = http_redir_loc();
rc = http_url_get_raw( url: 0: %paddr(stringReader));
enddo;
// -------------------------------------------------
// At this point, we should be logged in
//
// Create a web form containing the form to perform
// a search by zip code.
// -------------------------------------------------
form = WEBFORM_open();
WEBFORM_setVar(form: 'StateSelect' : 'Choose a State');
WEBFORM_setVar(form: 'CitySelect' : 'xxxx');
WEBFORM_setVar(form: 'CountySelect': 'xxxx');
WEBFORM_setVar(form: 'refnumber' : '');
WEBFORM_setVar(form: 'zip' : %char(%int(peZipCode)));
WEBFORM_setVar(form: 'Submit' : 'Begin Search');
WEBFORM_postData(form: formData: formDataLen);
// -------------------------------------------------
// Submit the search form to the server
// -------------------------------------------------
response = '';
rc = http_url_post_raw(
'http://www.myfir.com+
/myFir/search_results.asp'
: formData
: formDataLen
: 0
: %paddr(stringReader)
: 30
: *omit
: 'application/x-www-form-urlencoded' );
WEBFORM_close(form);
if rc <> 1 and rc<>302;
http_crash();
endif;
// the 'response' variable now contains an HTML page that
// was the result of the search...
//
// For the sake of keeping the example simple, I'm just going
// to display it on the screen (interactive job). You will
// need to write code to scan through the HTML to find the bits
// you need...
QUILNGTX( response
: %len(response)
: *blanks
: *blanks
: 0 );
*inlr = *on;
/end-free
//------------------------------------------------------------------
// receive HTTP response document into a string.
//------------------------------------------------------------------
P stringReader B
D PI 10i 0
D fd 10i 0 value
D data 65535a options(*varsize)
D len 10i 0 value
/free
if (len >= 1);
HTTP_xlatep( len: %addr(data): TO_EBCDIC);
response += %subst(data:1:len);
endif;
return len;
/end-free
P E -----------------------------------------------------------------------
This is the FTPAPI mailing list. To unsubscribe, please go to:
http://www.scottklement.com/mailman/listinfo/ftpapi
-----------------------------------------------------------------------